13 posts tagged

AI

20 Websites with a Free Data for Machine Learning

these websites provide free data, as well as an API with a real time data.

FiveThirtyEight Data This site offers data and code related to the articles and graphics at FiveThirtyEight, covering sports, politics, economics, and more. It’s a great resource for data journalists and enthusiasts.

CERN Open Data Portal: This portal provides access to a portion of the data from CERN experiments, with tools, resources, and guidelines to help researchers and students.

NASA’s Earth Observing System Data and Information System: EOSDIS provides a range of Earth observation data from satellites, aircraft, and ground stations, supporting research about the planet’s climate and environment.

NASDAQ Data: Offers various datasets related to the financial market, including historical stock prices, company information, and financial analytics.

Data.gov: The U.S. government’s open data site provides data, tools, and resources to conduct research, develop web and mobile applications, and design data visualizations.

Kaggle Datasets: Features a wide array of datasets for data science and machine learning, with user contributions and regular competitions.

Google Dataset Search: A tool to search for datasets across thousands of online repositories, useful for scholarly and academic research.

UCI Machine Learning Repository: Contains datasets specifically for machine learning, managed by the University of California, Irvine.

GitHub¬†‚Äď Awesome Public Datasets: A¬†comprehensive list of¬†public datasets on¬†GitHub, covering various subjects like government, finance, science, and¬†more.

Yahoo Finance: A comprehensive site offering free financial news, data, and resources. It’s particularly useful for historical and real-time stock data, financial reports, and market analysis.

NewsAPI: This API provides easy access to a range of news articles and headlines from various sources, ideal for aggregating news data for analysis or keeping up with current events.

Quandl: Offers a variety of financial, economic, and alternative data from multiple sources, useful for investment professionals and data analysts.

re3data.org: A global registry of research data repositories across various academic disciplines, offering a comprehensive directory for researchers looking for data.

UK Government Data: Provides a vast range of UK government data, including information on education, health, economics, and more, for public use.

European Data Portal: Offers access to public data resources across Europe, including a wide variety of datasets in different languages and formats.

data.world: A platform for modern data teamwork, providing a space where people can find, share, and collaborate on data.

International Monetary Fund Data: Provides access to macroeconomic statistics and financial data from countries all over the world, maintained by the IMF.

HealthData.gov: Dedicated to making high value health data more accessible to entrepreneurs, researchers, and policymakers in the hopes of better health outcomes for all.

Open Energy Information: A knowledge-sharing online community dedicated to connecting people with the latest in open energy data.

Gapminder: Offers global data on various aspects like health, income, and environment, aiming to promote a fact-based world view.

Nov 14   AI

Speech into text in Telegram

I have developed a @speech_into_text application that transcribes audio into text. In addition to simple transcription, the app offers text editing features to correct mistakes, and it can format the text to suit various purposes such as academic writing, emails, business plans, or glossaries.

I frequently utilize this app to compose articles, create prompts for chatGPT, send messages to friends, and articulate ideas and debates.

You can try this bot for just $1 per hour of audio. This fee covers the costs of server maintenance and API usage. If you find the app beneficial, please share your experiences and recommend it to friends or colleagues. If you have any knowledge or ideas to enhance the application, your feedback would be greatly appreciated.

2023   AI   Telegram

Financial Glossary

Join the Financial Terms Telegram channel and get a new financial term every day.

I created an app for this project that generates the terms and definitions in my style, turns them into an image, and posts it in Telegram at the specified time.

2023   AI   Finance   IT   Telegram

Financial Data Dashboard in Telegram

I created the financial dashboard @financial_data_bot. It is free to use.

Get stock data

/info msft y ma30

/info is a command that activates the get stock data function.

msft is a ticker name of Microsoft. insert any other ticker. Note, for now, not every ticker is available.

y is a date range. You may use w, m, 3m, 6m, y, 3y, or 5y. Also, you may enter specific dates, for example, 01-2021 03-2023 prints the data from January 2021 to March 2023.

ma30 is a monthly average line. it takes the average price of a stock in the last 30 days. You take any days you want: ma40 ma60 ma120 etc.

You can also print more than one line, add one more ma30 /info msft y ma60 ma80 ma240.

Get market data

/market USA 1y

Get the market data with the most popular metrics, such as federal debt, inflation, trade balance, unemployment etc.

Ask about financial stuff

/ask provides a definition of a financial instrument, information about a company, and events.
/ask what is CDS
/ask how to count the option price

price alerts

/alert msft 9:00 12:00 18:00

alert prints the price in the defined time every day, except of weekends

/alert currency 9:00¬†‚Äď prints the¬†eurrub and¬†usdrub prices
/alert brent 9:00¬†‚Äď prints brent oil price

/stop msft¬†‚Äď to¬†stop the¬†alert of¬†a¬†specific stock.

Get summary news (Soon)

to¬†get the¬†summary, I use Openai chatgpt. It is not¬†free¬†‚Äď each stock news analysis costs 50 cents daily. This is because the¬†bot analyses a¬†million words in¬†hundreds of¬†newspapers. If you need this function, get the¬†Openai API and¬†enter it in¬†/openai Your_API_Key, like /openai sk-jfkafjkwhlfhph82p9hi3u2hrbguashg98g3.

The bot will store the summary in the database, and share the news analysis with others

/summary msft d

d is a day. It can also be a week (w) or month (m). You may also choose a specific date, e. g., /summary msft 01-02-2023 03-02-2023.

/summary USA d gets the overall USA market news summary.

Additional data with APIs (Soon)

I want the bot to be free to use, but I have limits on the usage of specific data. For example, only 2 requests about the USD and EUR prices can be done in one minute. the stock news analysis is also very expensive.

Get these 3 APIs to use the bot for free:

  • Alpha vantage¬†‚Äď give the¬†currency prices, free to¬†use for¬†less than 5 queries per¬†minute
  • Fred¬†‚Äď American database of¬†the¬†market data. Requires a¬†2-minute registration
  • Openai¬†‚Äď Ghatgpt machine. Register and¬†get 5$ for¬†free. Get API.

More Features

I want to get real-time stock, bond, and forex prices. They cost a bit. for example, to buy the info on the Moscow Stock Exchange, I have to pay 100$ monthly for each type of security. I can’t afford it, but if there will be enough users, I will create the 1$ subscription, which will give all the features.

Also, I pay 4$ for the server, which is enough to handle 15 requests in one minute. I have to pay 2$ for every 20 additional requests per minute monthly. The database costs 5$ for 15GB. If you like the bot, you may help me to finance it via Koveh Pay

Some bugs with the graphs will be edited soon.

have any ideas, found a bug, wanna promote the project, or join a team, write a comment below or to daniil@koveh.com

2023   AI
2023   AI   Programming

DIY: Summarise Your Speech

I like discussing ideas out loud. I record my thoughts using Voice Memos on my iPhone. I talk for hours but never listen to my records, because it gets boring.

We will make an app, that transforms audio into text. And then, we will make a structured summary. And even a Business Model Canvas of an idea.

Plan

  1. Install Openai Library
  2. Audio to Text
  3. Audio to Summary
  4. Audio to Business Model Canvas
  5. Draw a Business Model Canvas Automatically

Requirements

You need Python and Openai account.

if you don’t have Python or don’t know how to work with libraries on Python, take a look at my previous tutorial about object detection in images, where we installed it on Mac and Windows.

If you have some errors or don’t understand something, ask chatGPT.

Install OpenAI Library

Open ‚ÄúTerminal‚ÄĚ on¬†Mac or¬†in¬†Visual Studio Code. type the¬†following:

pip install openai

The library is used for 3 purposes: ask ChatGPT, translate speech to text, and generate an image. We use ChatGPT and Speech-to-Text functions.

Get the API key

Openai is not free to use. We should pay 0.006$ per each minute of audio. Therefore, we need to register on the Openai website and get the API key, that we insert later in the code. Openai also gives 5$ to the new users.

get the API key after registration.

Record your voice

Use the¬†‚ÄúVoice Memos‚ÄĚ app on¬†Mac or¬†iPhone or¬†any other recording app. Most of¬†the¬†audio file formats are supported by¬†Whisper AI from OpenAI.

Speak for up to 10 minutes in English. Next, move the recording file to the same directory where you will store the Python file. You may also copy the path to the audio file instead of moving it.

Audio to Text

import openai

# insert your API key
openai.api_key = "sk-tIBD5UTbJ9RdqrwjP2dNT3BlfkFJzex4WGqvNsi3uAdMdC1V"

# Define the path to the file. if the file is in the same folder as the Python code, just write the name of a file
path = "audio.m4a"
audio_file= open(path, "rb")

# Select the AI to use (whisper-1 is the newest in April 2023)
transcript = openai.Audio.transcribe("whisper-1", audio_file)

print(transcript.text)

We have translated our speech into text.

Audio to Summary

We use ChatGPT-3 to make a summary. You may use gpt-3.5 that costs 10 times less or gpt-4 for a more precise summary of more than 4000 words.

The¬†maximum amount of¬†words is 4096. The¬†token is a¬†way how the¬†AI stores the¬†data. for¬†English, each token represents nearly one word (700 words are nearly 1000 tokens). For¬†Russian, German, and¬†French¬†‚Äď each token is nearly one letter (700 words are nearly 5000 tokens). Translate the¬†text to¬†English and¬†only then use chatGPT. You will save 5x dollars.

In Prompt we add the translated text and the task that we want an AI to do.

# save the translated text as text
text = transcript.text

prompt = (
        f"Make the summary of the text: \n\n{text}\n\n"
    )

response = openai.Completion.create(
    model="text-davinci-003", 
    prompt=prompt,
    max_tokens=3900 # 4096 minus the constant prompt message
    n=1,
    stop=None,
    temperature=0.5,
)

print(response.choices[0].text.strip())

combine this code with the previous one and run the script. That’s it.

Audio to Business Model Canvas

If I talk about founding a business, why not combine the text in a business model canvas.

prompt = (
    f"Create a business model canvas based on the idea explained below:\n\n{text}\n\n"
    "Use the following structure of 9 blocks:\n"
    "Customer Segments:\n"
    "Value Propositions:\n"
    "Channels:\n"
    "Customer Relationships:\n"
    "Revenue Streams:\n"
    "Key Resources:\n"
    "Key Activities:\n"
    "Key Partners:\n"
    "Cost Structure:"
)

You get the structured answer, and you can insert the data in your business model canvas template.

Draw business model canvas

Let’s make a real business model canvas out of the idea described in the audio. We need to do a simple website with CSS, HTML, JSON, and Python. I prepared it for you:
Download the folder

Install Flask library

pip install flask

Run text_to_bmc.py, audio_to_bmc.py, or bmc_web.py (if you don’t have openai account) within the given folder. Don’t forget to write Openai API key and the audio or text:

Open in the browser: http://127.0.0.1:5000. Or any other page that is written in your terminal

you may also run audio_to_bmc.py or text_to_bmc.py, where the full code is written, just edit the API key and that’s it.

That’s it. You don’t need any longer to think about how to structure the business model canvas.

Challenge

Create the House of Quality model, that tells the difference between you and competitors.

Create the¬†business plan template, which is created automatically based on¬†your recording¬†‚Äď add the¬†SWOT, PESTEL analysis, intro about the¬†product, executive summary, team structure, and¬†maybe competitor and¬†market analysis, using the¬†knowledge from my ‚ÄúDIY‚ÄĚ articles.

Share the¬†tutorial with your friends and¬†colleagues. If you want to¬†see how to¬†record one hour of¬†a¬†speech or¬†team meeting, and¬†how to¬†record in¬†any language and¬†cheaply, then leave a¬†comment. Meanwhile, read other DIY articles, you can find them, by¬†simply typing ‚ÄúDIY‚ÄĚ in¬†the¬†search on¬†the¬†top right. Also, check out the¬†articles about finances and¬†urban planning.

If you want to use a similar app casually, subscribe to my Telegram bot. @ @speech_into_text_bot can listen for up to 10 minutes of speech and transform it into text and summarise everything you said. It costs just 20 cents per 10 minutes of speech. The money is spent on Openai libraries and server maintenance.

2023   AI   DIY   Programming
Earlier Ctrl + ↓