A small Gleam project to continuously fetch, process, and store data from a stock market API, using Redis.
This project is a part of a larger project i'm working on.
Because of API limitations, I needed a way to get all the data I need and store it in an
easy-to-access way.
To get the stock data I need, I must make use of multiple APIs. Some of these have very strict
limitations on rate limits. To get around this I must space out my requests quite a bit.
The data also becomes stale quite fast, as it is stock data, which means that it must be updated frequently.
The solution is to have this service running all the time, continuously making requests every few seconds.
The data then goes through some simple processing before being stored in a Redis database.
Redis is suitable here as I can set TTLs on the data, having it expire after 24 hours, ensuring that the
data in the database is always fresh.
The project is currently not public on GitHub.