Build a custom web scraper to collect data on things that you are interested in
For this project I developed a webscraper that build a large database of endpoints from a popular REST API.
This API, taapi.io is a REST API that provides access to a large number of financial technical analysis indicators for crypto, stocks and forex.
The reason that I chose to build this scraper was because I am interested in building a trading bot that uses technical analysis indicators to make trading decisions. Taapi.io offers hundreds of indicators, but the problem is that it is not easy to gather the REST API endpoint for all of these indicators.
Using a scraper, I was able to build a database of all of the endpoints for the indicators that I am interested in, while saving hours of time that would have been spent manually searching for the indicators and compiling the endpoints.
See the output here