Curated List of Data Engineering tools and frameworks
-
Updated
Feb 17, 2017
Curated List of Data Engineering tools and frameworks
Predict stock price based on financial news feeds
This code is used to connect to fixer.io api, Download the currency exchange rates as keeping EUR as base currency. It also provides functions to download the historical data for any date and the average conversion date between 2 dates.
- Soal tes Ujian Bahasa Kita Data Engineer
Blockchain data pipeline using Airflow, Kubernetes, Redshift, and Grafana
AirAware, a distributed data platform for monitoring air quality at arbitrary U.S. locations, developed during the 18B Insight Data Engineering session
data modeling, design and ETL to create sparkify data warehouse on Postgressql
Sparkly data warehouse on AWS Redshift
Quantum Black Hackathon organised by Analytics Vidya
Pipeline validation using Great Expectations library
Compressing data using Python and compression techniques for better data storage and transfer.
Modeling and ETL to load sparkify data on cassandra
Project challenge part of the data engineering festival at HSBC
This is a quick-and-dirty data analytics platform based on Spark, Hadoop and Jupyterhub. All this tools are deployed automatically with docker and docker-compose.
Add a description, image, and links to the dataengineering topic page so that developers can more easily learn about it.
To associate your repository with the dataengineering topic, visit your repo's landing page and select "manage topics."