MSc. Data Engineering Project at Data ScienceTech Institute (DSTI )
-
Updated
Mar 8, 2021 - HTML
MSc. Data Engineering Project at Data ScienceTech Institute (DSTI )
Repositório para armazenar códigos do projeto.
Personal, cloud based (AWS), data lake for experimenting with cloud services.
Solução para buscar tweets com uma determinada “HashTag” e armazená-los em formato Parquet
How to combine smart store and ingest action for datalake use case
An Ansible Role to Configure and setup Hadoop Job Tracker Node.
This script calculates the size of each folder within an Azure Storage container and provides a summary of the folder sizes. The calculated sizes are then exported to a CSV file and displayed in the console for easy reference
Data Engineering Project on Covid19 Reporting – Using Azure Data Factory, Databricks, HDInsight, Azure Data Factory – An End to End ETL pipeline in addition to a Power BI report dashboard.
This project is about building a data lake and creating an ETL pipeline in Spark that loads data from Amazon S3, processes the data into analytics tables, and loads them back into S3
Datalake on AW
Big Data solutions for Sparkify (An online music streaming startup)
End-to-end scenario for Azure data services.
Udacity Data Engineering Nanodegree
Add a description, image, and links to the datalake topic page so that developers can more easily learn about it.
To associate your repository with the datalake topic, visit your repo's landing page and select "manage topics."