Udacity project to develop a data lakehouse solution for sensor data that trains a machine learning model

deltalake tutorial w/ spark, hive, hadoop

Generate calendar & time dimensions for the lakehouse

Distributed Systems - Principles and Paradigms

Learning Apache Spark

This project called "Building an Azure Data Lake for Bike Share Data Analytics" is an assignment from the UDACITY course "Data Engineering with Microsoft Azure".

I have forked this template to implement end to end Machine Learning Life cycle on Databricks Lakehouse

Inspect your lakehouse data by using PyArrow

This is a pyspark pipeline for consume message from kafka and insert into delta table

Formula1 ADF pipeline

minio-based distributed file storage and data lake with FoundationDB(FDB) as meta-data management

Analytical table access method for Postgres

Azure Machine Learning empowers data scientists and developers to build, deploy, and manage high-quality models faster and with confidence. It accelerates time to value with industry-leading machine learning operations (MLOps), open-source interoperability, and integrated tools.

Automatically provision a secure Databricks workspace with ACLs, standard policies, and Financial Services code assets to jumpstart your Lakehouse

The goal of this project is to analyse a dataset (made of CSVs and Jsons files) by using a Data Lakehouse with Snowflake. You will have to upload the data on a cloud storage, ingest the data into the Data Lakehouse, perform data transformation and finally analyse it.

Topic: Social network data processing is based on Google Cloud Platform technology.

🚀 Repositorio de soluciones a retos de comunidades en Pyspark 🐍, Python, y lenguaje {M} de Power Query 🔍 para análisis y consultas de datos.

Create a lakehouse solution in AWS using AWS Glue, AWS Athena, S3, Python, and Spark.