Example of how to use Kafka and Spark to handle streaming submissions of urls.
-
Updated
Oct 4, 2021 - Python
Example of how to use Kafka and Spark to handle streaming submissions of urls.
A transformation pipeline for Delta Lake using AWS SDK for Pandas
Spark Structured Streaming application transferring Avro data from Kafka with Schema Registry to Delta Lake
Data pipeline that processes Formula1 data with Azure Databricks, DeltaLake, and Azure Data Factory
Data Streaming with Debezium, Kafka, Spark Streaming, Delta Lake, and MinIO
Example of local pyspark setup including DeltaLake for unit-testing
Distributed Systems - Principles and Paradigms
Коллекция кейсов на базе платформы Databricks
Completed the SQL Basics for Data Science Specialization from the University of California, Davis, gaining proficiency in Data Analysis, SQL, Apache Spark, and Delta Lake.
Building an Azure Data Lake for Bike Share Data Analytics
🚦 Project of Data Warehouse in star architecture based on UK traffic data
A Delta Table pipeline in Rust, triggered by Azure Functions responding to blob storage events in a specific container subfolder. The pipeline processes CSV files, updating or creating Delta Tables as needed, using merges for row changes.
Contains script in Python for Data Pipeline solution on Azure Databricks
deltalake tutorial w/ spark, hive, hadoop
Add a description, image, and links to the delta-lake topic page so that developers can more easily learn about it.
To associate your repository with the delta-lake topic, visit your repo's landing page and select "manage topics."