-
Updated
Jun 1, 2024 - Jupyter Notebook
azure-data-factory
Here are 131 public repositories matching this topic...
ETL pipeline tailored for Olympics data
-
Updated
Jun 1, 2024 - Python
In this project we are going to create an end-to-end data platform right from Data Ingestion, Data Transformation, Data Loading and Reporting.
-
Updated
May 30, 2024 - Jupyter Notebook
A cutting-edge data project leverages Azure's suite of services to seamlessly transform raw data from GitHub into actionable insights. Using Azure Data Factory for data ingestion, Databricks for PySpark transformations, Synapse Analytics for advanced analysis, and Power BI for intuitive visualization, this project navigates complex data workflows..
-
Updated
May 27, 2024 - Jupyter Notebook
Databricks ETL Pipeline for retrieving and processing NI TestStand test results, featuring a well-documented notebook for ETL operations, Data Lake for storage, Spark SQL+Python for transformations, and Power BI as the final visualization of factory metrics.
-
Updated
May 22, 2024 - Jupyter Notebook
A stand-alone test framework that allows to write unit tests for Data Factory pipelines on Microsoft Fabric and Azure Data Factory.
-
Updated
May 22, 2024 - Python
-
Updated
May 18, 2024 - Jupyter Notebook
Open-source Repository of Useful Scripts and Solutions for Microsoft Data Engineers
-
Updated
May 16, 2024 - Jupyter Notebook
This repository contains code for an end-to-end IoT data pipeline using Azure services. It ingests, processes, and stores IoT device data from AWS S3 to Azure Data Lake Storage and Azure SQL Database, leveraging Azure Data Factory and Azure Functions for seamless integration and automation.
-
Updated
Mar 26, 2024 - Python
Created a movie recommendation system on Azure utilizing Spark SQL by analyzing the MovieLens dataset.
-
Updated
Mar 24, 2024 - Jupyter Notebook
A comprehensive ETL pipeline and sales analysis project leveraging Microsoft Azure and PySpark, designed to optimize e-commerce sales by providing actionable insights through detailed data analysis.
-
Updated
Mar 22, 2024 - Jupyter Notebook
This project demonstrates an ETL pipeline using Microsoft Azure for IMDb Movie Rating Dataset analysis. It covers data extraction from Azure Blob Storage, transformation with Azure Databricks, and loading into Azure SQL using Azure Data Factory. The pipeline automates insights generation and is a practical example of cloud-based data engineering.
-
Updated
Mar 22, 2024 - Jupyter Notebook
This repository contains azure data factory related notes, configurations, best practices & all deliverables such as Pipelies, DataSets, Linked-Services and so on.
-
Updated
Mar 16, 2024
💻 Migration of On-Prem database to Azure cloud, then transformation and reporting of data
-
Updated
Mar 13, 2024 - Jupyter Notebook
Data Engineering & Software Blog
-
Updated
Mar 2, 2024
Tools for deploying Data Factory (v2) in Microsoft Azure
-
Updated
Mar 1, 2024 - PowerShell
Pandemic Analytics with Data Factory
-
Updated
Mar 1, 2024 - PowerShell
We have dataset of IPL from 2008 to 2020 and we have to visualize analytics on Power BI dashboard. We have to upload that dataset into data lake. After that we have to process that data through pipeline and produce modeled data in warehouse. So, that we will be able to analyze the data in Power BI through pre-defined dashboards.
-
Updated
Feb 24, 2024
Terraform module to create data factory resource on AZURE. Azure Data Factory is a cloud-based data integration service provided by Microsoft Azure.
-
Updated
Feb 22, 2024 - HCL
Your guide to navigating the world of Business Intelligence (BI)
-
Updated
Feb 12, 2024 - Shell
Improve this page
Add a description, image, and links to the azure-data-factory topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the azure-data-factory topic, visit your repo's landing page and select "manage topics."