Monitoring Data Pipeline Design Example
-
Updated
Apr 27, 2023
Monitoring Data Pipeline Design Example
An End-To-End data pipeline integration from Website Source to analytical dashboard in AWS using Python flask, ML models, DynamoDB and other AWS services.
Script to write GFW country stats XLS files for download
This repo includes the code which will help beginners pick up tensorflow dataset and build effective pipelines for their Machine Learning and Deep Learning experiments built for my medium post on the same.
A single-file Dagster project for evaluating fake GitHub stars.
Cyber Attack Prediction Model
Contrail Data Pipeline for OpenStack Clouds
Building a highly scalable data streaming pipeline in python
Udacity's Data Engineering Nanodegree project: Data Pipeline with Airflow.
a micro batch processing pipeline
This project focuses on the Air Pressure system (APS), which creates pressurized air for truck brakes and gear changes. Positive datasets include component failures. APS component. Negative class trucks have non-APS-related component failures.
Modeling tool like DBT to use SQL Alchemy core with a DataFrame interface like
This Project aims to automate the process of infrastructure creation.
This ETL (Extract, Transform, Load) project aims to extract human resources data, clean it using PL/SQL and SQL, integrate it into a Snowflake data warehouse on Azure Cloud using Informatica, and visualize the insights in Power BI.
A topic modeling pipeline using the BERTopic model and state-of-the-art technologies
Build a model that detect toxic comments and different types of toxicity. Using Natural Language Processing and LSTM model. Deploying the model using Gradio app.
Add a description, image, and links to the datapipeline topic page so that developers can more easily learn about it.
To associate your repository with the datapipeline topic, visit your repo's landing page and select "manage topics."