In the Project Workspace, I'll find a data set containing real messages that were sent during disaster events. I will be creating a machine learning pipeline to categorize these events so that you can send the messages to an appropriate disaster relief agency. This project will include a web app where an emergency worker can input a new message …

This project showcases the implementation of a data pipeline using Apache Airflow. Leveraging the OpenWeather API, it efficiently fetches real-time weather data and performs ETL processing. Results are seamlessly stored in AWS S3 buckets for further analysis. Moreover, the integration of Slack notifications ensures timely alerts to myself.

Personal Apache Beam studies repository

ETL pipeline that extracts their data from S3, stages them in Redshift, and transforms data into a set of dimensional tables for their analytics team to continue finding insights in what songs their users are listening to. Then we will test the database and ETL pipeline by running queries given to us by the analytics team from Sparkify and compa…

This projects showcases a Dimensional Data Warehouse model, utilizing random data from Public APIs to populate the DW. The fact table is transactional based. Data endpoints are available through an Flask based API.

This project involves development of a ETL data pipeline that allows streaming millions of Amharic and Swahili speech audio files and speakers providing transcription texts for data collection in a web platforms.

Personal data pipelines project to document knowledge of open source software and ETL process

ETL using application streaming and creating a Data Lake

Analysis of NYC's citibike data. Technologies: Python , Prefect, dbt, Terraform , Looker data studio

Demographic Data Analyzer project for FreeCodeCamp Data Analyst with Python course.

Learning Data Science with Hacktiv8 using Python, Jupiter Notebook, and else

Reusable Modular ELT(Extract Load Transform) project for the traffic analysis data consisting of a warehouse using Postgresql, DBT(Data Build Tool) and Airflow.

This includes the final projects of my training as a data analyst.

Data Engineering Principles

This is a data engineering project, in which I use Python, MySQL and AWS Services to create and automatically update an online database.

Weather_Difference_Between_SFO/SEA

- Soal tes Ujian Bahasa Kita Data Engineer

Some useful stuff for Data Engineers

Load the Apache Hop (Incubating) community interaction (Mattermost chat, JIRA) to a Neo4j graph database