Data Engineering Portfolio

---

Introduction

This repo contains numerous projects that I have worked on using various technologies. Some of the tools the projects utilize are AWS Services, Python, PostgreSQL, Apache Kafka, Various ETL tools, GCP Services, Spark, Snowflake, Airflow, Superset and Tableau.

My background initially started with using more traditional data tools such as Oracle DB, Shell Scripting, Linux/UNIX, and various ETL Tools. Since then it has evolved quite a bit with all the new exciting technologies the data world has to offer. This portfolio will focus primarily on showcasing my projects and work utilizing modern tools & technologies.

Projects

• Tokyo Olympics Project End-To-End Project using Azure and Spark

• YouTube Analysis End-To-End Project using Python and AWS

• Uber Data Analytics - Modern Data Eng GCP Project (Cloud Storage/BigQuery/Mage/Looker)

• Stock Market Kafka - Real Time Streaming Project (AWS/Python/Kafka)

• Instacart Market Basket Analysis - Full End-To-End Project (Snowflake/AWS/Python/Tableau)

• Twitter Data Pipeline Project (Python/AWS/Airflow)