AWS Glue scripts and IAC templates to get started easy. code from AWS Glue Immersion days with Danske Bank 8th+9th of July 2022.

Docker container to create a local environment to develop in AWS glue

a toolkit that provides an object-oriented interface for working with parquet datasets on AWS

Implementation of ETL data pipeline to load data from S3 to snowflake and refresh tableau datasource in AWS

Data lake project for a US based Insurance Company

Intro to streaming data with Kafka, Spark and AWS Glue

The Practical Data Science Specialization brings together these disciplines using purpose-built ML tools in the AWS cloud. It helps you develop the practical skills to effectively deploy your data science projects and overcome challenges at each step of the ML workflow using Amazon SageMaker. This Specialization is designed for data-focused develop

Daily Incremental load ETL pipeline for Ecommerce company using AWS Lambda and AWS EMR cluster, Deployed using Apache airflow in a docker container.

AWS hosted enterprise Data Lake with both batch and realtime data pipelines.

Talks co-organized with PyAmsterdam about ETL/ELT in action in the cloud

A simple shell script to delete multiple tables based on table name prefix.

Data Lakehouse solution for data produced by STEDI Step Trainer sensors and the mobile app so that it can train the machine learning module.

This project is based for legacy applications that works with positional files to process data. The objetive is read these positional files when they arrives in AWS S3, and then send to a dataware-house like AWS Redshift, and finally read the results with a Business Intelligence tool as AWS QuickSight.

Get the dataset intro a S3 bucket, use AWS glue to transform the dataset, write a Lambda script to clean the dataset, query the dataset via AWS Athena then build a dashboard using AWS Quicksight.

Working with Glue Data Catalog and running the using S3 Event Notification and creating the entire stack using AWS CloudFormation

A small walkthrough how to create an AWS Glue Job Pipeline with AWS CDK

This project was completed as part of our Database Management System coursework, for MS in Data Analytics and received recognition as the best project in the class.

Movie Recommendation System using Big Data

ETL job with AWS Glue