BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

Terraform module to create AWS EMR resources 🇺🇦

Run a Spark job within Amazon EMR

Shell scripts for AWS EMR clusters

EMR + Hadoop to Redshift ELT workflow using spark steps API and orchestrated by Apache-Airflow, which ingests disparate datasets focused around 7Gb of I94 arrivals information to produce a simple star schema in Redshift

Lambda to start EMR and run a map reduce job

Daily Incremental load ETL pipeline for Ecommerce company using AWS Lambda and AWS EMR cluster, Deployed using Apache airflow in a docker container.

Load data from the Million Song Dataset into a final dimensional model stored in S3.

Detect Tight Communities in a social Network

Performing various product review analysis on Amazon dataset using Apache Spark and MongoDB

Credit defaulting results in a large profit loss to banks and other credit lenders. The success of the banking industry results in the ability to understand risk. This project uses big data technologies like Mapreduce, HDFS along with PySpark and AWS for analysis of credit history and its prediction

Data Engineering Projects including Data Modeling, Data Warehouse, Data Lake Development

An opinionated framework for running big data jobs

Data Pipeline Analytics Platform is an end-to-end generic Big Data pipeline. Involves following tech stack: AWS S3, AWS Redshift, AWS EMR Cluster, Apache Spark, Apache Airflow.

Stand-alone Scala & Java tool to anonymize OOXML Documents (DOCX)

AWS EMR backed Spark cluster for analyzing Yelp Data

PySpark RDD and DataFrame Examples

With this app, you can see what programming skills are most in-demand in the current job market.

Example for provisioning AWS EMR service with Terraform