Ce projet a pour but de réaliser une extraction de features, suivie d'une PCA sur des données volumineuses à l'aide de Spark dans le cloud.

With online sales gaining popularity, tech companies are exploring ways to improve their sales by analyzing customer behavior and gaining insights about product trends. Furthermore, the websites make it easier for customers to find the products they require without much scavenging.

apache spark

Data Modeling with Spark for a data lake hosted on S3

Implemented the PageRank algorithm in Hadoop MapReduce framework and Spark.

It's just my first repo, feel free to give feedbacks 😁

A robust data pipeline leveraging Amazon EMR and PySpark, orchestrated seamlessly with Apache Airflow for efficient batch processing

The goal of this repo is to analyze Amazon's digital product from different perspectives using AWS EMR.

The goal is to extract the data and gather insights from a real-life data set of an e-commerce company, using BIG Data tools like Hive, Hadoop, AWS etc.

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS using a Spotinst AWS MrScaler resource

Preventing churn is key to improving revenue for Sparkify, a subscription-based company (fictitious). This project is to analyze data from Sparkify to build a model to predict user churn. First, a sample dataset (128MB) was used on a local machine to explore relevant features and develop a working model. Then similar steps were used to develop a…

Parsing the common crawl database using Scala and Spark

Elastic Data Factory

In this project, we attempt to predict customer churn of a popular (not real) music service. We perform data analysis and machine learning model building on a large amount of data using Spark.

Creates a data lake by moving data held in an AWS S3 bucket to another S3 bucket after transforming it into tables based on a star schema.

This repository aims to capture and clean data from the twitter API in order to perform a sentiment analysis on an EMR cluster.

Criação de Esteiras de Deploy com Git Actions para subir uma infraestrutura na AWS com o Terraform fazendo controle da versão. Tecnologias utilizadas: escrita no formato Delta, Lambda Function, Kinesis Streaming, S3, Athena, Glue e EMR.

Reposits the resources used in this project: EMR on EC2 Cluster.