Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS using a Spotinst AWS MrScaler resource
-
Updated
May 21, 2024
Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS using a Spotinst AWS MrScaler resource
Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS
ETL Data pipeline using aws services
Reposits the resources used in this project: EMR on EC2 Cluster.
Ce projet a pour but de réaliser une extraction de features, suivie d'une PCA sur des données volumineuses à l'aide de Spark dans le cloud.
Experience with time-series analysis and forecasting models, large data sets, model development and visualisation, statistics.
Automate Amazon EMR clusters using Lambda for streamlined and scalable data processing workflows. Unlock the full potential of your data pipeline with LambdaEMR Automator.
A robust data pipeline leveraging Amazon EMR and PySpark, orchestrated seamlessly with Apache Airflow for efficient batch processing
Classwork projects and home works done through Udacity data engineering nano degree
This project demonstrates the use of Amazon Elastic Map Reduce (EMR) for processing large datasets using Apache Spark. It includes a Spark script for ETL (Extract, Transform, Load) operations, AWS command line instructions for setting up and managing the EMR cluster, and a dataset for testing and demonstration purposes.
Elastic Data Factory
Performed business operations using Big data technologies: AWS EMR, AWS RDS (MySQL), Hadoop, Apache Scoop, Apache HBase, MapReduce
Loaded, filtered and visualized Google Ngrams dataset, which was created by Google's research team by analyzing all of the content in Google Books from the 1800s into the 2000s, in a cloud-based distributed computing environment using Hadoop, Spark, and the AWS S3 file system.
Bits of code I use during live demos
Implemented the PageRank algorithm in Hadoop MapReduce framework and Spark.
Used Amazon AWS and PySpark to solve this EDA assignment
Guide: Executing a python script on AWS EMR for big data analysis.
An end-to-end data pipeline for building Data Lake and supporting report using Apache Spark.
Data Engineering Expert Nanodegree - Data Lake on AWS using Spark and S3
Add a description, image, and links to the emr-cluster topic page so that developers can more easily learn about it.
To associate your repository with the emr-cluster topic, visit your repo's landing page and select "manage topics."