awsglue
Here are 14 public repositories matching this topic...
This project focuses on real-time data streaming with Kinesis, using Flink for advanced processing and OpenSearch for analytics. This architecture has succinctly handled the complete lifecycle of data from ingestion to actionable insights, making it a comprehensive solution.
-
Updated
May 9, 2024 - Java
This projects uses ETL (Extract, Transform and Load) pipeline to extract data from Spotify using its API and loads the data to a data source(AWS Athena). The entire pipeline will be built using Amazon Web Services (AWS).
-
Updated
Jul 8, 2023 - Jupyter Notebook
In this project I have used the Trending YouTube Video Statistics data from Kaggle to analyze and prepare it for usage.
-
Updated
Nov 7, 2022
This project demonstrates how you can build downstream data pipeline using dbt in athena
-
Updated
Dec 24, 2022 - Python
Big data and Cloud Deployment
-
Updated
Jan 15, 2024 - Jupyter Notebook
This project showcases a data transformation pipeline utilizing AWS Glue and Amazon Athena to process Spotify data from CSV files. It involves loading, transforming, and storing data in an S3 datawarehouse, enabling seamless querying through Amazon Athena.
-
Updated
Mar 28, 2024 - Python
Projects on Big Data Using Pyspark and AWS
-
Updated
Apr 28, 2023 - Jupyter Notebook
This project offers a robust data pipeline solution designed to efficiently extract, transform, and load (ETL) Reddit data into a Redshift data warehouse. Leveraging a blend of industry-standard tools and services, the pipeline ensures seamless data processing and integration.
-
Updated
Feb 20, 2024 - Python
Transformed YouTube’s raw JSON data to parquet & loaded it in an S3 bucket, used Glue Data Catalog for storing metadata & Athena to query the cleaned data. Developed an ETL process using a Lambda job that would be triggered when raw data is loaded into an S3 bucket, processed, and stored for analytical purposes in an S3 bucket.
-
Updated
Feb 9, 2023 - Python
An End-To-End data pipeline integration from Website Source to analytical dashboard in AWS using Python flask, ML models, DynamoDB and other AWS services.
-
Updated
Mar 7, 2024 - HTML
Improve this page
Add a description, image, and links to the awsglue topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the awsglue topic, visit your repo's landing page and select "manage topics."