Archive of MaRDA Metadata Extractors Schema. See Datatractor Beam, below, for the current repository.

Archive. See Datatractor Yard, below:

A utility library for comparing and synchronizing different datasets.

This pipeline can be used to collect statistical information about all games, distributed through the Steam platform.

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.

This repo explains how ETL can be done in MySQL and PowerBi to generate insights!

Explore the transformative power of data analytics in my portfolio, where Google Analytics and Snowflake converge to provide comprehensive insights. This project leverages advanced ETL techniques and real-time data integration to enhance user engagement and optimize content delivery effectively.

Regtab is a Java library for data extraction from arbitrary tables represented in machine-readable formats

Michigan State University Data Analytics Project 2

AtliQ Grands hotel Data Analysis using Power BI

DocWire SDK: Award-winning modern data processing in C++20. SourceForge Community Choice & Microsoft support. AI-driven processing. Supports nearly 100 data formats, including email boxes and OCR. Boost efficiency in text extraction, web data extraction, data mining, document analysis. Offline processing is possible for security and confidentiality

Natural language processing of job postings in order to gain insight into the data science job market.

A group of python scripts that clean large data sets by removing duplicate data, putting data in correct formats, and removing redundant cells

This project focuses on using sensor data to predict human activity and is based on the ExtraSensory dataset, created by Ph.D. students and staff at the Department of Electrical and Computer Engineering, University of California, San Diego.

SEC Finance Data Engineering - ETL process for SEC Finance data of S&P 500 companies. Jupyter Notebooks to run ETL work flows. The final dataset is hosted in MongoDB Atlas(cloud). The API is written using Python with PyMongo and Flask libraries. The dashboards with charts are hosted in MongoDB Atlas.

This repository comprises the design, implementation, and analysis of a near real-time data warehouse prototype for an electronics business chain, utilising a multi-threaded Extract, Transform, Load (ETL) pipeline leveraging the efficient HYBRIDJOIN algorithm implemented with Java and MySQL on customer sales data.

Data Engineering Project on Supply Chain ETL. Creating a dynamic ADF pipeline to ingest both Full Load and Incremental Load data from SQL Server and then transform these datasets based on medallion architecture using Databricks.

This project provides Inventory Management using Power BI, extremely useful for Warehouse/ In-plant Inventory Managers to effectively control the Inventory levels and also maintain the Service Levels.

This Twitter ETL project is aimed at providing data to support UN SDG number 16. The project is directed at providing data to generate actionable insights to stakeholders; regarding the 2022 Presidential Elections, Police Brutality, and Propagation of Hate Speech on Twitter

This project takes the crowd funding data provided in excel files through Extract Transform and Load (ETL) process and makes it available in a relational database for further usage.