ETL Samples

This repository contains various code samples for ETL tasks written in Ballerina.

Subdirectories

Each subdirectory contains relevant code and instructions for the corresponding task. Explore the subdirectories for detailed code samples and documentation.

Data Extraction (E)

• Extract data from database: Sample code for extracting customer orders from database.
• Extract information from unstructured data: Sample code for extracting required information from unstructured data using AI.
• Extract data from GitHub: Sample code samples for extracting data from GitHub.

Data Transformation (T)

• Remove duplicates: Sample code for deduplication.
• Remove approximate duplicates: Sample code for approximate deduplication.
• Refine data using AI: Sample code for error correction in data.
• Clean and standardize data: Sample code for cleaning and standardizing data.
• Data mapper: Sample code for data mapping between different structures.
• Enrich data: Sample code to enrich data.

Data Loading (L)

• Load data to BigQuery: Sample code for loading data into BigQuery.
• Kafka database ingestion: Sample code for ingesting data from Kafka into a database.
• Load data to Google Sheets: Sample code for loading data into Google Sheets.