Pinned

1. data-normalize-with-etl-procesess data-normalize-with-etl-procesess Public

   I made various data normalization operations with python scripts. Target data in CSV format

   Python 3 1

2. Etl_processing Etl_processing Public

   I find Apache Airflow very useful for ETL work. Here I transferred data from the source database(mysql) to the target database(postresql) and used the Airflow Bash Operator.

   Python 1 1

3. filtering-process filtering-process Public

   You can do a lot of things with Apache Spark. What I've done here is to work with a static file and create a Batch ETL system.

   Python

4. get_users get_users Public

   After ETL done by reading static data, an API is designed with flask_sqlalchemy, the purpose is to show the top five users

   Python

5. random-data-generation random-data-generation Public

   Generating random data with Apache Kafka

   Python

6. Apache-Beam-examples Apache-Beam-examples Public

   liked Apache Beam for streaming data transformations

   Python