This project provides functionality for normalizing and denormalizing the data collected from the xlsx file. All the data is uploaded to SQL Server database.
https://dbdiagram.io/d/Normalization-66381fc05b24a634d090a57e
Reserved Space: 648KB
Used Space: 168KB
Reserved Space: 144KB
Used Space: 32KB
- Clone the repository:
git clone https://github.com/DimaKuriptya/Cars-Normalization.git
- Create a virtual environment:
python3 -m venv venv
- Activate the virtual environment:
source venv/bin/activate
- Install the dependencies:
pip install -r requirements.txt
- Create a file
config.conf
insideconfig
folder. Fill the file by the folowing template:
[db_connection]
server = host.docker.internal
database = normalization
driver = ODBC Driver 18 for SQL Server
username = sa
password = your_password
-
Install the driver if necessary and create an SQL Server database with the same name as in config.
-
Start the containers:
Run airflow-init:
docker-compose up airflow-init
Wait for airflow-init container to finish its job and then run the following command:
docker-compose up
-
Launch the Airflow web UI.
open http://localhost:8080