Skip to content

Creating a Dockerized stack environment using MLflow, mysql and Minio to manage the lifecycle of TensorFlow models.

Notifications You must be signed in to change notification settings

amine-akrout/tensorflow-tifecycle-management-with-mlflow

Repository files navigation

Tensorflow models LifeCycle managemenet Actions Status

This repository contains a docker-compose stack with MLflow, mysql, phpmyadmin and Minio. The networking is set up so running containers could communicate.

Quickstart

The easiest way to understand the setup is by diving into it and interacting with it.

1. Clone the repository and create a virtual environment

git clone  https://github.com/amine-akrout/Tensorflow-Lifecycle-management-with-MLFlow.git
cd Tensorflow-Lifecycle-management-with-MLFlow

This will clone the repo

pip install virtualenv

if you don't already have virtualenv installed virtualenv venv to create your new environment (called 'venv' here)

source venv/bin/activate

to enter the virtual environment

pip install -r requirements.txt

to install the requirements in the current environment

2. Lunch the Docker Stack

make sure you have Docker installed

docker-compose build

this will build mlflow_server image

docker-compose up -d

after this, you have all container running so you can start training ML models

Detail Summary

Container Port
MLflow_server 5000
Minio 9000
Mysql 80
phpmyadmin 3306

3. Start Training the ML models

python .\TensorFlow_training\baseline_training.py
 python .\TensorFlow_training\LSTM_training.py
 python .\TensorFlow_training\CNN_training.py
 python .\TensorFlow_training\swivel_training.py
 python .\TensorFlow_training\BERT_training.py

4. Visualize and compare models performances with MLflow UI to pick the best one

Access the MLflow Dashboard: http://localhost:5000

you can also access and query the database : http://localhost:3306

the model artifacts are saved in minio, to access : http://localhost:9000

And finally to visualize tensorboard charts run the following command :

tensorboard --logdir .\TensorFlow_training\logs\fit

the open it in the browser http://localhost:6000

MLflow UI

Mysql database in phpmyadmin

Models artifacts in Minio

tensorboard visualization

Web app demo

demo

About

Creating a Dockerized stack environment using MLflow, mysql and Minio to manage the lifecycle of TensorFlow models.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published