This repository contains scripts to train models to classify pictures of Porsche cars. Check out the PowerPoint presentation here.
The following different model types are available:
| 🚗 Model | 📝 Description |
|---|---|
| pre_filter | This model is responsible for identifying whether an image contains a Porsche car or not. It is the first step in the classification process, and if the model determines the image to be a Porsche, it proceeds to the next models for further classification.The model is used for the new architecture in the web_ui. |
| model type | This model predicts 10 broad Porsche car model types, such as 911, Cayman, and Panamera, among others. It is the second step in the classification process and helps to categorize the cars based on popular models. |
| car series | Trained to predict 30 classes, this model groups several years together to imitate Porsche car series like the 911 991 or 911 992. It is a more detailed classification that focuses on car series rather than specific years. |
| all car series | This model classifies images into 88 classes, corresponding to specific Porsche build years. It is the most detailed classification level, predicting not only the car model type but also the specific build year, e.g., a 911 from 2008. This model is suitable for users who want to identify and classify Porsche cars down to the finest level of detail. |
The Web UI is a simple local website that enables users to upload images of Porsche cars and get classification results from the models. The app is built using Eel, which allows Python and HTML/JS to communicate with each other.
Tip
For a free online version of the Web UI, check out PorscheInsight.
With the Web UI, non-car images tend to lead to more random predictions. I trained a model to classify Porsche, other car brands, and others, resulting in a two-step Web UI architecture for Porsche Identification and Classification.
pip install -m build_requirements.txtpython web_app/main.pyor download the executable.
The Web UI employs a two-step process involving two models. Initially, the pre_filter model determines if an image contains a Porsche. If a Porsche is detected, the image proceeds to the second model, which classifies the car according to the user's input.
To see the architecture of the experimental ensemble models or the local web UI, click the arrow below.
Ensemble models
App UI
To improve accuracy and prediction capabilities, I have experimented with two ensemble models. Ensemble models combine predictions from multiple models to give a final verdict, which often results in better prediction accuracy. These ensemble models are still in the experimental phase and can be found in the ensemble.
Note
The ensemble models are experimental and may not always provide accurate predictions. They are still in the testing phase and may be subject to change.
- Car Type Ensemble Model (weighted average)
- Description: Trained two models on the same objective. Each with its on strengths and weaknesses. The results get weighted (correct weights through intensive testing). The models also differ in model architecture.
- Achievements:
- More balanced predictions
- Better prediction accuracy
- Car Type hierarchy with car series (specific car type) (weighted average)
- Description: Created a hierarchy of models. The first models predicts the car type. The second model predicts the car series. After this the results get aligned with the car type model.
- Achievements:
- much less outlier predictions
- better prediction accuracy
They can be found at PorscheInsight-Ensemble.
- Add (explaining) Popup on first start of website
- Retrain all models with better dataset
- Improve pre_filter model/Use Grounded SAM
- Add Taycans to images/models
Completed Tasks
- Add ONNX export
- Implement better Testing
- Implement shap for feature importance
- Add confusion matrix
- Try a deeper/wider or different pretrained model to improve accuracy on the more classes dataset
- Add a (web) app to use the models
- Train on cleaned classes
- Add onnx models to web UI to speedup and reduce size
- Isnet model for bg remove
- Bundle in one Binary
- Add django web app
- Train on cleaned classes with Vision Transformer
- Add Android App
- Implement new Architecture: One model classifies if a car is present and a second model classifies the car
- Implement new architecture to online version
- Add release 1.0.0
- Add docker support (AWS Lambda)
- Display the mask of the predicted car
- Retrain car type model with efficientnet model
- Build an ensemble prediction
- Train a meta-learner
- Try autokeras
- Improve model predictions overall
- Evaluate feature engineering/ More data augmentation
- Build an ensemble model with car_type and car_series
- Experiment with EfficientNet-Lite4
- Compress images on upload
- Python >3.8, <3.11
- [optional] CUDA 11.1
- [optional] cuDNN 8.0.5
Install tensorflow, keras and the other dependencies with pip:
pip install -m requirements.txt| Model | Total params | Trainable params | Non-trainable params | Batch size | Accuracy Train % | Accuracy Val % | Number of classes |
|---|---|---|---|---|---|---|---|
| VGG16 pretrained* | 20,027,082 | 12,390,538 | 7,636,544 | 32 | 99 | 95 | 10 |
| efficientnetv2-b1(new head & faster) | 7,106,956 | 993,416 | 6,113,640 | 32 | 47 | 46 | 88 |
| efficientnetv2-b1 (cleaned classes) | 7,099,474 | 985,934 | 6,113,540 | 32 | 82 | 85 | 30 |
| efficientnetv2-b1-pre-filter | 7,095,991 | 982,451 | 6,113,540 | 32 | 98 | 99 | 3 |
Open to view older models
| Model | Total params | Trainable params | Non-trainable params | Batch size | Accuracy Train % | Accuracy Val % | Number of classes |
|---|---|---|---|---|---|---|---|
| without augmentation* | 11,239,850 | 11,239,850 | 0 | 32 | 98 | 78 | 10 |
| with augmentation* | 11,239,850 | 11,239,850 | 0 | 32 | 79 | 74 | 10 |
| old_pretrained* | 20,027,082 | 5,311,114 | 14,715,968 | 32 | 74 | 72 | 10 |
| VGG16 pretrained | 20,027,082 | 12,390,538 | 7,636,544 | 32 | 80 | 46 | 88 |
| efficientnetv2-b1 | 7,099,474 | 1985,934 | 6,113,540 | 32 | 49 | 46 | 88 |
| vit_b16 (cleaned classes) | 85,901,470 | 102,558 | 85,798,912 | 32 | 45 | 49 | 30 |
| efficientnetv2-b1-pre-filter-refactored | 7,095,991 | 982,451 | 6,113,540 | 32 | 97 | 98 | 3 |
| efficientnetv2-b1-car-type | 7,096,894 | 1,329,386 | 5,767,508 | 32 | 96 | 97 | 10 |
Important
The models with * were trained on the pre cleaned dataset.
Have a look at the releases to see the models and results folder to see the training results. You can have a look at the models structure with Netron or look at the model_structure folder to see the model insights.
Have a look at an example
To train a model you can use the train folder. You can choose the model, the dataset and the number of epochs. You can use the build in Discord Callback to get notfications on Discord after every epoch. You need to change the discord webhook url in the training file.
To predict with a model you can use the test_tf_model.py script. You can choose the model and the image you want to predict. If you want to predict with an onnx model you can use the test_onnx_model.py script.
I recommend to prepare the images with prepare_images.py before. Thus, an error-free and improved prediction is guaranteed.
To explain a model you can use the explainer.py script. You can choose the model and the image(folder) you want to get explanations. I recommend to prepare the images with prepare_images.py before.
After using shap values on the new efficientnet model and the vgg16 model, both on the old head, I found out that the vgg16 model found "better" spots to distinguish between classes, at least sometimes.
The confusion matrix shows the performance of a classification model. It shows the number of correct and incorrect predictions made by a classifier.
To get a confusion matrix of a model you can use the confusion_matrix.py script. You can choose the model and the test set you want to get the confusion matrix from.
Confusion Matrix
The confusion matrix of the specific model variants is too big to show it here. You can find it in the results folder.
Explained images
You can use the convert_to_onnx.py script to convert a keras(.h5) model to ONNX. You can choose the model you want to convert and a save path.
The dataset is from Github and contains ~32.000 pictures of potential Porsche cars. Since the source of the dataset is public the quality and the arrangement of the images was also not great. After the data was cleaned, there are ~30.300 pictures left. Several pictures were removed because they were not of a Porsche car or the picture was not clear enough.
Have a look at the data:
For the training of the pre_filter model a mixture of the porsche-pictures dataset, other Open Source datasets like cifar-10 were used.
Caution
This is not an official Porsche product and is not affiliated with or endorsed by Porsche AG
