Game trailer

My very first group coding project with a team of 5 people. We designed and developed a 3D Horror Game on Unity in C# within 1 week.

I mainly worked on the interactions with the environment, the fighting gameplay, and mobs' behavior.

You can download the game here.

Unity, Blender, C#

Our robot	Photo of the playground

Building an autonomous robot that moves in a defined environment and moves objects within a team of 11 people in about one year.

We had a fixed camera outside of the game board. I worked on the localization of the robot, the detection of the objects and obstacles from the camera, and the transmission of this information.

OpenCV, Python, Bluetooth

One-week project in collaboration with the Pasteur institute. Study of links between genes from reactions to different stimuli on R.

R Studio, Unsupervised learning, PCA, Joint graphical lasso

Visualization of geo data

Ranked 2nd over 72 in a course competition on tabular data competition.

Tabular data, XGBoost, Feature engineering, Light GBM, CatBoost, Random forest

Here are some images:

Generated cats

This was my very first Deep learning personal project. The goal was to generate cat images. I got a dataset from the internet and I took my very first step in Deep learning.

I started with a simple DCGAN, then a GAN with residual connexions changed the loss to a Wasserstein loss, and at the end I trained Progressive GAN. I read the papers for Style GAN but did not implement it.

To make it public, I served it on the website of my association with Tensorflow JS. Try it here.

Tensorflow, Keras, Tensorflow JS GAN, Resnet, Progressive GAN, Wasserstein loss, Style GAN

Neural Stryle Transfer Visualization

After GANs, I got hooked on perceptual losses. The idea of designing a "perceptual loss" instead of using a pixel-wise loss was so interesting that I had to implement it.

After implementing the vanilla version of neural style transfer, I wanted to have a quicker method to get stylish images so I implemented fast neural style transfer. It consists in using a generator network to directly transform an image to minimize perceptual loss. I managed to transfer style in real time from my camera.

Tensorflow, Keras, OpenCV Perceptual loss, Neural style transfer, Fast NST, VGG loss

Visualization of the dataset

Winning a competition on imbalanced image classification. This competition was the occasion to apply everything I learned in one year.

My best model was an ensemble of MobileNetv2 nets trained with semi-supervised learning and a lot of regularization (label smoothing, dropout, weight decay).

Here is the repository of my code for more details.

TensorFlow, Keras Imbalanced dataset, Resnet, MobileNetv2, ShuffleNetv2, Few-shot image classification, Semi-supervised learning, Regularization

Visualization of the game

This project solves a maze game only knowing the distance to the exit. When the game begins, the player has to provide a list of moves (right, left, top, right). Then the environment returns the distance to the exit after following the list of moves.

I used a genetic algorithm to solve this game only for the sake of having fun with a genetic algorithm.

The corresponding repository is here

OOP, Genetic algorihtm, Pygame

Creating CI/CD scripts on existing WebApps. Quality checkers, unit tests, automatic deployment.

Workflow, CI/CD, Gitlab, GitHub

Create a simple website with a Node JS Backend, and "handmade" Frontend with HTML and CSS. Encapsulating it with Docker and deploying it on a VM.

VM, Docker, Node JS, HTML, CSS

This project was about detecting and classifying radio signals in the IQ format. The IQ format is a time series of complex numbers, representing two orthogonal components of a radio signal.

A visualization of the Fast Fourier Transform of the signal was enough to convince us that Object detection was a good way to solve that problem.

YOLOv3 achieved 0.95 mAP @ IOU 0.5 on the task.

Pytorch, YOLO, RCNN, Object detection, Unet, Semantic segmentation, signal processing

This project is about lipstick retrieval from a selfie. The approach is to first, find the lips with Face landmarks detection, then crop on these lips and predict the optical properties of the lipstick. From these properties, find the best fit in a database of lipsticks.

I used Dlib out-of-the-box for the face landmarks detection. The regression task was made with a regression CNN. And then the matching was a weighted L2 score on optical properties.

The available data was generated with a GAN. For the colors, I had to move into the LAB space to have a perceptual distance. The client also wanted an estimation of the uncertainty for each prediction so I implemented Deep evidential regression.

Tensorflow, Deep regression, Uncertainty estimation, Dlib

This one is an exploration of everything that can be done to personalize detected objects and add classes to YOLOv5 with the minimum amount of manual annotation.

I did a huge literature review of Few-shot image classification, few-shot object detection, class agnostic detection, open-world object detection, CLIP, Referring expression comprehension.

The research in zero-shot learning got so hot at that time. I designed a solution with a class-agnostic detector and CLIP on top of it. It achieved 0.20 mAP on COCO. However, a few days before the end of my internship, One for all was released, and could do the same better and faster.

Pytorch, Hugging face Few-shot image classification, few-shot object detection, class agnostic detection, open-world object detection, CLIP, Referring expression comprehension

I worked on a project to assist diagnosis of learning differences. The global idea is that we record testing sessions of patients with a camera.

From these videos, we extract all the useful information as time series. And then, we use these time series to understand what happened at what moment because of what. This approach gets rid of high-dimensional video data. At the same time, it makes the global pipeline much more interpretable which is so important in the medical field where mistakes can cost a lot.

Here is a list of key features I worked on:

• Unifying all the data on AWS Storage
• Data cleaning and standardization
• Voice activity detection with Gaussian mixture models
• 3D face landmarks detection with Face alignment nets
• Benchmarking speech-to-text solutions (Whisper, AWS Transcribe, ...)
• Setting up AWS Batch pipelines to optimize feature extraction costs
• Time series visualization with Plotly and Streamlit
• Time series classification and breakpoint detection

AWS, Docker, Pytorch, Steamlit, Plotly

Face landmarks detection, Voice activity detection, Speech-to-text, Time series classification, and breakpoint detection

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (CVPR 2023) was published and a lot of AI avatar apps popped out of nowhere. However, no one could generate qualitative realistic images. We focused on that, and after a few tricks with Dreambooth, we got decent to really good results.

In front of the Eiffel Tower	In front of the Kremlin	In Rome	Professional picture

Automatic 1111, Hugging face, diffusers Dreambooth, Stable diffusion

AWS Dashboard

Turning all tests on Google Colab to production on AWS. Creating dashboards and alerts to monitor errors.

Two production pipelines on AWS:

• Inference with dynamic autoscaling group with warmup. Automatic scaling based on the monitoring of the number of SQS messages.
• Dreambooth fine-tuning with AWS Batch.

AWS, Docker, Fast API

I worked on how to decrease the number of required images for Dreambooth. It became a huge literature review and testing of the latest repositories on Pose Transfer, Instruct pix2pix, pictures light adaptation, Background matting, etc.

Visualization. Prompt: Wearing a red suit at Cannes

A few weeks of work on group photo generation with personalized Dreambooth weights.

Here are some visualizations for the following prompts:

Positive: fantasy themed portrait of {token} with a unicorn horn party hat, vibrant rich colors, pink and blue mist, rainbow, magical atmosphere, drawing by ilya kuvshinov:1.0, Miho Hirano, Makoto Shinkai, Albert Lynch, 2D

Negative: pictures from afar, bad glance, signature, black and white pictures

Positive: digital oil painting of {token} (with a comically large head:1.2), big forehead, (fisheye:1.1), unrealistic proportions, portrait, caricature, closeup, rich vibrant colors, ambient lighting, 4k, HQ, concept art, illustration, ilya kuvshinov, lois van baarle, rossdraws, detailed, trending on artstation

Negative: bad glance, signature, pictures from afar, black and white pictures

CLIP Seg, Background Matting, ControlNet, Dreambooth