StableDiffusion - Textual-Inversion

Stable Diffusion fine-tuned via textual inversion on images from "Canarinho pistola" Brazil's mascot during the 2006 World Cup.

---

This APP loads a pre-trained StableDiffusion model using the Keras framework and fine-tunes it using the Textual Inversion process, you will also find here how to serve StableDiffusion model's components using TFServing, and how to demo it using a Gradio app.

The model's weights are available at HuggingFace Models and you can also try a running version at HuggingFace Spaces

If you want you can also run this repository on Google Colab

Usage

This repository has a collection of Makefile commands that covers all the functionalities provided.

make train

Runs the textual inversion training script, you may want to customize the params.yaml file.

make app

Starts the Gradio app, this version of the Gradio app also loads the model for inference.

make app_serving

Starts the Gradio app, this version of the Gradio app used the TFServing endpoints for inference.

make serve

Starts the TFServing instance to serve the three models from StableDiffusion, you may want to customize the serving_config.config file.

make lint

Applies code linting and formatting.

make test

Runs unit tests.

make jupyter

Starts the JupyterLab instance.

make build

Builds the images for the the Gradio apps and the training feature.

Acknowledgments

This code was heavily inspired by the Teach StableDiffusion new concepts via Textual Inversion Keras code example from Luke Wood.

Disclaimer regarding StableDiffusion

By using this model checkpoint, you acknowledge that its usage is subject to the terms of the CreativeML Open RAIL-M license at https://raw.githubusercontent.com/CompVis/stable-diffusion/main/LICENSE, more information about the model, its usage, and limitations at the HuggingFace mode card.

References

• Keras.io blog Teach StableDiffusion new concepts via Textual Inversion
• Various ways of serving Stable Diffusion keras-sd-serving, Courtesy of my colleagues Chansung and Sayak.
• Paper An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion