stable-diffusion-txt2img

Stable Diffusion - Text-to-Image

Input

Text to render

• Example

a photograph of an astronaut riding a horse

Output

Requirements

This model requires additional module.

pip3 install transformers

Usage

Automatically downloads the onnx and prototxt files on the first run. It is necessary to be connected to the Internet while downloading.

For the sample image,

$ python3 stable-diffusion-txt2img.py

This will save each sample individually as well as a grid of size --n_iter x --n_samples option values.

If you want to specify the input text, put the text after the --input option.
You can use --savepath option to change the name of the output file to save.

$ python3 stable-diffusion-txt2img.py --input TEXT --savepath SAVE_IMAGE_PATH

Quality, sampling speed and diversity are best controlled via the --scale, --steps and --ddim_eta options. Higher values of scale produce better samples at the cost of a reduced output diversity.

Furthermore, increasing --steps generally also gives higher quality samples, but returns are diminishing for values > 250. Fast sampling (i.e. low values of --steps) while retaining good quality can be achieved by using --ddim_eta 0.0.

Change base model

Uses StableDiffusion v1.4 by default. BasilMix and vae-ft-mse can also be used with the command below.

python3 stable-diffusion-txt2img.py --sd basil_mix --vae vae-ft-mse --sampler "DPM++ 2M Kerras" -i "masterpiece, best quality, ultra detailed, sketch, oil painting, 1 girl has silver long hair, eyelashes, jewelry eyes, twinklee eyes, glowing eyes, smile, office lady, tight skirt, looking at viewer, outdoor, street, night" --n_prompt "worst quality, low quality, bad anatomy, bad hands, missing arms, text error, missing fingers, jpeg artifacts, long neck, signature, watermark, blurry, fisheye lens, animal, deformed mutated disfigured, mutated hands, missing hands, extra hands, liquid hands, poorly drawn hands, mutated fingers, bad fingers, extra fingers, liquid fingers, poorly drawn fingers, bad legs, missing legs, extra legs, bad arms, extra arms, long torso, thick thighs, partial head, bad face, partial face, bad eyebrows" --steps 20

Reference

• Stable Diffusion
• k-diffusion

Framework

Pytorch

Model Format

ONNX opset=12

Netron

Diffusion

Legacy multi model

• diffusion_emb.onnx.prototxt
• diffusion_mid.onnx.prototxt
• diffusion_out.onnx.prototxt

New one model

• diffusion.opt.onnx.prototxt

AutoEncoder

• autoencoder.onnx.prototxt