ComfyUI-IDM-VTON

ComfyUI adaptation of IDM-VTON.

⚠️ Work still in progress ⚠️

Installation

⚠️ Current implementation requires GPU with at least 16GB of VRAM ⚠️

Using ComfyUI Manager:

• In ComfyUI Manager, look for ComfyUI-IDM-VTON, and be sure the author is TemryL. Install it.

Manually:

• Clone this repo into custom_nodes folder in ComfyUI and install the dependencies.

cd custom_nodes
git clone https://github.com/TemryL/ComfyUI-IDM-VTON.git
cd ComfyUI-IDM-VTON
pip install -r requirements.txt 

Download weights:

Download the models weights from yisol/IDM-VTON in HuggingFace. The folder structure should be as follow:

models
└── idm_vton
    ├── image_encoder
    │   ├── config.json
    │   └── model.safetensors
    ├── scheduler
    │   └── scheduler_config.json
    ├── text_encoder
    │   ├── config.json
    │   └── model.safetensors
    ├── text_encoder_2
    │   ├── config.json
    │   └── model.safetensors
    ├── tokenizer
    │   ├── merges.txt
    │   ├── special_tokens_map.json
    │   ├── tokenizer_config.json
    │   └── vocab.json
    ├── tokenizer_2
    │   ├── merges.txt
    │   ├── special_tokens_map.json
    │   ├── tokenizer_config.json
    │   └── vocab.json
    ├── unet
    │   ├── config.json
    │   └── diffusion_pytorch_model.bin
    ├── unet_encoder
    │   ├── config.json
    │   └── diffusion_pytorch_model.safetensors
    └── vae
        ├── config.json
        └── diffusion_pytorch_model.safetensors

You can place this folder wherever you want as long as you specify the correct path to models/idm_vton in the Load IDM-VTON Pipeline node. Config files are already provided in this repo. We also provide a shell script to download the weights from HuggingFace:

./scripts/download_weights.sh

Mask Generation

The workflow provided above uses ComfyUI Segment Anything to generate the image mask.

DensePose Estimation

DensePose estimation is performed using ComfyUI's ControlNet Auxiliary Preprocessors.

⭐ Star Us!

If you find this project useful, please consider giving it a star on GitHub. This helps the project to gain visibility and encourages more contributors to join in. Thank you for your support!

Contribute

Thanks for your interest in contributing to the source code! We welcome help from anyone and appreciate every contribution, no matter how small!

If you're ready to contribute, please create a fork, make your changes, commit them, and then submit a pull request for review. We'll consider it for integration into the main code base.

Credits

• ComfyUI
• IDM-VTON
• ComfyUI Segment Anything
• ComfyUI's ControlNet Auxiliary Preprocessors

License

Original IDM-VTON source code under CC BY-NC-SA 4.0 license.