Dual GPU

[Robson1970]

How to run on multiple GPUs ?

[Ferniclestix]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file.
set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui.
I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

[maxchiron]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file. set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui. I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

Won't work now cause main.py would check the init args and reset the CUDA_VISIBLE_DEVICES.
So the working way now is:

python main.py --cuda-device 3

or add the --cuda-device arg to the .bat file's python's line.

[Ferniclestix]

recent update? haven't updated for two days.

[Ferniclestix]

huh, nope, seems to still work, just updated.

[nothingness6]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file. set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui. I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

Hi, could you tell me where the launch.bat file is? I'm using Stability Matrix and can't find it at all.

[nothingness6]

Thanks for the response. I added it and faced the error, RuntimeError: No CUDA GPUs are available

[brknsoul]

What GPU do you have?

[nothingness6]

RTX3060 6GB on laptop.

[brknsoul]

try setting it to 0, instead of 1. Some laptops can be weird.

[nothingness6]

Ok, the error message has gone, but it seems nothing changed. Probably, my laptop is one of the weird ones :(
I'm running the following workflow and it gives me Out of Memory. Could you tell me how to solve this issue, please?

animatediff-v2v-controlnet-openpose-HED-LCM-LoRA-detailer2.json

[h3llolovely]

Edit
ComfyUI/comfy/cli_args.py

Search for "--cuda-device"
Replace "default=None" to "default=1" (or whichever gpu you want to select. i.e 0-4+)

Good to go.

[brknsoul]

rather than editing, couldn't you just add --cuda-device 1 to Extra Launch Arguments at the bottom of Launch Options?

[h3llolovely]

If it works, sure.
I'm on a local install using pinokio.io
The start.json file didn't take that arg for me.

[brknsoul]

for local installs, you typically start ComfyUI with python main.py --args --go --here

[h3llolovely]

I tried that and it didn't work for my install.
If you have a run.bat (I think it's called), you can pass the args there as well.

[unphased]

https://github.com/Stability-AI/StableSwarmUI

[rbychn]

can this run on multiple GPUs at once? can we load any comfyUI workflow onto this?

[unphased]

yes

[rbychn]

just got to know it can only run 1 inference on 1 GPU at any given time, i wanted to speed it up by running 1 job on multiple GPUs but it can't do it unfortunately

[unphased]

Yeah diffusion models can't run efficiently spread across more than one gpu for the generation of a single image. But generating more than one image is a common use case. I would like to imagine that there could also be gains available from multiple gpus when rendering tiles (e.g. ultimate SD upscale). Same for video generation (SVD, AnimateDiff).

[drmbt]

when launching with --cuda-device 1, i see:

Set cuda device to: 1
Total VRAM 24564 MB, total RAM 65186 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync

it still won't run on second card. Anything I'm missing?