DataLoader worker (pid 12847) is killed by signal: Killed

[ljjcoder]

When I run python flexmatch.py --c ./config/flexmatch/flexmatch_cifar100_400_1.yaml, I always get the following error.
Can you help me to fix the mistake?

[ljjcoder]

[qianlanwyd]

You should use a CPU with a big RAM. May I ask what CPU are you using?

[18445864529]

There's probably an OOM. You can use dmesg -T to verify whether OOM occurred after encountering this error. If that's the case, consider reducing the batch size and num_workers.
The following links may provide some helpful insights into this error.
pytorch/pytorch#5040
pytorch/pytorch#1355

[avihu111]

I also encounter this OOM issue with 32GB RAM for running training on CIFAR10.
How can I optimize the memory consumption?
Edit - when using one GPU the RAM usage is only 4GB. (x8 less than with using 2 GPUs)

[18445864529]

I also encounter this OOM issue with 32GB RAM for running training on CIFAR10. How can I optimize the memory consumption? Edit - when using one GPU the RAM usage is only 4GB. (x8 less than with using 2 GPUs)

Hi, do you mean 32GB RAM of your CPU? I think this error is more a GPU OOM issue (check by dmesg -T whether there is cuda out of memory), could you indicate your GPU memory, the batch size and the num_workers that you were using? And to reduce CPU RAM usage you can try reducing the num_workers.