Get stuck

[dream-mjq]

Why is it that when I use v1 algorithm for data processing, I often get stuck in 3D processing stage1. And I always have the same problem. I do hope you can give me some suggestions.

[constantinulrich]

Hey, could you send some error messages?
Likely RAM is not sufficient

[dream-mjq]

there is not error message, just stunk in one case data processing. And waiting a few hours still is not continuous.
I saw my RAM is enough. my total's RAW is 64G which just used 30G

[felixkrones]

Not sure if this is related at all, but I had a very similar problem where my code just got stuck during training in nnUNetTrainer.py in line output = self.network(data).
The issue only occured when I defined a plan (e.g. nnUNetPlannerResEncL) since I got a warning that I was using an old one. It worked fine if I didn't define any plans and just used the code examples as give in the instructions.

[constantinulrich]

@dream-mjq would it be possible for you to switch to V2?

@felixkrones you are already on V2, right? The warning shouldn't stop your training. I would need more information to help you. Best you open a new issue

[dream-mjq]

there are the same problem, when I make a validation and test. the inference was done, but the output file just had 20(all is 300). the program is closed.

[constantinulrich]

Could you share the data? How many classes do you have?
When having more classes or the image sizes are too big, the resampling of the softmax probabilities to the original data size needs a lot of ram. My gues is still that you might run OOM there.

[dream-mjq]

I can share the data for you. how can I send to you. there are 31 classes for my dataset.

[dream-mjq]

Can I add my ram of my server to solve this problem?

[constantinulrich]

I can share the data for you. how can I send to you. there are 31 classes for my dataset.

Thats quite a lot. Most likely your Ram is the issue and if you have more RAM on a server would switch to that server