Issue when I try to use nnUNet to other model

[chenzhang9476]

Hi, all.

I know this isn't your obligation, but just wanna post and see if any of you tried to do similar thing like me before.
I'm trying to use nnUNet framework with Swin-Unet, which is transformer-based network. This is what i encountered.

As you can see, all the loss become a and pseudo dice is nan, this seems cannot be modified, I tried several times.
I simple put Swin-Unet under build_network_architecture function (but only used when training, converting data is still unet framework, otherwise cannot success.)

Thank for any advice.

[FabianIsensee]

Hey, let me tag @TaWald and @saikat-roy here since they have the most experience with this kind of stuff. My 2 cents:

• nnU-Net default initial LR is way too high for those architectures
• you may want to use AdamW instead of SGD here

Best,
Fabian

[saikat-roy]

Hey @chenzhang9476. Just following up on @FabianIsensee here. In our experience, when we trained SwinUNet using nnUNet as the training framework, we had to reduce the learning rate to 1e-4. We did use AdamW as the optimizer instead of SGD. But my guess is that, you would probably need to reduce the learning rate on SGD as well.

[chenzhang9476]

Thank you. But I’m confussing about the deep supervision.

…

On Tue, 28 May 2024 at 7:15 PM, Saikat Roy ***@***.***> wrote: Hey @chenzhang9476 <https://github.com/chenzhang9476>. Just following up on @FabianIsensee <https://github.com/FabianIsensee> here. In our experience, when we trained SwinUNet using nnUNet as the training framework, we had to reduce the learning rate to 1e-4. We did use AdamW as the optimizer instead of SGD. But my guess is that, you would probably need to reduce the learning rate on SGD as well. — Reply to this email directly, view it on GitHub <#2197 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWAKKBLQZKYKEPZYRNGNOKDZERDMHAVCNFSM6AAAAABH3WHPXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZUG4ZDQMZWGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[saikat-roy]

Hey @chenzhang9476 . Can you clarify what you mean by confused? Are you trying to switch off deep supervision or are you trying to use it but are unsuccessful?

[chenzhang9476]

Is deep supervision compliable with the other new framework like Swin-Unet?