SE is not working properly after 4th layer

[Fazlik995]

Hello,

SE layer is working fine after 1st, 2nd and 3rd layers.

However, when I apply it after 4th layer my test accuracy is lower than usual(about 2%). My train percentage is getting started from lower value.

I could not find the reason.

If i could explain and if you understand the question, what do you think about this issue?

Do you have any explanation?

Thank you

[hujie-frank]

Could you give more details about your problem, e.g. network architecture, task?

[Fazlik995]

Thank you fast reply.

Model architecture : ResNext-101, HMDB51 dataset

I am using SE module for 3D CNN action recognition to detect human actions from videos
https://github.com/craston/MARS (code which i am trying to implement)

As i mentioned SE is not working fine only after 4th layer, in all other cases it is working fine

In case of SE after 4th layer my result is lower.

Problem i got is below:

This val_epoch resulst are in normal case:

Val_Epoch: [1][1/47] Time 2.534 (2.534) Data 2.093 (2.093) Loss 3.7578 (3.7578) Acc 0.094 (0.094)
Val_Epoch: [1][2/47] Time 0.457 (1.495) Data 0.000 (1.047) Loss 3.6949 (3.7263) Acc 0.125 (0.109)
Val_Epoch: [1][3/47] Time 0.442 (1.144) Data 0.000 (0.698) Loss 3.7885 (3.7471) Acc 0.062 (0.094)
Val_Epoch: [1][4/47] Time 0.442 (0.968) Data 0.000 (0.523) Loss 3.6572 (3.7246) Acc 0.250 (0.133)
Val_Epoch: [1][5/47] Time 0.554 (0.886) Data 0.117 (0.442) Loss 3.7682 (3.7333) Acc 0.094

SE after 4th layer:

Val_Epoch: [1][1/47] Time 2.436 (2.436) Data 1.991 (1.991) Loss 3.8625 (3.8625) Acc 0.031 (0.031)
Val_Epoch: [1][2/47] Time 0.444 (1.440) Data 0.000 (0.995) Loss 3.8611 (3.8618) Acc 0.062 (0.047)
Val_Epoch: [1][3/47] Time 0.443 (1.108) Data 0.000 (0.664) Loss 3.8732 (3.8656) Acc 0.094 (0.062)
Val_Epoch: [1][4/47] Time 0.454 (0.944) Data 0.000 (0.498) Loss 3.8900 (3.8717) Acc 0.094 (0.070)
Val_Epoch: [1][5/47] Time 0.449 (0.845) Data 0.001 (0.398) Loss 3.8738 (3.8721) Acc 0.031 (0.062)
Val_Epoch: [1][6/47] Time 0.460 (0.781) Data 0.000 (0.332) Loss 3.9112 (3.8786) Acc 0.031 (0.057)

As you can see in 2nd case val_epoch value is lower than 1st case.

However, in case of https://github.com/kenshohara/3D-ResNets-PyTorch (the same with 1st code) SE worked fine even after 4th layer, in spite of the ResNext-101 model is the same.

I could not find the reason from the code.

I am not understanding how this could possible.