about the global and local features in fig 3

[sanwei111]

as we know，the conventional attention module can capture features like fig 3.b（including diagonal and other positions）. THIS ability is its nature，BUT i JUST wonder that when we add a branch that can capture local features，the attention module can not capture feature like before，i.g，（including diagonal and other positions），while it just capture global feature！！！

[wwwadx]

Same question, how the model make sure that the attention layers capture the global information and the CNN layers capture local information with only one NLL loss? Have you figure it out?

as we know，the conventional attention module can capture features like fig 3.b（including diagonal and other positions）. THIS ability is its nature，BUT i JUST wonder that when we add a branch that can capture local features，the attention module can not capture feature like before，i.g，（including diagonal and other positions），while it just capture global feature！！！

[sanwei111]

of course

…

---Original--- From: ***@***.***> Date: Tue, May 24, 2022 09:54 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [mit-han-lab/lite-transformer] about the global and localfeatures in fig 3 (#35) Same question, how the model make sure that the attention layers capture the global information and the CNN layers capture local information with only one NLL loss? Have you figure it out? as we know，the conventional attention module can capture features like fig 3.b（including diagonal and other positions）. THIS ability is its nature，BUT i JUST wonder that when we add a branch that can capture local features，the attention module can not capture feature like before，i.g，（including diagonal and other positions），while it just capture global feature！！！ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>