You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This note shows equivalence of KAN to MLP in the piecewise linear approximation. I guess non-linearity of spline might help in some cases, but would be cool to have it as a baseline. Here's the reddit discussion
The text was updated successfully, but these errors were encountered:
hypnopump
changed the title
MLP Equivalent
KANs are MLPs? MLP Equivalent
May 6, 2024
It's equivalent to activating the same hidden state with multiple activation functions and then use a wider linear transformation to shrink it back. Somewhat like Gated Linear Unit, but somewhat in reverse: linear transformation goes after the broadening activation.
This note shows equivalence of KAN to MLP in the piecewise linear approximation. I guess non-linearity of spline might help in some cases, but would be cool to have it as a baseline. Here's the reddit discussion
The text was updated successfully, but these errors were encountered: