New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Visualize value vectors too? #31

Open

catherinesyeh opened this issue Jan 16, 2023 · 1 comment

Labels

data low priority

Owner

catherinesyeh commented Jan 16, 2023

Could plot value vectors too for each attention head... would definitely add to computational load though.

catherinesyeh added the data label

catherinesyeh added the low priority label

Owner Author

catherinesyeh commented Feb 27, 2023 •

edited

Could try scaling attention weights by value norms too

catherinesyeh referenced this issue


          adding weight by value norm for attention

f11b947

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment