Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Foward Transfer metric in metrics_summary.csv is the same across methods #225

Open
alexanderbuchholz opened this issue May 9, 2023 · 0 comments
Labels

Comments

@alexanderbuchholz
Copy link

alexanderbuchholz commented May 9, 2023

The forward transfer metric in the metrics_summary.csv is the same across different methods.

Run with Offline-ER and Joint updater on Mnist.

Expected behavior
I would not expect this to be (strictly) same for different methods.

Example
Joint training

Task ID Average Accuracy Forgetting Forward Transfer Backward Transfer
1 0.999054 0.000000 0.000000 0.000000
2 0.968869 0.010875 -0.206170 -0.010875
3 0.913238 0.043618 -0.103085 -0.043618
4 0.853757 0.092997 -0.190409 -0.092997
5 0.646799 0.273982 -0.142806 -0.273982

Offline-ER

Task ID Average Accuracy Forgetting Forward Transfer Backward Transfer
1 0.998582 0.000000 0.000000 0.000000
2 0.977127 0.006619 -0.206170 -0.006619
3 0.956068 0.016186 -0.103085 -0.016186
4 0.948742 0.013383 -0.190409 -0.013383
5 0.932628 0.013746 -0.142806 -0.013746
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant