New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorporating Corrected Implementation of ROME in EasyEdit #233
Comments
Dear Akshat, Thanks for your great work. We will add r-ROME in EasyEdit. We are recently rushing for the NeurIPS deadline and might need some time to integrate. If you are interested, you can also submit a PR. Best, Team of EasyEdit |
Will do. Thanks so much. Will reach out here when we submit a PR. |
Just submitted the PR to implement R-ROME: #237 |
Thanks for your contribution, I will reorganize your code and rename it to |
Hi! Again, thanks so much for considering our work to be part of the easyedit offering! Wanted to check in to see if there is an update or an estimated timeline for incorporating r-ROME into Easyedit? |
Neurips submission is coming soon. I will restart this matter after 5.22 and support it as soon as possible. Thank you for your timely follow-up. If you are interested, I also welcome you to submit a new PR. 😊 |
Hi,
Recent works found some instability when using ROME resulting in model collapse. We found that the original implementation of ROME was not the correct representation of update equation derived in the paper, and we've created a repo with the corrected implementation of ROME, a version we're calling r-ROME.
We've explained in the paper the exact details of the irregularities in the original implementation in our paper here - https://arxiv.org/abs/2403.07175
We also find that the corrected implementation has better overall performance and better generalization and localization, and also allows for stable sequential editing without model collapse.
We also provide the corrected code here - https://github.com/scalable-model-editing/rebuilding-rome
We would be grateful if our work can be incorporated in your repository and framework. We also believe it will add value to the EasyEdit community.
Thank,
Akshat
The text was updated successfully, but these errors were encountered: