New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Got bad results using MEMIT to edit Qwen model #248
Comments
Thank you very much for your interest in EasyEdit. We apologize for our limited availability as we are currently busy with the nips submission deadline. We haven't attempted editing on qwen-14b before. However, based on our experience, you can try setting the Tokenizer's padding_side to "right" during editing. This means you would need to change the code here to |
Thanks for your quick response and wish you good luck for the NIPS submission! As shown below, the first prompt is edited in key fact now but some hallucination is caused as well;
Do I have other options to improve this performance? Thanks! |
I noticed that you're editing three pieces of data simultaneously, and the continuous editing effect is indeed average. You can try editing one piece of data at a time. Also, since the config we provide is for Qwen-7B, you can try adjusting the parameters to fit Qwen-14B. Furthermore, you can also try changing the input, as the editing method cannot guarantee success for every input. |
I used MEMIT to edit Qwen-14B model, running the demo adapted from the official example, and the result is pretty bad (shown as below).
The edited model only generated exclamation marks for the first and second prompt.
The third result seems effective.
Are these results as expected? Did I make some mistake in my code? How can I improve it?
Please advise! Thanks!
[MY RESULTS]
[MY CODE]
[MY HYPERPARAMS]
The text was updated successfully, but these errors were encountered: