New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请问有支持PoSE的打算吗？ #644

Open

2793145003 opened this issue May 6, 2024 · 0 comments

Labels

feature request

2793145003 commented May 6, 2024

PoSE：https://arxiv.org/abs/2309.10400
OpenAccess-AI-Collective/axolotl#1567
不需要全长度微调，无需修改 attention
好像可以用短上下文训练长上下文

pppppM added the feature request label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment