Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can Open-Sora-Plan model be utilized to predict or generate the next frame of a video based on preceding frames? #258

Open
freshcoffee22 opened this issue Apr 22, 2024 · 1 comment

Comments

@freshcoffee22
Copy link

If so, could you please provide guidance on how to implement this in programming?
If direct prediction and generation of the subsequent video frame are not feasible, I would like to know how to craft cue words or prompts to assist in predicting and generating the next frame of the video.

@LinB203
Copy link
Member

LinB203 commented Apr 25, 2024

Sorry, since he's not an autoregressive model, it may not be possible to generate a new frame based on the previous one. But maybe we can explore considering the previous frame as a condition and control the generation of the condition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants