Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Video to Video generation #243

Open
alfredplpl opened this issue Apr 15, 2024 · 1 comment
Open

Support Video to Video generation #243

alfredplpl opened this issue Apr 15, 2024 · 1 comment
Assignees

Comments

@alfredplpl
Copy link

I would like to v2v by your model.
I think we need to add two points on opensora/sample/pipeline_videogen.py.

  1. Create the encode_videos function like the follow:
    def encode_videos(self, videos):
        latents = self.vae.encode(videos)
        ...
        return latents
  1. Add code on the prepare_latents function so that the function copes with noised latents.

Any idea?

@LinB203
Copy link
Member

LinB203 commented Apr 18, 2024

Very creative idea. We can make further attempts. But I believe that before that, our model needs to be generalised enough, which means a lot of data to train.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants