Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Projection of real video with multiple frames #35

Open
hse1032 opened this issue May 25, 2023 · 0 comments
Open

Projection of real video with multiple frames #35

hse1032 opened this issue May 25, 2023 · 0 comments

Comments

@hse1032
Copy link

hse1032 commented May 25, 2023

Hello,

I have some questions about the inversion of StyleGAN-V generator in FaceForensics dataset.

In case of image (a single frame), projection works well.
However, when I tried to project the video (multiple frames at once),
I found that the projected video contains almost identical frames in the entire time step.

Is this a normal phenomenon?

For projecting the video (16 frames in my case), I change some codes in "src/scripts/project.py" as below:

  1. adjust times step (0 to 16 frames)
    In line 59, from
    ts = torch.zeros(num_videos, 1, device=device)
    to
    ts = torch.arange(num_videos, 16, device=device)

  2. make motion code trainable (comment out the line 110 and uncomment line 109)

  3. extract target_features of real videos per frame, and change the distance as being measured between videos, not frames.
    For example, in line 140,
    dist = (target_features - synth_features).square().sum()
    In batch dimension of target_features and synth_features, they have 16 frames of a single video, not different images as original code does.

Thanks,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant