-
Notifications
You must be signed in to change notification settings - Fork 264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Motion blur magic #833
Comments
If I understand correctly, do you want to process all frames with the neural network or only certain frames with a certain amount of blur? The example on replicate takes around 17 seconds for a single frame on a Nvidia T4 GPU, im my opinion way to long to process all frames. Looking at my footage with long exposure times (walking), only around 10% of the frames have significant motion blur. Things I am asking myself:
|
Only some frames, the ones with most blur, based on the movement amount from gyro data. Each frame will likely need to be stored to a file. Hard to tell exactly how it will behave in real world, but I imagine we can add some blending or something if there's too much inconsistency. I'm sure the network will be much faster in Rust than in Python. Python does a lot of slow things before and after actual neural network, we'll do all of them on a GPU in a single step instead. |
That makes sense. I never used cog, but with TensorRT, the inference time didn't change significantly between the Python and C++. Why do we need to add motion blur back in Gyroflow? In my view, motion blur is a totally separate feature. The only advantage I see adding motion blur in Gyroflow is a reduced crop, if it is possible to apply it to the full unstabilized frame with the information of the stabilization. |
Because motion blur looks good if it's done right. That's why people use ND filters and slow shutter speeds, but this is incompatible with digital stabilization, so we can add it artificially after stabilization. |
We could implement a way to remove the motion blur using AI model: https://github.com/google-research/maxim
Then using the new AI-based optical flow, we can add the motion blur back after stabilization
For reference:
The text was updated successfully, but these errors were encountered: