Experimenting with non-GoPro video overlay

[mfloryan]

First, thanks for a great project and especially the included docker image which makes running it nice and easy.

I was hoping to try out joining a phone-captured video with a Strava .gpx file.

Following the experimental suggestions from the docs I noticed two things:

• The example (that has GPX file and an input video) does not include explicit --overlay-size but the app seems to require it with --gpx-only
• When trying to run it with a gpx file and an input video file (after adding explicit overlay-size) I get an error.

The full command is:

gopro-dashboard.py \
  --layout xml --layout-xml layout.xml \
  --video-time-start file-created --use-gpx-only --gpx track.gpx --overlay-size 3840x2160
  phone_video.mp4 output.mp4

And the error is:

Traceback (most recent call last):
  File "/venv/bin/gopro-dashboard.py", line 144, in <module>
    video_duration = frame_meta.duration()
  File "/venv/lib/python3.10/site-packages/gopro_overlay/framemeta.py", line 217, in duration
    return self.framelist[-1]
IndexError: list index out of range

[time4tea]

Hi - Thanks for the interest! - Yes - this should be possible... let me see if i can reproduce this.

[time4tea]

OK - I think I know what is happening here - The error message is pretty horrible - that's something I can look into.

When creating an overlay video from two different sources, the program needs to know how to synchronize the GPX and the video file. There are a few options. I hope this makes sense, and isn't just stating the obvious.

• assume that the two sources are totally synchronised - so the video and the GPX start at the same instant - this is the ideal, but realistically, never going to happen. You run this like:
  • gopro-dashboard.py --layout-xml layout.xml --use-gpx-only --gpx track.gpx --overlay-size 3840x2160 phone_video.mp4 output.mp4
• assume that the create time of the video was when the video started, and the times inside the GPX file are correct.
  • gopro-dashboard.py --layout-xml layout.xml --use-gpx-only --gpx track.gpx ---video-time-start file-created -overlay-size 3840x2160 phone_video.mp4 output.mp4
  • in this mode, we filter any points in the GPX file that are before the "video-time-start" and then use a clock that ticks forward from that time until the end of the GPX file.

This happens similarly for the other "--video-time-start" modes, just using the "access" or "modified" dates of the video file instead.

What this can mean, and I believe is happening here - is that once all the GPX points that are "before" the date on the video are filtered out, there are none left. But like I said, the error message isn't helping here to tell you that!

It is of course possible there is a bug here.

What perhaps you could check is the date inside the GPX file for the first point. It will look something like:

<trkseg>
   <trkpt lat="xxxxx" lon="xxxx">
    <ele>27.0</ele>
    <time>2023-02-02T11:40:43Z</time>

Then compare that to the "file created" date of the video file - I'd use the stat command for this (on Linux or Windows WSL)

$ stat VID.MP4
  File: VID.MP4
  Size: 228499315 	Blocks: 446296     IO Block: 4096   regular file
Device: fd03h/64771d	Inode: 60575716    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  richja)   Gid: ( 1000/  richja)
Access: 2023-02-02 19:48:30.533545827 +0000
Modify: 2021-06-12 13:14:59.345588000 +0100
Change: 2023-02-02 19:48:09.033904114 +0000
 Birth: 2023-02-02 19:48:08.685909912 +0000

In my example - the two things don't overlap at all - so I get the same error as you. (Video created at 19:48, but GPX is from 11:40)

This is not terribly user-friendly at the moment!

Other people report good success in making the "overlay" by itself, then using a video editor to sync the two videos. There is a guide to that linked from the main page.

[time4tea]

You also mentioned that overlay size is required. I think this is both an error in the docs, and a missing feature in the software - it should be possible to use the size of any video, not just a gopro one.
I suspect that we might see more variability in dimensions using any video source, so the user is likely to have to create their own overlay temple... but let's see

[mfloryan]

Yes, generating just the overlay and syncing in a video editing software works really nice and I have already done that.
Would love to be able to skip that step.

[time4tea]

@mfloryan - hiya - new version 0.83.0 should address these issues - at least in terms of the error messages you get...

Now you'll see something like:

venv/bin/gopro-dashboard.py --video-time-start file-created --profile nvgpu --use-gpx-only --gpx somefile.gpx VIRB0031.MP4 output.mp4 
Starting gopro-dashboard version 0.83.0
GPX/FIT file:     2023-02-02T11:40:43+00:00 -> 2023-02-02T12:16:36+00:00
Video File Dates: 2023-02-02T19:48:09+00:00 -> 2023-02-02T19:51:30+00:00
Video file and GPX/FIT file don't overlap in time -  See https://github.com/time4tea/gopro-dashboard-overlay/tree/main/docs/bin#create-a-movie-from-gpx-and-video-not-created-with-gopro

Thanks for reporting this!

[mfloryan]

Thank you so much for getting into the details. I will try out the new version and report the results :)

[mfloryan]

Yes, the times seem to be all over the place. I have a video file recorded on the same phone during a Strava track recording and these are the results:

GPX/FIT file:     2023-02-02T14:33:17+00:00 -> 2023-02-02T14:37:04+00:00
Video File Dates: 2023-02-02T15:14:08+00:00 -> 2023-02-02T15:17:15+00:00

The file dates are all over the place:

Access: Thu Feb  2 16:14:08 2023
Modify: Thu Feb  2 16:14:08 2023
Change: Thu Feb  2 21:31:24 2023
 Birth: Thu Feb  2 16:12:45 2023

The exiftool dates of the video file are closer (but still not correct)

Create Date                     : 2023:02:02 14:36:49
Modify Date                     : 2023:02:02 14:36:49

and they match the ffprobe metadata

[time4tea]

Interesting.... You could try something like:

touch -d '2023-02-02T14:33:17Z' phone_video.mp4

then use --video-time-start file-modified ??

[time4tea]

hmmm. although if they are exactly aligned - then you don't need to use any --video-time-start at all - the default is (should be) to assume they are perfectly aligned.