Generate MKV Sidecar File Directly From FFmpeg

[briandwagner]

Hello!

This actually relates to a documentation issue, not a code issue. Sorry if this isn't the right place to ask this. If this question needs to be posted somewhere else instead, please let me know!

I'm trying to make some QCtools sidecar files entirely from ffmpeg. I found this page http://bavc.github.io/qctools/data_format.html that explains how to do make the xml.gz files from ffprobe, but it doesn't give an example on how to generate the newer mkv type.

Can the documentation be updated to include how to do this?

Thank you so much!

[dericed]

Hi @briandwagner, thanks for pointing this out. Earlier in qctools it would mimic the xml output of ffprobe, so it was feasible to make roughly the same xml from either ffprobe or qctools. Now that qctools is using a lot more pre-generated visualization, like thumbnails and panel tracks, we wrap those images and the xml into Matroska so it's all together. FFmpeg wouldn't be able to make this all in one line, but QCTools has a command line tool called qcli which can. Is there a reason to prefer ffprobe over qcli in this instance? Else I did have a script to help test out the qctools.mkv format at https://gist.github.com/dericed/643574dc5327ec7207c4850730ce83eb. This shows the panel tracks being added to an existing qctools file which contains thumbnails and the xml.

[briandwagner]

Hi @dericed! Great to meet you!

I'm trying to generate some qctools.mkv files at the time of conversion and/or capture via ffmpeg. Since ffmpeg is having to decode the video anyway, I was hoping to save some time by splitting ffmpeg's output so the video only has to be decoded once. I tried piping out of ffmpeg directly into qcli already, but qcli doesn't seem to want to grab video from stdin, even when setting /dev/stdin as the input file. Perhaps I'm doing something wrong?

I took a look at one of the qctools.mkv files with ffprobe, and I tried to recreate it myself using:

ffprobe -f lavfi -i "movie=original.avi:s=v+a[in0][in1],[in0]signalstats=stat=tout+vrep+brng,cropdetect=reset=1:round=1,idet=half_life=1,deflicker=bypass=1,split[a][b];[a]field=top[a1];[b]field=bottom,split[b1][b2];[a1][b1]psnr[c1];[c1][b2]ssim[out0];[in1]ebur128=metadata=1,aformat=sample_fmts=flt|fltp,astats=metadata=1:reset=1:length=0.4[out1]" -show_frames -show_versions -of xml=x=1:q=1 -noprivate | gzip > original.avi.qctools.xml.gz  

ffmpeg -i original.avi -filter_complex "[0:v]split=2[qcli_v1][qcli_v2]; [qcli_v1]scale=72:-1[qcli_v1_scaled]; [qcli_v1_scaled]setparams=range=pc:color_primaries=bt470bg[qcli_thumbs]; [qcli_v2]scale[qcli_v2_1]; [qcli_v2_1]format=rgb24[qcli_v2_2]; [qcli_v2_2]crop=1:ih:iw/2:0[qcli_v2_3]; [qcli_v2_3]tile=layout=512x1[qcli_v2_4]; [qcli_v2_4]setsar=1/1[qcli_v2_final]" -map '[qcli_thumbs]' -metadata:s:v:0 title="Frame Thumbnails" -pix_fmt yuvj420p -c:v mjpeg -map '[qcli_v2_final]' -metadata:s:v:1 title="Tiled Center Column" -metadata:s:v:1 FILTERCHAIN='scale,format=rgb24,crop=1:ih:iw/2:0,tile=layout=512x1,setsar=1/1' -metadata:s:v:1 VERSION="1.0" -metadata:s:v:1 YAXIS='Bottom:Top' -metadata:s:v:1 LEGEND="Tiled
Center Column" -metadata:s:v:1 PANEL_TYPE="video" -pix_fmt yuvj420p -time_base "100/2997" -r "2997/100" -c:v mjpeg original.avi.thumbs.mkv

ffmpeg -i original.avi.thumbs.mkv -attach original.avi.qctools.xml.gz -metadata:s:2 mimetype="application/x-gzip" -map 0:0 -c:v copy -map 0:1 -c:v copy original.avi.qctools.mkv

In this instance, I'm not doing any other conversion with ffmpeg. However, in reality, I would add other, non-qctools related outputs. I think the first and second commands could also be combined into one line by splitting the video another time and piping that directly into ffprobe. Even though that would still be a total of two lines, the big video file is only being decoded one time for all of my conversion and qctools work. Muxing the xml.gz and the thumbnails together takes only a second or two. This would save many hours of time once applied to hundreds or even thousands of videos.

Obviously, all that filtering was me mostly guessing based on the ffprobe output of a genuine qctools.mkv file. I think your script will get me much closer. QCtools would actually take my file and it mostly worked, but my video would not playback correctly inside the qctools gui. It would play the video at about 10x speed, but the audio at the normal rate. Although I suppose it's also possible my original test video is just malformed somehow.

Thanks so much!