Some question about VVenC at the current stage

[ZenKiyoshi]

I have some question about VVenC at the current stage, and sorry for my bad English, my text may be rude at some points.

1. GOP size
I see in VVenC help, the GOP size is 16, 32. So does VVenC use fixed GOP only currently?
Could VVenC use configurable GOP size like --key-int, --min-keyint in future?

2. Scene cut
If VVenC uses fixed GOP, I guess VVenC don't have scene cut yet, doesn't it? Just place I-frame for every 32 frames.

3. Blurry in motion scene
Yeah, I have opened an issue about this already, but I want emphasize this point a bit.
Enable QPA make blurriness in motion scenes. Disable QPA make motion scenes look better but still blurry when comparing with x265.
There are something else beside QPA taking bitrate out of motion scene, it could be the "offset table" I saw from config *.cfg files, I wonder.
x265 has some parameters for user configure the distribution of bitrate for motion scenes
--aq-mode, --aq-strength
--cutree
--qcomp
--ipratio, --pbratio
when I encode high quality, I have a lot bitrate to use, so I want every scenes look good include low motions, high motions, everything.
In x265 I configure those parameters to improve quality for motion scenes

Another circumstance, I encode the video with amount of bitrate, I check and see static scenes are good enough, then I want to improve for motion scenes.
I reencode, increasing bitrate, but I want those extra bitrate will go into motion scenes.

I hope VVenC could do it.

4. QP & CRF
Is --qp fixed qp?
Will VVenC use CRF like x264, x265?

5. Container
I can put vvc bitstream into mp4 container through mp4box dev, but I really need to put vvc into mkv container which I can merge with advanced substation alpha subtitle (the *.ass file).
Any plan to make vvc bitstream being compatible with mkv container?

6. Playable video player
This question, you may be asked alot.
But anyway, when can I play vvc natively in popular video players?

[adamjw24]

1. GOP size

GOP size is not the same as you might know from AV1 or x265, which is more comparable to our intra-period. Think about our GOP size as golden-frame period in aomenc. Still, for now, intra-period has to be a multiple of the GOP size (32 or 16). We are working on allowing arbitrary intra-periods indepdent of the GOP size. Work in progress, expect in 3-6 months.

2. Scene cut If VVenC uses fixed GOP, I guess VVenC don't have scene cut yet, doesn't it? Just place I-frame for every 32 frames.

We are just working on a first crack at scene detection, which will be handling scene changes a bit differently though that you might expect. Still, this will have a major impact with longer intra periods (>2s) and sequences with freuqent scene-changes - expect in March. Placing intra frames at scene cuts exactly is work in progress, but is blocked by the arbitrary intra-period feature - expect in 4-7 months.

3. Blurry in motion scene

We make it a point to create an encoder that just work and does not require a lot of manual tuning. Do not expect a whole zoo of fine-tuning options to be available and supported in VVenC.

We took the feedback and are evaluating the weighting of temporal vs spatial activity for QP adaptation (i.e. bitrate distribution within frame). This is work in progress.

But: VVenC is a video encoder, and single frames are only to be evaluated in context of the video during playback.

Another circumstance, I encode the video with amount of bitrate, I check and see static scenes are good enough, then I want to improve for motion scenes. I reencode, increasing bitrate, but I want those extra bitrate will go into motion scenes.

I hope VVenC could do it.

Not really. You could have a step in your framework in which you adapt the stats and fake the rate distribution from the first pass to achieve your options (basically changing the RC stats file). Otherwise the bits are distributed in a way we deem optimal for the video experience and this cannot be changed with a parameter. We might make it more adaptive in due course of VVenC development.

There are options in the FF-encoder you could use, but those are undocumented and you'd need in-depth knowledge of VVC to understand and use them.

4. QP & CRF

The new lookahead based single-pass RC will have a more constant rate. It is already the new default in the master and will be so in 1.4.

What is the exact specs of CRF that you are looking for?

5. Container

Thats not up to us. Please check with the governing body of the MKV spec, and the MKV tools developers.

6. Playable video player

See #140. There are options, there isn't really a good option, though. Please check with the developers of those players for the reasoning behind the lack of VVC support.

[ZenKiyoshi]

It's good to see Intra-period and Scene cut are coming. (in next few months)

We make it a point to create an encoder that just work and does not require a lot of manual tuning. Do not expect a whole zoo of fine-tuning options to be available and supported in VVenC.

Oh, that isn't my expectation.
I was shocking how many of x265 parameters show up in help. At that time, I only adjust few rate control, speed parameters to get big/small file or run fast/slow and leave rest of other default.
But when I read docs, go to encoding forums like doom9, videohelp to ask others: what purpose of this param, that params. How good/bad when increase/degrease this. Their experiences with their tweaks.
The conversations in those forum help me understand more about video codec. More conversations give more feed back/ideas to devs. Good conversation also make people want to use the encoder.

It's joyful and also headache. This is just my opinion.

I hope VVenC with full features (vvencFFapp) keep exposing more parameters beside normal VVenC.
Something like:
If you want it just work, use vvencapp.
If you want tweak, more headache, use vvencFFapp.

What is the exact specs of CRF that you are looking for?

1 pass variable bitrate.

I chose wrong word to ask in my first comment. "fixed qp" (the wrong word) I mean constant bitrate (CBR), distributing same bitrate for every frames, this is type of rate control I don't want to use. (may be useful in streaming)
I see VVenC --qp is CQP ralate to this document https://slhck.info/video/2017/03/01/rate-control.html & https://slhck.info/video/2017/02/24/crf-guide.html
In x265, CRF go with --qcomp, another parameter to modify bitrate in motion. When increase --qcomp to 1.0, CRF becomes CQP (https://x265.readthedocs.io/en/master/cli.html#cmdoption-qcomp).
So, as I understand, rate control modes of VVenC are:
--QP: Constant QP.
2-pass Rate Control: 2-Pass Average Bitrate.
1-Pass Rate Control: this look like constant bitrate (CBR).
Look-ahead Rate Control: I don't understand this, I feel it look like CRF.

Sorry if my questions disturb you, I am not dev, I don't know how much effort to make it work.
I am user, so the only thing I can do well is complaint (a lot)

[adamjw24]

Don't worry about asking questions, that's alright.

About the options.

In vvencFFapp we have more options, but only ones that we deem meaningful. For example we do the preset derivation using the options that are available with the FF-app, so those operating mode controls and tool-switches we test have to be exposed. Also, the FF-app contains some expert options, for some special use-cases, like sub-pictures. Every option there is has to be maintained, that's why we don't want to be adding options that we think have no benefit for the results, even in the FF-app.

About the RC. Yes, the new 1-pass RC (and frankly the legacy one too) is kind of CRF. We call it look-ahead based RC because of the way it operates, but its CRF when you consider what it does to the output.

The 2-pass RC is exactly what you say, just average bitrate over the whole sequence, no temporal restrictions.

[ZenKiyoshi]

VVenC 1.5 has released with "arbitrary intra period (independent of GOP size)" feature.

Does it actually work yet? (with scene cut)

I ask because I don't see any sign tell me that it actually work
Still 2 old options --IntraPeriod, --RefreshSec
The gop still 32 (and 16), the I-frame still insert after 32 frame as I see in command line encoding process.

[adamjw24]

* GOP: A group of picture with predefined hierarchical dependencies. A GOP would start with a key-frame (not necessarily an intra frame or RAP). If using open GOP (refresh types cra and cra_cre), the GOP would also use the first picture of the next GOP (a keyframe) as a reference picture. Don't use this feature if you don't know exactly know what you're doing and why. Now that I think about it, we will probably remove the option to set GOP size from the simple encoder (with arbitrary intra period support its not required anymore).

Minor correction. The difference between open and closed GOP only concerns the last GOP within an intra period. So in open GOP, the last GOP would use the first first frame of the next GOP and intra period (the RAPs I-frame) as a reference picture.

[ZenKiyoshi]

Thank for explanations, very helpful informations.

I have few more questions.

1. Could you explain more about predefined hierarchical dependencies of GOP. As I know, that is the "LambdaFromQp" table in config file (in cfg folder) and relate to rate control. I tried playing with this table by changing all row at QPoffset column to 0 and see in command line encoding process that all frames not key frame now are same QP, but bitrate of invidual frame are still variant. I try to change motion scenes QP equal with static scene QP.

2. Will VVenC scene cut have --min-keyint option which define minimal number of frames after scene cut.

3. If scene cut happen in middle of GOP, does it break hierarchical dependencies?

4. GOP size is independent with intra period size. So intra period has multiple of GOPs. In long static scene, it may waste of bitrate when this intra period will has many GOPs with many large size key-frame.
   Example: 10s scene at 24fps, number of GOP will be 240 / 32 = 7.5 => 8 GOPs so 8 key-frames
   In this circumstance, may be one GOP with distance as long as intra period that use only 1 key-frame is less bitrate (but harder for decoding?).
   I notice that change GOP size from 32 to 16 and the final filesize increase when encode the same file.
   Long static scenes are regular in anime

5. VVenC doesn't use any P-frames. In command line encoding process, I only see I-frame and B-frame. When I use mp4box to mux, it show 0 P-frames were used in video file. The number of I-frames are larger than x265, x264 encoding with same file.

And like question (4), use more I-frame may waste of bitrate?

[adamjw24]

Lets start with the easy questions.

5. Who even still uses P-frames? The 2000s are long gone and VVenC is not a real-time encoder. But jokes aside, there isn't really much reason for using P-frames in an offline encoder. The only difference between P and B frames is that the latter allow the so called bi-prediction (i.e. two motion compensated predictors that are blended together, usually just by building a 50/50 average of the two). Both predictors might even use the same reference frame, but e.g. with a small displacement. There isn't really much advantage of using P over B frames in offline scenarios, and a B frame can do all the things a P frame can do, and more. And the increased signaling caused by the larger option space is easily compensated by improved prediction.

Now to the tough questions.

1. A group of picture defines the prediction hierarchy. Pictures within a GOP are reordered and coded at different qualities. In our case, we would first code a key-picture of a GOP (pic 0), than the key-picture of the next GOP (pic 32, which can only use pic 0 as reference). Those are level-0 pictures. Next coded picture would be 16, temporally in middle of the two coded references, i.e. it has two equally valuable references. Since the references are closer temporally (expecting less change), and more, the QP for that picture can be increased a bit. Than picture 8 would be coded, equidistantly placed between pics 0 and 16. It can use pictures 0, 16 and 32 as references, while it is expected that it will mostly use pics 0 and 16, since they are closer. And so on, and so on. Uneven pictures (pics 1, 3, 5... 31) are on the top of this hierarchy and are coded with the highest QP values. The bitrate is variant for no QPoffset because of the increased prediction accuracy (because of less temporal distance to the reference pictures). One additional settings is the IntraQPoffset, which is applied to all intra-frames (both RAP and non-RAP).
2. It will probably be called differently but possibly yes. We would go that way if the adaptive intra period prooves better than what we already have.
3. If the cut happens in the middle of the GOP, be it because of a scene-change or because the arbitrary intra period feature that asks an I-frame be inserted not where a key-frame would, stuff gets complicated. What we do now, we close the current GOP up to the new keyframe, as if it was the end of the video, and just open a new GOP where the new I-frame is supposed to be. For open GOP encoding, we would first encode the start of the old GOP, the oddly placed start of the new GOP, and than all frames in-between using both keyframes as references (basically like the end of a video, with the additional future reference). Its the reason why feature took so long, there's a lot of details to look for.
4. No. Dont forget that the keyframes of most GOPs are still B-pictures. If the scene is static and there is not much change between the keyframes of two consecutive GOPs, the encoder would detect that and encode that second keyframe using cheap to-signal prediction modes and with small residuals. What you mean is if a short intra-period would increase the size unnecessarily. It would increase the filesize, but not unnecessarily. Don't forget intra-period is mostly an usability feature that allows you to seek in the video, chunk it for straming, or tune-in when switching TV channels.
   About the increase in file-size for GOP16 over GOP32. That is expected. Thats why we use and encourage the use GOP32. There is really no disadvantage to it, none. We only allowed GOP16 so people could use intra-periods like 48, but now this is also available with GOP32, and its more efficient. If GOP16 is ok for your application, GOP32 is also.

So... this became quite long. Hope it helps.

// edits: typos

[adamjw24]

And a little bit more about 5. The number of I-frames should be exactly what you choose it to be. If you choose the intra-period to be equal in x265, x264 and VVenC, the differences should be marginal. You can use -ip -1 to skip RAP I-frames alltogether, but this would kill seeking in the video (there would still be some I-frames to deal with scene-changes).

[ZenKiyoshi]

Thank for explanations, some stuffs are complicated with me. It's good to see every thing are evolving.

I still concern about rate control, quality of motion scenes (blurry, lost details...). But, it's a big topic, may be early at the moment, I will comeback in next few months for more conversations about it.

[ZenKiyoshi]

I try VVenC 1.6.0 final and see VVenC now is more blurry than x265 even in static scenes, I spot this behavior since ver 1.5.0

My command
vvencapp -i - --y4m --preset fast --qpa 1 --qp 21 --threads 12 -o "fast_qpa1_qp21.266"

1.6.0 has improved for 1 pass RC, so I make one.
vvencFFapp -i - --y4m -c randomaccess_fast.cfg --NumPasses 1 --LookAhead 1 --TargetBitrate 3.0M --PerceptQPA 1 --Threads 12 -b "fast_qpa1_1passRC-3.0M.266"

Comparison: x265 vs fast_qpa1_qp21 vs fast_qpa1_1passRC-3.0M
https://slow.pics/c/U2amV29y
You could see at the left or right building.

My old comparison shows that older version of VVenC can keepup details in static scenes like x265.
I don't remember what VVenC version, 1.4.0 probably.
https://slow.pics/c/ZG4IcvMp (Any thing else not inside the red mark)

Origin video: https://drive.google.com/file/d/1snMfBndQO18C_njP1U7aZU6t0w8fDDNY/view?usp=sharing
Encoded: https://drive.google.com/drive/folders/15zPQe2uFqplm6qTa7lsZpb7BRzkspqak?usp=sharing