Benwaggoner HEVC encoding challenge

[benwaggoner]

Originally Posted by rwill

Is this size easier to read?

Yeah. I've hoped someone else would, but I may get curious enough to do it myself one of these days.

x265 and Beamr have also gotten better since the most recent HEVC submissions, so fresh ones of those would make sense.

[Z2697]

Originally Posted by benwaggoner

Yeah. I've hoped someone else would, but I may get curious enough to do it myself one of these days.

x265 and Beamr have also gotten better since the most recent HEVC submissions, so fresh ones of those would make sense.

I think the "core functions" of x265 hasn't been updated for years, there will be no difference if the parameters are identical.

[benwaggoner]

Originally Posted by Z2697

I think the "core functions" of x265 hasn't been updated for years, there will be no difference if the parameters are identical.

There have been a decent number of minor bug fixes that can slightly improve quality since, say, five years ago. The bigger differences will be from us having learned more about how to tune x265, and having new parameters to use. That said, a lot of newer stuff is about improving quality @ perf, not improving quality at ultra placebo settings.

This test is, sheesh 6.5 years old now. 8-bit HD SDR really isn't the cutting edge anymore!

We should do a new challenge using StEM 2 as source (https://dpel.aswf.io/asc-stem2/).

For the modern era, what? Just throwing the below as a starting point for discussion:

1. StEM v2 "Quicktime with stereo, 1.78 @ UHD, 1000 nits ProRes4444XQ, Rec2020"
2. (should we make a shorter edited version for faster encoding and easier testing? The full thing is 23 minutes long)
3. 5 Mbps ABR, 10 Mbps peak (challenging enough)
4. 24.00 fps
5. 384x2160p
6. Frame resizing techniques allowed
7. Film grain synthesis allowed? A with-FGS and without-FGS version?
8. 10-bit HDR PQ Rec. 2020 primaries limited range
9. 2-5 second variable GOP duration (fixed 2 seconds is more challenging, but increasingly less mainstream)
10. Closed GOP for adaptive streaming (techniques like RADL allowed as long as each fragment is independently decodable)
11. Objective metrics don't matter, subjective evaluation only.

How's that look? Any questions or suggested modifications? This represents all of 5 minutes of my consideration, so I hope there are!

[tormento]

Originally Posted by benwaggoner

Film grain synthesis allowed? A with-FGS and without-FGS version?

I haven't yet understood if FGS is really a thing or not in x265 and, actually, I don't know of players that can recreate it when reproducing HEVC video.

Does it work as in AV1, where is removed and when played reconstructed or in some other way?

Can you give us some examples of how to create a proper FGS video with x265?

[Z2697]

126 GiB, that's a challenge by itself

[Z2697]

Eh, I downloaded it and it's 17:26 long, did I downloaded the wrong one?

The pixel format is YUV444P12, shall we specify the method of subsampling and dithering? Or even encode as is?

[excellentswordfight]

Originally Posted by Z2697

126 GiB, that's a challenge by itself

Originally Posted by Z2697

Eh, I downloaded it and it's 17:26 long, did I downloaded the wrong one?

The pixel format is YUV444P12, shall we specify the method of subsampling and dithering? Or even encode as is?

Sounds like you downloaded the IMF-version (J2K compression). And yes that is the length of it.

I think StEM2 is a pretty neat source, its very representive of "modern content", i.e. HDR UHD shot digitally with a rather clean image, I also like that there are both SDR and HDR version available which can make for some nice comparisons. My only complaint would be that the credits are a fairly big portion of the title.

I have been using the IMF versions with this commandline:

"ffmpeg.exe" -probesize 1000MB -ss 00:00:08 -i "hdr\StEM2_HDR_Rec2020PQ_444F_IMF_2160p24_178.mxf" -vf scale=out_color_matrix=bt2020ncut_h_chr_pos=0ut_v_chr_pos=0 -pix_fmt yuv420p10le -an -f yuv4mpegpipe -strict -1 - | "x265.exe" --y4m ...

And using the SDR version for 1080p tests:

"ffmpeg.exe" -probesize 1000MB -ss 00:00:08 -i "sdr\StEM2_SDR_Rec709_444F_IMF_2160p24_178.mxf" -s 1920x1080 -sws_flags spline -pix_fmt yuv420p10le -an -f yuv4mpegpipe -strict -1 - | "x264.exe" --demuxer y4m ...

[rwill]

Originally Posted by benwaggoner

We should do a new challenge using StEM 2 as source (https://dpel.aswf.io/asc-stem2/).

I don't know about the 5Mbit.. have you seen my StEM 2 HEVC encodes on the previous page? Either I just cannot see the problems at 4Mbit or 4Mbit was too much already.

We need a Video Buffer Size too.. I suggest 2sec of peak rate.

Z2697 already raised a valid concern about 444 -> 420 and bitdepth reduction being an undefined preprocessing step. So someone has to make a 10bit 420 of the source or alternatively just let everyone do their own thing. Or just define some recent ffmpeg version + cmdline to get to yuv420p10le.

@Z2697: 17:26 min duration sounds about right.

[Z2697]

I downloaded the ProRes4444XQ 4K HDR version, the IMF version is much larger.

A fairly big portion of the file is credits, that's "time speaking", the bitrate of the credits portion is much lower compared to the main portion.
Probably also why the seemlingly low bitrate look ok, it's averaged out by the credits part, the main part get a somewhat higher average bitrate.

If the encoding time is a concern, perhaps we can cut the credits part off... however I'm not sure if it will violate the license, it does say "Redistributions of these digital assets or any part of them must include the above copyright notice" though.

[excellentswordfight]

Originally Posted by Z2697

I downloaded the ProRes4444XQ 4K HDR version, the IMF version is much larger.

Is it? StEM2_HDR_Rec2020PQ_444F_IMF_2160p24_178.mxf is 124GiB over here. Maybe they have updated the packages? Or maybe the IMF download includes both 4K and UHD versions, it was a few years ago I downloaded it.

I havnt encoded the UHD HDR version at bitrates bellow 10Mbps, but I agree with the points above, as the source is rather easy to compress we need to choice a bitrate were modern codecs struggle, or at least will display differences that is relevant under normal viewing conditions. Has someone done a sanity check at 5Mbps?

edit.
I also now remembered this post i made: https://forum.doom9.org/showthread.p...13#post1977513 and especially this:

"Its still a bit interesting that even now when when I convert it in rather controlled fassion, there is both a luminance shift and colorshift (mostly reds as usual) compared to the Prores & AVC version that they offer. But I also saw encoding errors on the prores version, so Im not sure how carefully they have treated those versions... "

That was for the SDR-version mind you, but you might wanna doublecheck the prores HDR source as well (this is how the SDR prores looked https://ibb.co/MCFvGF7, and this is how it should look https://ibb.co/dkNhgqB)

So please be wary of that, if I also remember correctly, there are at least one bad frame in all soruces, that looks like an in-camera/capture issue.

[rwill]

Originally Posted by excellentswordfight

Is it? StEM2_HDR_Rec2020PQ_444F_IMF_2160p24_178.mxf is 124GiB over here.

I havnt encoded the UHD HDR version at bitrates bellow 10Mbps, but I agree with the points above, as the source is rather easy to compress we need to choice a bitrate were modern codecs struggle, or at least will display differences that is relevant under normal viewing conditions. Has someone done a sanity check at 5Mbps?

You can always click on the link Ben has in his post to see the more or less official releases. There the ProRes is 126 GB and the IMF is 118, 243 or 554 GB.

Regarding a 5Mbit encode you can, as I stated above, always go one page back in this thread and check out my 4 and 2 Mbit encode.

[excellentswordfight]

Originally Posted by rwill

You can always click on the link Ben has in his post to see the more or less official releases. There the ProRes is 126 GB and the IMF is 118, 243 or 554 GB.

The 118 IMF is the SDR-version (that I also have downloaded), I was more confused about that there was no one matching the size of the 124GiB version that I have. But I did an edit on my post, I think the 243GB one might have both a (DCI)4K and a UHD version included.

Edit. Yes thats the case, I still had the xml-files, they reference a 4K files as well: StEM2_HDR_Rec2020PQ_444F_IMF_1716p24_239.mxf, as well as StEM2_HDR_Rec2020PQ_444F_IMF_2160p24_178 (the one that I kept).

[excellentswordfight]

I did "baseline" encode using your critieras @benwaggoner (selected a 4s fixed IDR I-frames GOP, with adaptive non IDR i-frames placement), with just straight up preset slower and no tweaking.

"x265.exe" --y4m --preset slower --profile main10 --level-idc 50 --bitrate 5000 --vbv-maxrate 10000 --vbv-bufsize 10000 --keyint 96 --min-keyint 96 --rc-lookahead 96 --no-open-gop --pass 1 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,50)" - -o NUL
x265.exe" --y4m --preset slower --profile main10 --level-idc 50 --bitrate 5000 --vbv-maxrate 10000 --vbv-bufsize 10000 --keyint 96 --min-keyint 96 --rc-lookahead 96 --no-open-gop --pass 2 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,50)" - -o out

https://limewire.com/d/3010926b-e7fa...ThVOwMp-JGr6qk

And yes, we are going to need a harder sample that stresses the encoder more, even at what is a relative low bitrate for UHD at 5Mbps, its gonna be hard to see any meaningful differences, at least for good encoders.

[benwaggoner]

Originally Posted by tormento

I haven't yet understood if FGS is really a thing or not in x265 and, actually, I don't know of players that can recreate it when reproducing HEVC video.

Does it work as in AV1, where is removed and when played reconstructed or in some other way?

Can you give us some examples of how to create a proper FGS video with x265?

Yeah, it would work the same as AV1, using a similar workflow.

x265 can mux FGS metadata from a sidecar file, so it is a thing to that degree. You'll need a tool to create the FGS metadata and players that can correctly parse that and generate the FGS overlay, which is the harder part. AV1 is really the only codec where most player software implements the FGS rendering.

There's been discussion of using either the old MPEG AVC FGS technology or using the AV1 one with HEVC, VVC, and other codecs. The AV1 is "funky" in that a lot of random seeds produce weird patterning. Tools are getting much improved year-on-year, but I'm not aware of any mainstream streaming service using AV1 FGS yet; some early devices shipped with defective implementations, not all of which have been patched since.

I've not spent enough time with practical implementations of the MPEG approach to know how well it compares, or how good the available tools are. The hardest part is the grain-parametrization-and-removal analysis preprocessing in the end. Grain removal hasn't historically been something you'd batch process whole titles through. AI is proving to be a good tool to use in this, and I think it's on the cusp of being production ready.

[benwaggoner]

Originally Posted by rwill

I don't know about the 5Mbit.. have you seen my StEM 2 HEVC encodes on the previous page? Either I just cannot see the problems at 4Mbit or 4Mbit was too much already.

We need a Video Buffer Size too.. I suggest 2sec of peak rate.

I was thinking the lower of 25 Mb (max VBV for HEVC Level 5.0) or the max GOP length at peak rate. Larger allows for better rate control and multipass/lookahead encoding to improve quality. Of course, lower also stresses how well the codec handles QP spikes. Open to discussion here.

Z2697 already raised a valid concern about 444 -> 420 and bitdepth reduction being an undefined preprocessing step. So someone has to make a 10bit 420 of the source or alternatively just let everyone do their own thing. Or just define some recent ffmpeg version + cmdline to get to yuv420p10le.

For the first test I made a canonical .y4m to download, which could make sense to eliminate any risk of divergent processing. And probably edit it down some from the full length (speed up the credits?). That length wouldn't be a serious issue with HEVC or AV1 these days, but could be for VVC and AV2 due to lack of encoder maturity.

We need to be cognizant of the specifics of the Creative Commons license being used.

[benwaggoner]

Originally Posted by excellentswordfight

I havnt encoded the UHD HDR version at bitrates bellow 10Mbps, but I agree with the points above, as the source is rather easy to compress we need to choice a bitrate were modern codecs struggle, or at least will display differences that is relevant under normal viewing conditions. Has someone done a sanity check at 5Mbps?

Good point. Should we identify some content with film grain? Alternatively, we could argue that in the future everything would use FGS, so how we encode no-low grain content is really the important question.

Thoughts? Any alternatives to suggest?

[tormento]

Originally Posted by benwaggoner

Yeah, it would work the same as AV1, using a similar workflow.

Given your last paragraph conclusion, do you have any idea about how to do it in real terms? Having working grain generation in hevc would be really a gamechanger.

[benwaggoner]

Originally Posted by tormento

Given your last paragraph conclusion, do you have any idea about how to do it in real terms? Having working grain generation in hevc would be really a gamechanger.

I haven't messed with tools for the MPEG FGS since the HD-DVD era.