x265's CRF versus low contrast HDR sources..

[Boulder]

Originally Posted by benwaggoner

Going to a fixed file size at a reasonably challenging bitrate (so you're going to see some artifacts) is what I've found as the most efficient way to do these comparisons.

Then again, if bitrate is a property that I don't need to control (because I cannot tell a static average bitrate for all material), I shouldn't worry about it. As CRF is as close to constant quality that we can ever get, I have been able to set a satisfying CRF level which means it should be fixed to that.

For example, I just ran two test encodes; aq-mode 1, strength 1.0 gave me 6013 kbps and strength 1.8 needed 11610 kbps for the same clip with all the other settings kept the same. Running a 2-pass encode comparison at 6000 kbps, it's quite easy to predict that strength 1.0 will have less artifacts or is sharper and more detailed. It still doesn't mean it's any better at my desired base quality level

[benwaggoner]

Originally Posted by Boulder

Then again, if bitrate is a property that I don't need to control (because I cannot tell a static average bitrate for all material), I shouldn't worry about it. As CRF is as close to constant quality that we can ever get, I have been able to set a satisfying CRF level which means it should be fixed to that.

For example, I just ran two test encodes; aq-mode 1, strength 1.0 gave me 6013 kbps and strength 1.8 needed 11610 kbps for the same clip with all the other settings kept the same. Running a 2-pass encode comparison at 6000 kbps, it's quite easy to predict that strength 1.0 will have less artifacts or is sharper and more detailed. It still doesn't mean it's any better at my desired base quality level

The challenge is how to respond when something is, say 11% smaller but looks a little bit worse.

Because CRF is really not that close to constant quality. It's just a psychovisual offset from QP, and lots of changes can impact how good a given CRF value looks in practice. For example, try adding --nr 1000 to your string. It'll be a lot smaller and will look quite different. Pretty much all the psychovisual stuff changes appearance and efficiency with a given CRF value.


One can think of codec parameter tuning as finding ways to make the worst parts of the video look better so that a higher CRF value can be used for the same perceptual quality.

[K.i.N.G]

Originally Posted by Boulder

On HDR sources, mode 2 seems to produce noticably smaller files for the same CRF than mode 1.

With the footage I'm currently encoding (yes, its HDR) it's the other way arround.
AQ-Mode 1 turned out a lot smaller than AQ-Mode 2.
So it is highly dependant on the type of footage (grain, motion, contrast,...).

So, as Ben says, its always recommended to use 2-pass when comparing.
Or even 3-pass because there still is quite a difference between the resulting bitrates when using 2-pass.
Maybe 3-pass is more accurate? (don't have the time to test it, sadly)

So far, my (quick) tests on current project AQ-Mode 2 turned out as the 'winner'.
Its HQ footage with slight (but noticeable) noise. AQ-Mode 2 preserved details better in the chroma channels and resulted in quite a bit less distortion.
Keep in mind though this probably varies from type of footage... When having footage which has a lot of grain, then distortion might be a more acceptable type of artifact and result in a perceptually better image. (ill test that probably another time)

[K.i.N.G]

Originally Posted by Boulder

Using aq-strength 1.8 for mode 1 makes the bitrate shoot through the roof compared to strength 1.0.

Yeah, thats normal. Just use higher CRF.

When testing to compare quality:
Test using 2-pass and then when you find your desired 'target' try to match that by raising/lowering CRF until you get close enough to the desired result you got with 2pass.

[Boulder]

Originally Posted by K.i.N.G

With the footage I'm currently encoding (yes, its HDR) it's the other way arround.
AQ-Mode 1 turned out a lot smaller than AQ-Mode 2.
So it is highly dependant on the type of footage (grain, motion, contrast,...).

So, as Ben says, its always recommended to use 2-pass when comparing.
Or even 3-pass because there still is quite a difference between the resulting bitrates when using 2-pass.
Maybe 3-pass is more accurate? (don't have the time to test it, sadly)

So far, my (quick) tests on current project AQ-Mode 2 turned out as the 'winner'.
Its HQ footage with slight (but noticeable) noise. AQ-Mode 2 preserved details better in the chroma channels and resulted in quite a bit less distortion.
Keep in mind though this probably varies from type of footage... When having footage which has a lot of grain, then distortion might be a more acceptable type of artifact and result in a perceptually better image. (ill test that probably another time)

I've compared them on two quite different sources, the first Harry Potter movie (lots of noise) and then the Solo movie (quite clean). Aq-mode 1 kept the flat backgrounds better looking, mode 2 oversmoothed them even if I raised aq-strength to get the bitrate closer to what mode 1 gets. Didn't try lowering strength yet to see what happens. Nevertheless, the background looked ugly at default strength and mode 2 in both cases. Also in both cases the difference in bitrate was huge, mode 1 much higher than mode 2.

[Boulder]

Out of interest, would it be possible to run a 2-pass CRF encode so that the first pass uses a manually tonemapped version of the video and then the actual encode uses the real one? This would be really interesting to test at least

[benwaggoner]

Originally Posted by Boulder

I've compared them on two quite different sources, the first Harry Potter movie (lots of noise) and then the Solo movie (quite clean). Aq-mode 1 kept the flat backgrounds better looking, mode 2 oversmoothed them even if I raised aq-strength to get the bitrate closer to what mode 1 gets. Didn't try lowering strength yet to see what happens. Nevertheless, the background looked ugly at default strength and mode 2 in both cases. Also in both cases the difference in bitrate was huge, mode 1 much higher than mode 2.

Yeah, that's what I mean about VBR comparisons: "looked worse at a much lower bitrate" isn't that informative . It could be that aqm 2 and lowering CRF by 2 could be better and smaller.

[Boulder]

Originally Posted by benwaggoner

Yeah, that's what I mean about VBR comparisons: "looked worse at a much lower bitrate" isn't that informative . It could be that aqm 2 and lowering CRF by 2 could be better and smaller.

Well, going down on CRF so that the bitrates are close to each other still shows the ugly banding-like artifacts of mode 2

I think I also need to compare aq-modes 1 and 3 with SDR sources in a similar way.
EDIT: tested, and aq-mode 3 looks better at least in the video I checked. So it would seem to be the go-to setting for SDR but jury's very much out on aq-mode 2 and HDR.

[foxyshadis]

Originally Posted by Boulder

What does the "auto variance" actually mean? I've always thought that aq-mode 1 is already some kind of automatically varying mode based on qg-size.

mode 1 is based entirely on the energy of each individual block. mode 2 is based on the energy of the block vs the energy of the frame as a whole. mode 3 is mode 2 with dark-bias, of course.

hdr-opt basically steals bits from the dark to the light, the inverse of mode 3, since that's half the point of HDR. It also does something to the chroma, but I can't make heads or tails of that.

[K.i.N.G]

Originally Posted by Boulder

Well, going down on CRF so that the bitrates are close to each other still shows the ugly banding-like artifacts of mode 2

I think I also need to compare aq-modes 1 and 3 with SDR sources in a similar way.
EDIT: tested, and aq-mode 3 looks better at least in the video I checked. So it would seem to be the go-to setting for SDR but jury's very much out on aq-mode 2 and HDR.

It may also be useful to mention this:
I had a HDR video which when viewed on a non-HDR monitor through MadVR's tonemapper looked fine, but when viewed on an actual HDR monitor or TV there was obvious banding visible (in the lighter parts).

Which makes sense, since a HDR -> SDR tonemapper is essentially a soft clipping curve that compresses the highlights + some color space convertions.

[Blue_MiSfit]

I've noticed that watching HDR content via MadVR does absolutely require you to enable fullscreen exclusive mode in order to get 10 bit output, else you get banding.

This is obvious in hindsight, but I missed it at the time