Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th July 2018, 11:57   #1  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
x265's CRF versus low contrast HDR sources..

So, I'm trying to encode a UHD HDR sources using x265 with verry slow preset @ CRF17...

Things looks acceptable most of the time but near flat/low contrast areas get too much compression because x265 doesn't seem to take in account the contrast of how the HDR source will be displayed and it seems to be only analyzing the low contrast version (as it is played on a non-HDR screen)... or am I wrong?

Anyway, which parameter is recommended to adjust to get more detail preservation in low contrast areas (and without boosting bitrates in other areas as those are fine, so not just lowering crf for example)?
K.i.N.G is offline   Reply With Quote
Old 30th July 2018, 16:07   #2  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Try:
--aq-mode 3 --hdr-opt
(with 10 bit encoding)
https://x265.readthedocs.io/en/default/cli.html
sneaker_ger is offline   Reply With Quote
Old 31st July 2018, 08:19   #3  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by sneaker_ger View Post
Try:
--aq-mode 3 --hdr-opt
(with 10 bit encoding)
https://x265.readthedocs.io/en/default/cli.html
Im already using those
K.i.N.G is offline   Reply With Quote
Old 1st August 2018, 19:49   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by sneaker_ger View Post
Try:
--aq-mode 3 --hdr-opt
(with 10 bit encoding)
https://x265.readthedocs.io/en/default/cli.html
--aq-mode 3 is basically --aq-mode 2 plus a bias towards low luma. This matters because cheap LCD displays often don't have enough contrast in that range to not cause banding/blocking. Also, Rec 709 isn't really perceptually uniform, and so is somewhat starved for code values near black and has an excess of code values near white.

HDR-10 uses PQ (Perceptual Quantizer) which doesn't have this issue, and has way more code values in low luma - the problem aq-mode 3 addresses doesn't exist. So --aq-mode 2 is appropriate when PQ is being used. Using aq-mode 3 will result in more bits being spent in low luma. You can probably switch to using --crf 16 and still get a bitrate reduction switching to --aq-mode 2.

--aq-mode 3 is good for SDR and probably HLG.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 1st August 2018, 19:44   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
Things looks acceptable most of the time but near flat/low contrast areas get too much compression because x265 doesn't seem to take in account the contrast of how the HDR source will be displayed and it seems to be only analyzing the low contrast version (as it is played on a non-HDR screen)... or am I wrong?
I don't know how you can usefully QA HDR on a non-HDR display. You could well be getting artifacts from the HDR->SDR tone mapper. To really know what's going on, you need to be able to look at the source and output on a real HDR display to confirm you're seeing a significant reduction in quality.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd August 2018, 10:05   #6  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Is --aqmode 2 recommended over 1 for HDR encodes? The docs really don't say much.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 2nd August 2018, 19:28   #7  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Boulder View Post
Is --aqmode 2 recommended over 1 for HDR encodes? The docs really don't say much.
I generally prefer aq-mode 2 over 1, in the context of other tuning. aq-mode 1 is "safer" when bitrate isn't the paramount concern, but better low-bitrate quality is possible with aq-mode 2 (and particularly 3 for SDR - there's not an aq-mode 1 with low-luma bias).

The advantage of 2 over 1 in HDR is smaller than that of 3 over 1 with SDR. aq-mode 3 requires careful tuning with CRF, as it increases bitrate a decent amount in CRF mode, but since it improves quality in low luma, which is the place where low CRF is most needed, aq-mode 3 allows for higher CRF values to be used, and thus improved compression efficiency.

Getting aq-strength tuned in correctly is important; too much can waste bits in CRF, or hurt quality in highly detailed regions in lower bitrate CBR/ABR encodes.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 4th August 2018, 15:41   #8  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by benwaggoner View Post
The advantage of 2 over 1 in HDR is smaller than that of 3 over 1 with SDR. aq-mode 3 requires careful tuning with CRF, as it increases bitrate a decent amount in CRF mode, but since it improves quality in low luma, which is the place where low CRF is most needed, aq-mode 3 allows for higher CRF values to be used, and thus improved compression efficiency.

Getting aq-strength tuned in correctly is important; too much can waste bits in CRF, or hurt quality in highly detailed regions in lower bitrate CBR/ABR encodes.
Some extra questions regarding these ones. I usually use CRF 19 for 720p or 1080p encodes (with mattes cropped) with --aq-mode 1. If I use --aq-mode 3, is it generally OK to raise CRF accordingly (and probably tune down --aq-strength to 0.8-0.9) to get around the same average bitrate? That is, is the biggest difference inside the areas of lower luma in the frames anyway or is there a danger of making "normal" areas worse by accident? In a short test, --aq-mode 3 required 40% more bits for the same CRF with all the other settings kept as they were.

EDIT: one more thing: --aq-mode 3 is meant for 8-bit SDR sources even though I use the Main10 profile to encode?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 4th August 2018 at 16:46.
Boulder is offline   Reply With Quote
Old 7th August 2018, 00:15   #9  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Boulder View Post
Some extra questions regarding these ones. I usually use CRF 19 for 720p or 1080p encodes (with mattes cropped) with --aq-mode 1. If I use --aq-mode 3, is it generally OK to raise CRF accordingly (and probably tune down --aq-strength to 0.8-0.9) to get around the same average bitrate? That is, is the biggest difference inside the areas of lower luma in the frames anyway or is there a danger of making "normal" areas worse by accident? In a short test, --aq-mode 3 required 40% more bits for the same CRF with all the other settings kept as they were.

EDIT: one more thing: --aq-mode 3 is meant for 8-bit SDR sources even though I use the Main10 profile to encode?
--aq-mode 3 is meant for SDR sources. 8 versus 10 bit doesn't matter that much, although 8-bit tends to exacerbate the problems.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd August 2018, 21:04   #10  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Very interesting, thank you for the detailed explanation. I really wish there were proper, up-to-date tunings in x265 because these kind of things would definitely belong there. I for one have never tried changing the AQ settings because I don't have enough understanding of the details.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 22nd August 2018, 17:54   #11  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Thanks for the replies!

So HDR10 footage doesnt really benefit that much from --aq-mode 3 even at low'ish bitrates (let's say 4K @ ±14Mbit/sec)?
(The x265 docs mention it does help also for low bitrate 10bit footage, but that is for SDR)

Last edited by K.i.N.G; 22nd August 2018 at 17:58.
K.i.N.G is offline   Reply With Quote
Old 22nd August 2018, 21:52   #12  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
Thanks for the replies!

So HDR10 footage doesnt really benefit that much from --aq-mode 3 even at low'ish bitrates (let's say 4K @ ±14Mbit/sec)?
(The x265 docs mention it does help also for low bitrate 10bit footage, but that is for SDR)
At any given bitrate, aq-mode 3 should help most SDR content, but hurt most PQ HDR content. Since HDR’s PQ is much more perceptually uniform than gamma, and HDR displays do a much better job rendering low luma, shifting more bits into low luma will hurt mid-high luma regions more than it will help low luma regions.

In general, aq-mode 3 will significantly increase bitrate at a given CRF, or increase the QP of mid-high luma at a fixed bitrate. It is just aq-mode 2 plus a negative offset to QP in low luma regions.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 23rd August 2018, 03:27   #13  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by benwaggoner View Post
...and HDR displays do a much better job rendering low luma.
Yes, (I just bought an oled and its amazing) but I'm not sure why you're mentioning the ability of a screen to render low luma as if it's a reason to not put any attention to it...

You can have a TV that can render the highest quality low luma but if they are encoded badly that doesn't matter. They will still look bad.

My question is/was does x265's adaptive CRF algorithm take the HDR PQ into account or does it distribute bits using the old SDR gamma'ish curve(s)? (and thus resulting in loss of detail because the resulting loss in contrast)

I got this impression of this happening, because i have to use a much lower CRF to get something more decent and there is stil a lot of quality lost in low contrast areas (more than expected compared to other parts).
Sadly I currently have not enough time to experiment more...
K.i.N.G is offline   Reply With Quote
Old 28th August 2018, 18:32   #14  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
Yes, (I just bought an oled and its amazing) but I'm not sure why you're mentioning the ability of a screen to render low luma as if it's a reason to not put any attention to it...
We're going to pay attention to low luma! But with HDR, the steps between low-luma code values aren't more important than with mid or high luma. With SDR, a block of Y'=16 next to a block of Y'=17 can be painfully obvious. But in HDR, Y'=64 and Y'=68 are almost indistinguishable.

SDR needs lower QP in low luma. But HDR, with the Perceptual Quantizer is much more perceptually uniform, so the same QP works well independent of mean luma.

Quote:
My question is/was does x265's adaptive CRF algorithm take the HDR PQ into account or does it distribute bits using the old SDR gamma'ish curve(s)? (and thus resulting in loss of detail because the resulting loss in contrast)
My understanding is the all AQ modes other than aq-mode 3 assume perceptual uniformity. Although x264 being what it is, there may be some frame level luma offset constants in there somewhere.--aq-mode 3 exists exactly because AQ wasn't adequately sensitive to different QP needs at different luma values.

Quote:
I got this impression of this happening, because i have to use a much lower CRF to get something more decent and there is stil a lot of quality lost in low contrast areas (more than expected compared to other parts).
Sadly I currently have not enough time to experiment more...
Using -aq-mode 3 will generally use more bits at the same CRF than --aq-mode 2, due to reduced QP in low luma settings. However, since low luma issues is often what drives picking a lower CRF level, you can often get away with a higher CRF with --aq-mode 3 in SDR. Comparing at the same --bitrate is the proper testing procedure of course.

Using --aq-mode 3 in HDR still gives you the bitrate hit due to lower QP in darker regions, but no visible quality improvements. At the same bitrate HDR --aq-mode 3 will look worse than --aq-mode 2 because the reduced QP in low luma results in higher QP in higher luma, which is a net reduction in perceptual quality.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 29th August 2018, 05:55   #15  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by benwaggoner View Post
We're going to pay attention to low luma! But with HDR, the steps between low-luma code values aren't more important than with mid or high luma. With SDR, a block of Y'=16 next to a block of Y'=17 can be painfully obvious. But in HDR, Y'=64 and Y'=68 are almost indistinguishable.

SDR needs lower QP in low luma. But HDR, with the Perceptual Quantizer is much more perceptually uniform, so the same QP works well independent of mean luma.


My understanding is the all AQ modes other than aq-mode 3 assume perceptual uniformity. Although x264 being what it is, there may be some frame level luma offset constants in there somewhere.--aq-mode 3 exists exactly because AQ wasn't adequately sensitive to different QP needs at different luma values.


Using -aq-mode 3 will generally use more bits at the same CRF than --aq-mode 2, due to reduced QP in low luma settings. However, since low luma issues is often what drives picking a lower CRF level, you can often get away with a higher CRF with --aq-mode 3 in SDR. Comparing at the same --bitrate is the proper testing procedure of course.

Using --aq-mode 3 in HDR still gives you the bitrate hit due to lower QP in darker regions, but no visible quality improvements. At the same bitrate HDR --aq-mode 3 will look worse than --aq-mode 2 because the reduced QP in low luma results in higher QP in higher luma, which is a net reduction in perceptual quality.
That all has more to do with 8bit vs 10bit than HDR vs SDR...
Not that anyone in his right mind would ever encode an HDR video with 8bit, but it's technically possible.
Anyway, that was not what my question was about.

I think you are not quite understanding what I'm asking.

If we would encode HDR footage but tell de decoder that its a SDR video. Then when we play it it would be all washed out/low contrast, right?
This shows that post decoding there are some color adjustments going on in order to display the data/footage correctly (that is why we put flags in the video; the decoder needs to know how to interpret the data).
Just because the data has internally encoded a block as Y'= n doesn't mean it is displayed as Y' = n. It is most likely it never is.
When this data is decoded then the decoder checks which color space is assigned and which color space should be used to display the color.
Just to be sure I'm not misunderstood, I'm not talking about the TV's custom screen settings. I'm assuming a correctly calibrated screen.
There is a difference between a rec.709 rec.2020 flag, as you undoubtedly know. So x265's adaptive algorithms should be adjusted accordingly. If not, it's bit distribution would be much

One would expect x265 to do this, but the (few) tests I've done made me doubt this.

Last edited by K.i.N.G; 29th August 2018 at 06:17.
K.i.N.G is offline   Reply With Quote
Old 29th August 2018, 06:54   #16  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 56
Actually a more precise question of your problem is requested when others have to guest and obviously guest wrong. Someone once recalled before that the HDR footage stuffs are not linearly distributed. Either way is to use high-bit-depth accuracy transform to linear distributed plane and do all kind of process or what then transform back, or use the --aq-mode 2 with higher --aq-strength strength to compromise the lack of variance in large flat blight area. I assumed that there are some upscaling in your encoding process.

Sometimes the background and sky will get too much bits with a high --aq-strength. And, maybe a --aq-mode 4 for BT.2020 ?
alex1399 is offline   Reply With Quote
Old 29th August 2018, 19:59   #17  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Yes, I am aware of the fact that I might be not doing a great job at explaining my thoughts on this.
And i mean no disrespect to benwaggoner at all. His signature mentions he's Principal Video Specialist, Amazon Instant Video.. So I'm sure he knows more about this than I do.

I do a lot of 3D rendering for work. We use something called adaptive sampling and importance sampling to speed things up.
Areas of an image that are more perceptually important to the human eye get more samples than areas in which the human eye (and brain actually) don't notice much.
Combine that with the fact that things are rendered in linear space but when shown on a screen a color profile is applied, which changes the look of the image drastically.
Thus this color profile needs to be taken into account or the adaptive algorithms will make things look bad.
The same should be applied to a video encoder that relies on psychovisual algorithms, no?

So it is not enough that we have algorithms that only takes the human perception into account.
It first should consider how it will actually be displayed (color profile) and apply the adaptive psycho visual algorithms based of human perception on top of that.
So my question is; does x265 do that?

As far as I know aq-modes 1,2 or 3 will assign bits the same way wether you are encoding a rec.709 SDR video or bt.2020 HDR video.
This makes no sense to me because the data will be shown completely different on a screen.

Also (a bit of topic maybe), people seem to think that 10bit HDR will produce more smoother gradients than 8bit SDR.
But if you take into account that HDR has a much higher peak brightness those bits are getting stretched over a bigger range.
I know SDR has no defined brightness, so I added a bit more to what the average google results gave me, just to be sure.
8bit SDR = 256 steps stretched over about 250nits.
Lets say 256nits to make the math easy.
10bit HDR = 1024 steps stretched over 1000nits (or even +4000nits theoretically).
Lets say 1024nits to make the math easy.
So, both (in this case) have 1 gradient step per nit.
Thus 10bit wont prevent much banding in HDR footage as much as it would in SDR content.

Last edited by K.i.N.G; 29th August 2018 at 20:11.
K.i.N.G is offline   Reply With Quote
Old 29th August 2018, 23:56   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
I do a lot of 3D rendering for work. We use something called adaptive sampling and importance sampling to speed things up.
Areas of an image that are more perceptually important to the human eye get more samples than areas in which the human eye (and brain actually) don't notice much.
Combine that with the fact that things are rendered in linear space but when shown on a screen a color profile is applied, which changes the look of the image drastically.
Thus this color profile needs to be taken into account or the adaptive algorithms will make things look bad.
The same should be applied to a video encoder that relies on psychovisual algorithms, no?
I long for the world where all video is done in floating point linear light, but EXCEPT for 3D rendering and some VFX, most "normal" video stuff is done on some kind of curve. This is changing in film production with ACES.

No codec takes linear light as input, nor do codecs themselves do color volume conversions (encoders may do it, but that's upstream of the codec). That all gets done outside of the codec, which starts with subsampled Y'CbCr code values.

Quote:
So it is not enough that we have algorithms that only takes the human perception into account.
It first should consider how it will actually be displayed (color profile) and apply the adaptive psycho visual algorithms based of human perception on top of that.
So my question is; does x265 do that?
It doesn't happen automatically. One could argue that it should default to --aq-mode 2 when HDR metadata or --hdr-opt is set, and otherwise default to --aq-mode 3, but nothing like that actually happens.

Quote:
As far as I know aq-modes 1,2 or 3 will assign bits the same way wether you are encoding a rec.709 SDR video or bt.2020 HDR video.
Pretty much, although --hdr-opt does some QP adjustments itself.

Quote:
This makes no sense to me because the data will be shown completely different on a screen.
I shall not argue that video technology works the way it would work if we could start over today from scratch .

Quote:
Also (a bit of topic maybe), people seem to think that 10bit HDR will produce more smoother gradients than 8bit SDR.
But if you take into account that HDR has a much higher peak brightness those bits are getting stretched over a bigger range.
I know SDR has no defined brightness, so I added a bit more to what the average google results gave me, just to be sure.
8bit SDR = 256 steps stretched over about 250nits.
Lets say 256nits to make the math easy.
10bit HDR = 1024 steps stretched over 1000nits (or even +4000nits theoretically).
Lets say 1024nits to make the math easy.
So, both (in this case) have 1 gradient step per nit.
Thus 10bit wont prevent much banding in HDR footage as much as it would in SDR content.
Actually, video only uses the 16-235 or 64-960 range; values above and below are supposed to be still max black or max white.

And TVs are all over the place with what they use peak white for. If anything, the main reason HDR gets higher peaks in practice is that you never get a 1000 nit white frame in HDR, while a frame of Y'=230 isn't that uncommon. So with thermal/power limitations, HDR can have more small points of brighter display, since the average frame light level really isn't higher than for SDR on a typical consumer displays.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 31st August 2018, 14:25   #19  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
I only mentioned the fact that video editing and 3D rendering is done in linear space because that is an easy example.

Quote:
Originally Posted by benwaggoner View Post
Pretty much, although --hdr-opt does some QP adjustments itself.
Ahh, and there we go! So there is room for improvement.
They could adjust the adaptive quantizer's algorithm depending on what color space is selected.
And I'm convinced this could potentially increase efficiency/quality by quite a margin.

Quote:
Originally Posted by benwaggoner View Post
I shall not argue that video technology works the way it would work if we could start over today from scratch .
Sure, but this specifically 'only' requires adjusting the encoder not the entire history of how video works/evolved.

Last edited by K.i.N.G; 31st August 2018 at 14:51.
K.i.N.G is offline   Reply With Quote
Old 31st August 2018, 16:22   #20  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
I only mentioned the fact that video editing and 3D rendering is done in linear space because that is an easy example.
Very little video editing is done in linear. It's mainly seen in VFX and film coloring. Tools like After Effects and Premiere can have particular projects set to run in 32-bit linear light, but it's not the default. I do a lot in linear myself, but that's more doing corrections and conversions, not creative work.

Quote:
Ahh, and there we go! So there is room for improvement.
They could adjust the adaptive quantizer's algorithm depending on what color space is selected.
And I'm convinced this could potentially increase efficiency/quality by quite a margin.
I'm not sure about "quite a margin" but it is a historically undervalued aspect of psychovisual optimization.

One could actually think of --aq-mode 3 as "--sdr-opt."

Quote:
Sure, but this specifically 'only' requires adjusting the encoder not the entire history of how video works/evolved.
Yeah. Although an encoder that took linear light into the quantization stage and then quantized based on the output color volume could be awesome. It's always bothered me that we convert to final bit depth before doing the frequency transform, even though the iDCT values get more bits and those bits don't have 1:1 mapping with pixels anyway.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.