Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th August 2020, 20:38   #21  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by huhn View Post
for 1080p is pretty easy to show "major" difference been chroma scaler.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.
I am sure there are 8k and 12k raw samples online you can demosaic to 4:4:4.
Cary Knoop is offline   Reply With Quote
Old 27th August 2020, 23:01   #22  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,871
Quote:
Originally Posted by huhn View Post
for 1080p is pretty easy to show "major" difference been chroma scaler.
Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.

Quote:
for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.
Studio-provided IMF mezzanines often will be 12-bit 4:4:4. And most pro sources are at least 10-bit 4:2:2. Some of the blender.org projects are available as 16-bit RGB 4:4:4 frames as well.

Game footage is actually pretty good for finding chroma edge cases, since it gets rendered per-pixel and anti-aliasing tech is far from perfect. HUD details like red text in a compass or something can have quite sharp chroma details.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th August 2020, 00:29   #23  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by benwaggoner View Post
Studio-provided IMF mezzanines often will be 12-bit 4:4:4.
But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.
Cary Knoop is offline   Reply With Quote
Old 28th August 2020, 01:51   #24  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 8,115
Quote:
Originally Posted by benwaggoner View Post
Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.
we do these test with real world samples from actual content not test pattern. we used test patterns to but there is s reason for quite a lot of chroma optimised scaler clearly not for test pattern.

for game footage i could just take rimworld make a screen it's extrem obvious because red text is used every where the esport titles should show this very easily to. these are not edge cases it's visible the whole time.

for real world camera footage you nearly always have to edge cases but not everything is made with a camera.
huhn is offline   Reply With Quote
Old 28th August 2020, 11:17   #25  |  Link
foxyshadis
Angel of Night
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,560
Quote:
Originally Posted by huhn View Post
does 4:4:4 decoding really cost more silicon when a hardware decoder can do 8K 4:2:0 and 4:4:4 is limited to 4k in this case?

BTW. freesync and gsync screen work out of the box with pretty old hardware and even entry level screens have often support for it (mostly useless like TVs).
and player seem to support it too. a high end gaming PC is clearly not needed.
https://github.com/mpv-player/mpv/issues/6137

https://forums.blurbusters.com/viewtopic.php?t=3509
There is a silicon cost, in doubling the available memory -- which admittedly isn't really all that expensive anymore, but every bit counts that deep. The bigger cost is the upfront engineering labor that they don't expect any ROI on, so why spend it in the first place?

This might change now that screensharing has suddenly become something everyone is doing, instead of just for the rare business meetings.
foxyshadis is offline   Reply With Quote
Old 28th August 2020, 11:34   #26  |  Link
foxyshadis
Angel of Night
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,560
Quote:
Originally Posted by Cary Knoop View Post
But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.
That's beside the point; 4:4:4 is about keeping all available chroma information, instead of throwing away 75% of it. For naturally captured videos, losing 75% of the chroma is essentially meaningless. Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.
foxyshadis is offline   Reply With Quote
Old 28th August 2020, 22:05   #27  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by foxyshadis View Post
Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.
I disagree with that.

I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.
Cary Knoop is offline   Reply With Quote
Old 29th August 2020, 02:36   #28  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 8,115
and what is not digitally created these days?
huhn is offline   Reply With Quote
Old 30th August 2020, 18:05   #29  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,425
Quote:
Originally Posted by Cary Knoop View Post
I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.
I do wonder how long it will take to discard all that quality damaging baggage from the video standards developed for the analog-converted-to-digital days. That all standard UHD still uses 4:2:0 video level encoding is pretty sad really.

Even YouTube and Netflix et al. are encoding in YUV 4:2:0 video levels.

Probably because the video decoding side has traditionally also been really poorly supported on computers. Until hardware decode became standard (very recently) no one payed attention to how to do things correctly. GPU drivers still cannot get basic standards correct all the time. This means there would have been terrible issues for a lot of people for a while after any change, so no one was ever willing to change.
Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard.

AV1 should take the opportunity to go pure full-range 4:4:4, but they won't.
__________________
madVR options explained

Last edited by Asmodian; 30th August 2020 at 18:08.
Asmodian is offline   Reply With Quote
Old 30th August 2020, 18:52   #30  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by Asmodian View Post
Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard.
But I would venture you actually save bandwidth if you go 4:4:4 and let the encoder do its perceptual compression.

4:2:0 is simply a rather crude form of perceptual compression. By using 4:4:4 you give the encoder the full and much more flexible power on how to compress efficiently.
Cary Knoop is offline   Reply With Quote
Old 31st August 2020, 04:27   #31  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 8,115
that's not what he means decoding 4:4:4 needs double the memory as 4:2:0 the image is now 2 times the size it is as it is.

and full range has other issues as long as YCbCr is used it can create out of range values this could be "fixed" with YCoCg or ICtCp. RGB is not an option.

the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma
huhn is offline   Reply With Quote
Old 31st August 2020, 04:44   #32  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by huhn View Post
the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma
Actually the original gamma OETF/EOTF SD video system was designed to provide an overall 1.2 resulting gamma not a net 1.0 gamma as the document indicated.

It's a jungle out there though: sRGB, Rec709, BT1886.
True gammas, gammas with a linear segment, camera gamma without linear segment.

Last edited by Cary Knoop; 31st August 2020 at 04:49.
Cary Knoop is offline   Reply With Quote
Old 31st August 2020, 18:45   #33  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,871
Quote:
Originally Posted by Cary Knoop View Post
But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.
And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!


The only TRUE 4:4:4 content is computer rendered, where it is actually created at native RGB 4:4:4.

Also, the human visual system is far more sensitive to edges in the luma domain than to chroma overall. While we have highly accurate color vision, we process it with much less spatial and temporal detail than basic black-and-white vision which is what most of our evolutionary ancestors only had. Luma is what keeps you from getting eaten by a tiger stalking you across a treeline. Chroma is what keeps you from eating unripe or poisonous fruit. But seeing precise color detail in something that's moving just isn't something we're wired for.

We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.

Fortunately the misbegotten YUV-9 (one chroma sample per 4x4 block of luma) used in 90's codecs like Sorenson Video and Indeo died. That was definitively TOO supersampled. Especially with 320x240 video, which would have only 80x60 chroma samples. Colored text was a nightmare.

Another classic problem was going from NTSC DV25, which used 4:1:1 subsampling (one chroma per four pixels horizontally) to DVD (4:2:0) which netted out 4:1:0 color, so one chroma per 4x2 block of pixels. Key was to do all motion graphics after DV25, rendering as 4:2:0. Natural images were okay-ish with 4:1:1, as long as all graphics were done with more precision.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 31st August 2020, 18:55   #34  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by benwaggoner View Post
And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!
Perhaps you should pay a bit more attention to what I actually write?

I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design

Last edited by Cary Knoop; 31st August 2020 at 19:04.
Cary Knoop is offline   Reply With Quote
Old 31st August 2020, 18:59   #35  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by benwaggoner View Post
We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.
A clear argumentum ad antiquitatem "we have always done it like that" fallacy.

By leaving the question on how to do the best perceptual compression to the codec on a frame by frame basis you will get the best solution, not by some arbitrary constraint impacting every frame the same way before you even start the compression.

Last edited by Cary Knoop; 31st August 2020 at 19:03.
Cary Knoop is offline   Reply With Quote
Old 31st August 2020, 19:06   #36  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,871
Quote:
Originally Posted by Cary Knoop View Post
That is why I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design
So, like using a 6K sensor for making 4K video? Alas, no one in Hollywood is actually doing post at >4K outside of tech experiments. Whatever the native camera format, it's always turned into a XYZ, RGB, or YUV 4K (or lower) before any real creative work is done. That said, if it's using ACE or ProRes 4444 XQ, it isn't subsampled. That's seen more in higher-budget films, with TV production being mostly done in 4:2:2.

Something thing people often miss is that a 4K sensor with only one color per "pixel" is actually LOWER detail than 4K with 4:2:0, since luma and chroma samples are colocated. going from a 4096x sensor to a 3840x file helps that a bit, even though they are both called "4K."

There is some very interesting alchemy that goes into making a good source out of camera's native formats!
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 31st August 2020, 19:09   #37  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by benwaggoner View Post
So, like using a 6K sensor for making 4K video?
Blackmagic Design just came out with a 12k (non-Bayer) sensor.
Cary Knoop is offline   Reply With Quote
Old 1st September 2020, 19:13   #38  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,871
Quote:
Originally Posted by Cary Knoop View Post
Blackmagic Design just came out with a 12k (non-Bayer) sensor.
What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 1st September 2020, 19:17   #39  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 398
Quote:
Originally Posted by benwaggoner View Post
What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).
It is an RGBW sensor using a 6x6 grid (6xG, 6xB, 6xR plus 18x white pixels).

Here the link to the patent app:
https://patents.google.com/patent/US...ttner&sort=new

Personally I think it is a step in the wrong direction, sure, it may be that this configuration turns out to be superior dynamic range wise, but the demosaicing is more complex (and perhaps even theoretically "wrong").

What we really need is to eventually get rid of Bayer or other sensors that require demosaicing.

Requiring demosaicing, just like interlaced video, chroma subsampling, fixed-bit code values, all those things are not features they are hacks.

Last edited by Cary Knoop; 1st September 2020 at 19:34.
Cary Knoop is offline   Reply With Quote
Old 2nd September 2020, 05:42   #40  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 8,115
i can "just" go and record a lossless 4:4:4 4K RGB game video.
or 1440p/1080p if my disk or CPU can't handle so much throughput and just do the old test of if subsampling actually improves image quality with the same bit rate?

i know this has been tested with x264 back in the days but i can't remember x265.

i mean we can talk a lot but test are the real thing.
huhn is offline   Reply With Quote
Reply

Tags
2160p, bluray, staxrip, upscale, x265

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:19.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.