Transparent encodes of fake 4k blurays

[Cary Knoop]

Originally Posted by huhn

for 1080p is pretty easy to show "major" difference been chroma scaler.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.

I am sure there are 8k and 12k raw samples online you can demosaic to 4:4:4.

[benwaggoner]

Originally Posted by huhn

for 1080p is pretty easy to show "major" difference been chroma scaler.

Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.

for UHD there is no 4:4:4 source i know of so i have to take game footage to show the difference which is again not that hard. games are simply different from movies.

Studio-provided IMF mezzanines often will be 12-bit 4:4:4. And most pro sources are at least 10-bit 4:2:2. Some of the blender.org projects are available as 16-bit RGB 4:4:4 frames as well.

Game footage is actually pretty good for finding chroma edge cases, since it gets rendered per-pixel and anti-aliasing tech is far from perfect. HUD details like red text in a compass or something can have quite sharp chroma details.

[Cary Knoop]

Originally Posted by benwaggoner

Studio-provided IMF mezzanines often will be 12-bit 4:4:4.

But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.

[huhn]

Originally Posted by benwaggoner

Yeah, if it's done wrong it can be very visible. I got paid a good fee in the mid aughts to make chroma upsampling error triggering test patterns.

we do these test with real world samples from actual content not test pattern. we used test patterns to but there is s reason for quite a lot of chroma optimised scaler clearly not for test pattern.

for game footage i could just take rimworld make a screen it's extrem obvious because red text is used every where the esport titles should show this very easily to. these are not edge cases it's visible the whole time.

for real world camera footage you nearly always have to edge cases but not everything is made with a camera.

[foxyshadis]

Originally Posted by huhn

does 4:4:4 decoding really cost more silicon when a hardware decoder can do 8K 4:2:0 and 4:4:4 is limited to 4k in this case?

BTW. freesync and gsync screen work out of the box with pretty old hardware and even entry level screens have often support for it (mostly useless like TVs).
and player seem to support it too. a high end gaming PC is clearly not needed.
https://github.com/mpv-player/mpv/issues/6137

https://forums.blurbusters.com/viewtopic.php?t=3509

There is a silicon cost, in doubling the available memory -- which admittedly isn't really all that expensive anymore, but every bit counts that deep. The bigger cost is the upfront engineering labor that they don't expect any ROI on, so why spend it in the first place?

This might change now that screensharing has suddenly become something everyone is doing, instead of just for the rare business meetings.

[foxyshadis]

Originally Posted by Cary Knoop

But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.

That's beside the point; 4:4:4 is about keeping all available chroma information, instead of throwing away 75% of it. For naturally captured videos, losing 75% of the chroma is essentially meaningless. Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.

[Cary Knoop]

Originally Posted by foxyshadis

Only digitally generated or highly manipulated video benefits from full chroma, so how it's captured and demosaic'd isn't going to matter.

I disagree with that.

I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.

[huhn]

and what is not digitally created these days?

[Asmodian]

Originally Posted by Cary Knoop

I know the "fossil" attitudes in the video industry are still strong but it's time to do away with fixed-bit encodings, video vs data levels, interlacing (it is still done), and chroma subsampling.

I do wonder how long it will take to discard all that quality damaging baggage from the video standards developed for the analog-converted-to-digital days. That all standard UHD still uses 4:2:0 video level encoding is pretty sad really.

Even YouTube and Netflix et al. are encoding in YUV 4:2:0 video levels.

Probably because the video decoding side has traditionally also been really poorly supported on computers. Until hardware decode became standard (very recently) no one payed attention to how to do things correctly. GPU drivers still cannot get basic standards correct all the time. This means there would have been terrible issues for a lot of people for a while after any change, so no one was ever willing to change.
Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard.

AV1 should take the opportunity to go pure full-range 4:4:4, but they won't.

[Cary Knoop]

Originally Posted by Asmodian

Edit: bandwidth costs money, even only in-device bandwidth... probably the real reason 4:2:0 is still standard.

But I would venture you actually save bandwidth if you go 4:4:4 and let the encoder do its perceptual compression.

4:2:0 is simply a rather crude form of perceptual compression. By using 4:4:4 you give the encoder the full and much more flexible power on how to compress efficiently.

[huhn]

that's not what he means decoding 4:4:4 needs double the memory as 4:2:0 the image is now 2 times the size it is as it is.

and full range has other issues as long as YCbCr is used it can create out of range values this could be "fixed" with YCoCg or ICtCp. RGB is not an option.

the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma

[Cary Knoop]

Originally Posted by huhn

the real reason no one fixed this is 4:2:0 is good "enough" for these casual professionals that create charts like these:
https://www.unravel.com.au/understanding-gamma

Actually the original gamma OETF/EOTF SD video system was designed to provide an overall 1.2 resulting gamma not a net 1.0 gamma as the document indicated.

It's a jungle out there though: sRGB, Rec709, BT1886.
True gammas, gammas with a linear segment, camera gamma without linear segment.

[benwaggoner]

Originally Posted by Cary Knoop

But that is not necessarily true 4:4:4.

In order to have true 4K/UHD 4:4:4 the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design.

And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!


The only TRUE 4:4:4 content is computer rendered, where it is actually created at native RGB 4:4:4.

Also, the human visual system is far more sensitive to edges in the luma domain than to chroma overall. While we have highly accurate color vision, we process it with much less spatial and temporal detail than basic black-and-white vision which is what most of our evolutionary ancestors only had. Luma is what keeps you from getting eaten by a tiger stalking you across a treeline. Chroma is what keeps you from eating unripe or poisonous fruit. But seeing precise color detail in something that's moving just isn't something we're wired for.

We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.

Fortunately the misbegotten YUV-9 (one chroma sample per 4x4 block of luma) used in 90's codecs like Sorenson Video and Indeo died. That was definitively TOO supersampled. Especially with 320x240 video, which would have only 80x60 chroma samples. Colored text was a nightmare.

Another classic problem was going from NTSC DV25, which used 4:1:1 subsampling (one chroma per four pixels horizontally) to DVD (4:2:0) which netted out 4:1:0 color, so one chroma per 4x2 block of pixels. Key was to do all motion graphics after DV25, rendering as 4:2:0. Natural images were okay-ish with 4:1:1, as long as all graphics were done with more precision.

[Cary Knoop]

Originally Posted by benwaggoner

And even that's not really accurate. A Bayer pattern has 2 green sensors for every blue and red, mimicing human visual sensitivities. As Y' is over half green, one could argue that there is chroma subsampling at the sensor, even!

Perhaps you should pay a bit more attention to what I actually write?

I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design

[Cary Knoop]

Originally Posted by benwaggoner

We've got some decades of digital image processing under our belt, and we've consistantly seen that 4:2:0 delivers more visual value per bit than 4:4:4 when bitrate is contstrained. Chroma subsampling was part of JPEG, even, at the birth of this technology.

A clear argumentum ad antiquitatem "we have always done it like that" fallacy.

By leaving the question on how to do the best perceptual compression to the codec on a frame by frame basis you will get the best solution, not by some arbitrary constraint impacting every frame the same way before you even start the compression.

[benwaggoner]

Originally Posted by Cary Knoop

That is why I wrote: the camera needs a very high-resolution sensor due to the Bayer (or other) sensor design

So, like using a 6K sensor for making 4K video? Alas, no one in Hollywood is actually doing post at >4K outside of tech experiments. Whatever the native camera format, it's always turned into a XYZ, RGB, or YUV 4K (or lower) before any real creative work is done. That said, if it's using ACE or ProRes 4444 XQ, it isn't subsampled. That's seen more in higher-budget films, with TV production being mostly done in 4:2:2.

Something thing people often miss is that a 4K sensor with only one color per "pixel" is actually LOWER detail than 4K with 4:2:0, since luma and chroma samples are colocated. going from a 4096x sensor to a 3840x file helps that a bit, even though they are both called "4K."

There is some very interesting alchemy that goes into making a good source out of camera's native formats!

[Cary Knoop]

Originally Posted by benwaggoner

So, like using a 6K sensor for making 4K video?

Blackmagic Design just came out with a 12k (non-Bayer) sensor.

[benwaggoner]

Originally Posted by Cary Knoop

Blackmagic Design just came out with a 12k (non-Bayer) sensor.

What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).

[Cary Knoop]

Originally Posted by benwaggoner

What kind of sensor?

Oversampling is always a good thing. Having a Beyer source at the same resolution as a colocated output was okay with SDR output, as the camera's much higher dynamic range provided additional details. But a 4K Beyer sensor for 4K 4:2:0 HDR content can be suboptimal versus capturing more, since the output can use the captured dynamic range.

I doubt it's that material in practice, though, since visually resolving individual 4K pixels is pretty impossible in moving video outside of specific test content. But supersampling can reduce noise some (although having smaller sensor elements adds per-element noise, so it can be pretty complex to model and predict).

It is an RGBW sensor using a 6x6 grid (6xG, 6xB, 6xR plus 18x white pixels).

Here the link to the patent app:
https://patents.google.com/patent/US...ttner&sort=new

Personally I think it is a step in the wrong direction, sure, it may be that this configuration turns out to be superior dynamic range wise, but the demosaicing is more complex (and perhaps even theoretically "wrong").

What we really need is to eventually get rid of Bayer or other sensors that require demosaicing.

Requiring demosaicing, just like interlaced video, chroma subsampling, fixed-bit code values, all those things are not features they are hacks.

[huhn]

i can "just" go and record a lossless 4:4:4 4K RGB game video.
or 1440p/1080p if my disk or CPU can't handle so much throughput and just do the old test of if subsampling actually improves image quality with the same bit rate?

i know this has been tested with x264 back in the days but i can't remember x265.

i mean we can talk a lot but test are the real thing.