x266 VVC Encoder

[ksec]

Originally Posted by birdie

The last webinar questions have been answered.

I'm a little bit dumbfounded because seemingly some questions have been misunderstood, some others have very basic and uninformative answers.

Well I don’t think they misunderstood any question. I think they intentionally avoid answering it directly. It is just part of PR.

I would love to test x266. Both as still image like HEIC and video. At least both Mediatek and Qualcomm are on board with VVC. If Apple follows through that is potentially 95%+ of Mobile market.

[benwaggoner]

Originally Posted by birdie

The last webinar questions have been answered.

I'm a little bit dumbfounded because seemingly some questions have been misunderstood, some others have very basic and uninformative answers.

Is Ultraziq a rebranded/enhanced implementation of UHDKit?

[Blue_MiSfit]

It sure seems to be.

[kurkosdr]

Originally Posted by birdie

The last webinar questions have been answered.

So, we finally have a date: H2 2023. This means that everyone will have to withhold complaints about when x266 will be released to 1st Jan 2024.

BTW I also think some of the answers are being evasive, but people need to understand that corporations can share only what they can share. But they shared a release date, so that's good.

Originally Posted by ksec

At least both Mediatek and Qualcomm are on board with VVC. If Apple follows through that is potentially 95%+ of Mobile market.

Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.

[benwaggoner]

Originally Posted by kurkosdr

Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.

Chrome added support for HEVC in the fall, if a system HEVC decoder is available. So it's plausible they'll do the same for VVC in a few year (and plausible they won't).

[kurkosdr]

Originally Posted by benwaggoner

Chrome added support for HEVC in the fall, if a system HEVC decoder is available.

So, Google decided to screw over Linux and Mozilla after initially joining them in their decision to not implement patent-encumbered formats newer than H.264. Why I am not surprised?

[benwaggoner]

Originally Posted by kurkosdr

So, Google decided to screw over Linux and Mozilla after initially joining them in their decision to not implement patent-encumbered formats newer than H.264. Why I am not surprised?

I think it is more a matter of prioritizing users over ideological purity. AV1 hasn't really caught on for HDR content, nor have practical benefits of it over HEVC been demonstrated. Everything but browsers can do HEVC these days, and the browser HDR market hasn't been big enough to justify spending 2-4x more time to encode AV1 HDR files for them. Allowing HEVC passthrough to system decoders doesn't require any patent engagement by Google, and adds compatibility with a whole lot of premium content that would otherwise not be available in a browser versus a standalone app.

And Linux systems certainly do support HEVC playback. There's not a build-in software player, but CPU/GPU HW HEVC decoders can certainly be accessed under lots of Linux distributions.

I can't speak to Mozilla's plan, but they certainly could add the "we won't have a decoder, but we'll pass HEVC on to one that exists."

Chrome and Firefox supported HEVC passthrough around a decade ago, as codec passthrough used to be done by default if a decoder was available. That functionality was explicitly blocked by the browsers later.

Supporting VVC in the same way will be quite straightforward once we start seeing PC systems with VVC decoders (I'd expect some PC CPU or GPUs with support to launch by late 2024.).

[ksec]

Originally Posted by kurkosdr

Out of curiosity, what will be the case for this? Firefox and Chrome (which are the majority of the browser market) have decided to not adopt any ISO/ITU standard beyond H.264, so that rules web video out, and broadcast and Blu-Ray UHD content belongs to HEVC (too much of an installed base to revert now, imagine if MPEG4 ASP had become the standard for FullHD, that's what essentially happened with UHD and HEVC), which means VVC-encoded videos will be unplayable in the majority of UHD televisions. My only guess is 8K UHD content, which is generally assumed to be VVC-encoded, but this raises the question if smartphones can encode 8K UHD content, in VVC.

Future TV broadcast standard like Brazil have chosen VVC. HDR and 4K ( It doesn't even need to be 8K ) Broadcast will also likely be using a new video codec standard. There are also other Streaming Services which doesn't rely on Web. Especially outside of EUR and North America. Even in 4K HDR Content VVC will still be much better than HEVC.

[FranceBB]

Originally Posted by benwaggoner

Allowing HEVC passthrough to system decoders doesn't require any patent engagement by Google, and adds compatibility with a whole lot of premium content that would otherwise not be available in a browser versus a standalone app.

Yep, that's exactly it, in fact I really don't see anything wrong with this approach. After all, if a user has an H.265 capable hardware decoder in his GPU, it means that the GPU manufacturer already paid the royalty to be able to decode it (and therefore included it in the final GPU price that the user paid). In other words, given that the user has *theoretically* already paid the fee, having such a functionality artificially blocked inside the browser wouldn't make any sense, so I'm totally in favor of this approach. I, myself, have an H.265 capable GPU in terms of decoding and indeed chrome://gpu shows it:



and indeed if I try to play an H.265 stream it works like a charm:




I'm using the DASH reference player: https://reference.dashif.org/dash.js...yer/index.html to test with the following stream: https://dash.akamaized.net/dash264/T.../MultiRate.mpd

[kurkosdr]

Originally Posted by benwaggoner

And Linux systems certainly do support HEVC playback. There's not a build-in software player, but CPU/GPU HW HEVC decoders can certainly be accessed under lots of Linux distributions.

No, not consistently at least. HEVC hardware decoding acceleration on Desktop Linux depends on the driver used, since GPU vendors are unclear on whether the royalty is paid on the hardware sale or on the driver download, which scares away open-source driver authors. Fedora had to disable hardware decoding of patent-encumbered formats for that very reason. And then there is the problem of perfectly functional GPUs which however don't have HEVC hardware decoding acceleration.

Google decided to screw Desktop Linux users over and treat them as lesser Chrome users after promising they wouldn't do that (in order to drum up support for VP9). Why am I not surprised? The next step is Mozilla getting blamed for not treating Desktop Linux users as lesser Firefox users too.

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.

[birdie]

Originally Posted by kurkosdr

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.

I do, the only video format at the time being numb 8bit palette GIF.

Neither APNG, nor MNG ever took off.

[rwill]

Originally Posted by birdie

I do, the only video format at the time being numb 8bit palette GIF.

Neither APNG, nor MNG ever took off.

That was because GIF being a completely open format and totally not patent encumbered was a perfect match for the Web of the People. Right comrades ?

*edit*
So whats going on with x266? Cant be that hard to write a H.266 encoder. Looking at their feature roadmap I think it would take me just 2-3 man-months to get to a working release....

[kurkosdr]

Originally Posted by rwill

That was because GIF being a completely open format and totally not patent encumbered was a perfect match for the Web of the People.

I kind of expected this comment, despite being plain wrong: The GIF compression patent was a patent ambush. In other words, the patent holder came to assert and collect after the format had been already widely implemented. The royalty-free PNG was invented shortly afterwards precisely because the web was always meant to be universally accessible, but inertia was king (as always). After the GIF compression patent expired, universal accessibility was restored. Then YouTube made the Flash Player plugin and H.264 mandatory. Even today, the legacy of Flash Player lives on because H.264 was implemented at the browser level in order to encourage websites to move on from Flash Player. But what's the point of HEVC on the web? A slightly worse format than AV1 that is patent-encumbered? And has the patent mess of HEVC been sorted out yet or there are still multiple patent pools that you need to negotiate with? And how many patent pools exactly are essential to HEVC this week? What a mess. But again, I am not surprised Google proved to be a backstabbing company. They are an ad agency masquerading as a tech company after all.

[FranceBB]

Originally Posted by kurkosdr

Fedora had to disable hardware decoding of patent-encumbered formats for that very reason. And then there is the problem of perfectly functional GPUs which however don't have HEVC hardware decoding acceleration.

This is a video shot with my Google Pixel 6 Pro in H.265 the other day:

General
Complete name : /home/FranceBB/Downloads/PXL_20230309_182128484.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 102 MiB
Duration : 13 s 592 ms
Overall bit rate : 63.1 Mb/s
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43
xyz : +45.4339+9.2417/

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6.1@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 13 s 589 ms
Bit rate : 62.8 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 60.000 FPS
Minimum frame rate : 45.662 FPS
Maximum frame rate : 93.652 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.126
Stream size : 102 MiB (100%)
Title : VideoHandle
Language : English
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 13 s 592 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 319 KiB (0%)
Title : SoundHandle
Language : English
Encoded date : UTC 2023-03-09 18:21:43
Tagged date : UTC 2023-03-09 18:21:43

Other
Type : meta
Duration : 13 s 589 ms
Bit rate mode : Variable

This is me playing it with hardware decoding through MPV on my Fedora 37 x64: https://i.imgur.com/YXrv2At.png


as you can see, being an 8bit yv12 (4:2:0 planar) Full Range BT709 60p UHD video, it's being hardware decoded by the GPU in nv12 using vaapi.


To do this, you need to add the following lines in mpv.conf

#Here we enable hardware decoding for everything
hwdec=auto
hwdec-codecs=all
vd-lavc-check-hw-profile=no

#Here we fallback to software if there's no hardware decoding
vd-lavc-software-fallback=yes

#Here we set the video sync (useful with GNOME)
video-sync=display-resample-desync

in case anyone needs it, here's my full mpv.conf that needs to be placed in /etc/mpv: Link


As far as chrome is concerned, unless you need to stay on the official Google Chrome, you can use chromium, in particular chromium-freeworld from RPM Fusion which still includes all the decoders which are not included in the standard version of chromium afaik.

https://github.com/rpmfusion/chromium-freeworld

just

sudo dnf install chromium-freeworld

and that's it.

[kurkosdr]

Originally Posted by FranceBB

This is a video shot with my Google Pixel 6 Pro in H.265 the other day:




This is me playing it with hardware decoding through MPV on my Fedora 37 x64: https://i.imgur.com/YXrv2At.png


as you can see, being an 8bit yv12 (4:2:0 planar) Full Range BT709 60p UHD video, it's being hardware decoded by the GPU in nv12 using vaapi.


To do this, you need to add the following lines in mpv.conf

#Here we enable hardware decoding for everything
hwdec=auto
hwdec-codecs=all
vd-lavc-check-hw-profile=no

#Here we fallback to software if there's no hardware decoding
vd-lavc-software-fallback=yes

#Here we set the video sync (useful with GNOME)
video-sync=display-resample-desync

in case anyone needs it, here's my full mpv.conf that needs to be placed in /etc/mpv: Link


As far as chrome is concerned, unless you need to stay on the official Google Chrome, you can use chromium, in particular chromium-freeworld from RPM Fusion which still includes all the decoders which are not included in the standard version of chromium afaik.

https://github.com/rpmfusion/chromium-freeworld

just

sudo dnf install chromium-freeworld

and that's it.

Ok, how did you do this? Proprietary Nvidia or AMD drivers? Not everyone has an Nvidia or AMD GPU, most systems have Intel GPUs. Fedora with the default drivers can't decode HEVC even if the GPU has hardware acceleration:
https://www.reddit.com/r/Fedora/comm...ardware_video/
This is due to GPU vendors' unwillingness to clarify whether the royalty is paid on the hardware sale or on the driver download.

Also, Chromium can't sync to Google Account anymore so it's useless for most people (another little bit of backstabbing by Google, surprise surprise).

Please note that while I don't hate HEVC or VVC per se, I hate those HEVC and VVC fanboys that try to downplay the massive IPR problems of those codecs (and how this could affect the universal accessibility of the web) by pointing to system codecs as the solution, conveniently ignoring the fact system codec support is inconsistent and hence not universal. And yes, HEVC and VVC do have massive IPR problems: Again, how many patent pools exactly are essential to HEVC this week? Same for VVC?

[FranceBB]

Originally Posted by kurkosdr

Ok, how did you do this? Proprietary Nvidia or AMD drivers?

I totally get the pain of using proprietary NVIDIA drivers on Linux, particularly 'cause for me NVIDIA drivers are almost always broken at every kernel update and the fact that NVIDIA isn't willing to open source a damn thing is pretty annoying as it left the Nouveau project guys in the dark, with GPUs running at the lowest possible clock speeds with no hope for re-clocking. Sure, the latest changes bring a bit of hope as they would allow for some GPUs to be reclocked, however it only works for newer GPUs so anyone with a 900 series like me will be left in the dark with either a buggy proprietary driver or a slow unusable open source one.

Leaving this sad chapter aside, that thing was actually done from my old 2016 era laptop, which has an Intel i7 6700HQ (which has the HD Graphics 530 inside it) and an NVIDIA GTX950M. The latter runs on Nouveau drivers and is almost never used as it's totally useless: official NVIDIA drivers are almost always broken while Nouveau drivers don't allow reclocking so...



Anyway, long story short, the test was actually run using the HD Graphics 530 which can decode H.265 8bit (but not 10bit, why Intel, whyyyyyyyyy?!) via hardware and it did actually work. Of course, if I use the official Google Chrome and not the freeworld chromium version, on the other hand, the screen will stay pitch black with just the audio playing.

Originally Posted by kurkosdr

Fedora with the default drivers can't decode HEVC even if the GPU has hardware acceleration

I asked DBelton from the Fedora community and he helped me long time ago. I can't remember which Intel drivers and decoders he and Marko made me install through both the command line and Fedy, but the whole thing worked and it's still working today. You should definitely go to the Fedora Forum and ask, they'll help you just like they helped me back in 2016.

Originally Posted by kurkosdr

Also, Chromium can't sync to Google Account anymore so it's useless for most people (another little bit of backstabbing by Google, surprise surprise).

I'm totally with you on this, to be fair.
That has been one of the things that annoyed me incredibly 'cause it was a bit of a stab in the back to all the open source developers who worked on the various chromium forks.
I have some friends of mine who actively contribute and when that thing happened they didn't really take it well.
I, myself, didn't take it well either 'cause on the little Windows XP community I'm part of we have some people providing some constantly backported version of chromium (the last one being 108) and of course I haven't been able to sync my account ever since Google blocked it for third party forks...

Originally Posted by kurkosdr

Please note that while I don't hate HEVC or VVC per se, I hate the HEVC and VVC fanboys that try to downplay the massive IPR problems of those codecs

I mean, this is Doom9 not a reddit subforum, so professional people are here, so it's unlikely you'll ever meet any fanboy and I can assure you neither me nor Ben is.
But again, MPEG codecs have always been here and they're here to stay, so even if you were to get a couple more people on Doom9 jumping on the AV1 (and possibly AV2) bandwagon, it would change literally nothing (I know it's hard, but it's true). I mean, think about every 8K broadcaster, every future 8K BD manufacturer etc, what do you think it's gonna happen? Love it or hate it, H.266 VVC will be a thing just like H.265, H.264 and MPEG-2 have and there will be hardware encoders, hardware decoders, playback ports for playout systems and indeed GPUs manufacturers supporting those for consumer-tier profiles (NVIDIA almost definitely and then Intel and probably AMD) as well as hardware decoding support in mobile phones.

This is inevitable, it is going to happen and, in my opinion, it's not a bad thing.


By the way, to prove that I'm not biased in any way, even though my job is literally to create mezzanine files which are by definition going to be MPEG codec based (like XDCAM-50 and XAVC Intra Class 300) and will be played on hardware playback ports in playout systems, I did create (very recently) some AV1 files too for distribution as I was asked to do so for a few trailers (around 50 clips) that were gonna be played over the internet on web pages of our website. So, you see, although AV1 (and in the future AV2) might become a web thing, I can almost definitely guarantee you that it will never happen for linear broadcasting anywhere in the world and that's just the way it is.

[kurkosdr]

Originally Posted by ksec

Future TV broadcast standard like Brazil have chosen VVC. HDR and 4K ( It doesn't even need to be 8K ) Broadcast will also likely be using a new video codec standard. There are also other Streaming Services which doesn't rely on Web. Especially outside of EUR and North America. Even in 4K HDR Content VVC will still be much better than HEVC.

With some rare exceptions like Brazil, most countries chose HEVC for their UHD broadcasts years ago, so VVC for broadcasting is irrelevant for most parts of the world. The following is a good example:
https://www.google.com/search?q=site...itrate.com+vvc
https://www.google.com/search?q=site...trate.com+hevc

The only broadcasting area I see VVC succeeding is 8K UHD content, which will probably be 2-3 channels per satellite or so, considering how scarce bitrate is in most satellites. Irrelevant in the big scheme of things, but still technically interesting.

Streaming boxes could be a big win for VVC, but it faces competition from AV1 and the existing HEVC installed base. That's a space to watch.

[kurkosdr]

Originally Posted by FranceBB

I asked DBelton from the Fedora community and he helped me long time ago. I can't remember which Intel drivers and decoders he and Marko made me install through both the command line and Fedy, but the whole thing worked and it's still working today. You should definitely go to the Fedora Forum and ask, they'll help you just like they helped me back in 2016.

That's why I asked what are you using btw, because you have to do at least some things to get HEVC working in Fedora (and other Desktop Linux distros). Having to install codecs is the problem here, because officially recommending those codecs to people can be considered as "inducing" patent infringement. It's why Fedora can't show a pop-up saying "run the following commands to install such and such codec".

This is the problem I am highlighting here: Once you have to tell people that they need to have this and that, and can't even tell them how, the whole universal accessibility concept of the web is lost.

[benwaggoner]

Originally Posted by kurkosdr

No, not consistently at least.

Well, it is Linux we're talking about there's not a lot it has done consistently across distributions in terms of advanced digital media and graphical stuff.

Google decided to screw Desktop Linux users over and treat them as lesser Chrome users after promising they wouldn't do that (in order to drum up support for VP9). Why am I not surprised? The next step is Mozilla getting blamed for not treating Desktop Linux users as lesser Firefox users too.

Remember when the web was universally accessible and not subject to whether you've gone through the MPEG LA and Access Advance tollbooths? I do.

I certainly don't remember that for digital video, ever. I spent years making a lot of money encoding RealVideo, Windows Media, and QuickTime versions of the same content for the web as plenty of customers only had one of the three. When we got to more universal playback, it was due to the ubiquity of Flash's H.263 decoder support. We did have a while where browsers became powerful enough at H.264 + AAC-LC because universal enough that a single file could play on >99% of web browsers, but it was only a few years between that becoming reliable and HDR (with a 10-bit encoding requirement) becoming important.

Browsers have been about the only place where HEVC hasn't been universal the last five years.

[ksec]

Originally Posted by kurkosdr

With some rare exceptions like Brazil, most countries chose HEVC for their UHD broadcasts years ago, so VVC for broadcasting is irrelevant for most parts of the world. The following is a good example:
https://www.google.com/search?q=site...itrate.com+vvc
https://www.google.com/search?q=site...trate.com+hevc

The only broadcasting area I see VVC succeeding is 8K UHD content, which will probably be 2-3 channels per satellite or so, considering how scarce bitrate is in most satellites. Irrelevant in the big scheme of things, but still technically interesting.

Streaming boxes could be a big win for VVC, but it faces competition from AV1 and the existing HEVC installed base. That's a space to watch.

Because you are comparing the current state and future state. The world is much bigger than North America and EUR. VVC are already in use in India. And China is getting ready too.