Enhanced Compression Model (ECM) / H.267: VVC successor

[birdie]

Exploring the future of video codecs at ITU's 2025 MPEG Workshop, where industry leaders discuss the next decade of video compression technology.

Some murky news on H.266/H.267 and the future of MPEG video codecs.

The video of the conference is here.

MPEG seems not to understand why H.266 adoption has been so bad/slow/non-existent. Looks like no one even raised the question of a very bad licensing situation in regard to H.266.

[Z2697]

It has now gotten to the point like "whatever man, whatever".
AOM will take over.

[benwaggoner]

Originally Posted by birdie

MPEG seems not to understand why H.266 adoption has been so bad/slow/non-existent. Looks like no one even raised the question of a very bad licensing situation in regard to H.266.

It's not that MPEG members don't know. I've talked to plenty of people who were involved in that session, and they know. IP issues were called out as a substantial problem that needs to be addressed in a number of the presentations.

It's more that the MPEG and ITU processes don't allow discussion about IP issues, just technical ones. They literally aren't allowed to consider patents and licensing in designing the technical specifications, in the presumption that "industry will figure it out later, because all stakeholders are invested in a new format's success."

Which obviously isn't true at all these days, but would require some amendments to the group charters to allow.

And from a purely technical perspective, a purely technical approach has good dividends. VVC as a codec outperforms AV1 in compression efficiency with substantially less complex decoder requirements. If VVC had AOM licensing, it would be broadly used today. But it doesn't, and so VVC use is only planned in particular markets (Brazil) that believe they've handed the IP issues for themselves.

[kurkosdr]

Originally Posted by benwaggoner

It's not that MPEG members don't know. I've talked to plenty of people who were involved in that session, and they know. IP issues were called out as a substantial problem that needs to be addressed in a number of the presentations.

It's more that the MPEG and ITU processes don't allow discussion about IP issues, just technical ones. They literally aren't allowed to consider patents and licensing in designing the technical specifications, in the presumption that "industry will figure it out later, because all stakeholders are invested in a new format's success."

These processes were established under the assumption that patent holders want the new standard to achieve mass adoption and want repeat customers. This assumption stopped being true when the patent holders realised they had lost the future of web video to WebM (which would later become AOM) and HEVC would never become universal on the web like AVC is. So, the patent holders focused on milking their remaining customers, more specifically broadcasters and premium video providers like Netflix (who charge subscribers extra for UHD quality and hence want guaranteed compatibility with UHD Smart TVs). The fossil record is a bit unclear here, but it happened around the time Chrome refused to license and bundle an HEVC decoder in Chrome and around the time VP9 was announced (becoming the first "better than H.264" format that wasn't an ISO/ITU format). The walkout from the MPEG LA pool and the founding of the HEVC Advance pool happened shortly after. Sure, the patent holders could have offered better terms for HEVC (for example, no royalties for software decoders and no content fees on free web video) and make HEVC as universal as AVC, but that's not what they wanted to do.

VVC is the same milking attempt but worse, and ECM will be even worse. HEVC will be the last major ISO/ITU format.

[kurkosdr]

Originally Posted by benwaggoner

And from a purely technical perspective, a purely technical approach has good dividends. VVC as a codec outperforms AV1 in compression efficiency with substantially less complex decoder requirements. If VVC had AOM licensing, it would be broadly used today. But it doesn't, and so VVC use is only planned in particular markets (Brazil) that believe they've handed the IP issues for themselves.

Ok, you've saved a couple of cents in silicon costs and you've saved a couple of cents in bitrate costs. Now how much do you have to pay to the 20 different patent licensing entities in VVC? Didn't this undo your savings? Also, silicon costs will go down in time, and so will bitrate costs, but the VVC patent holders will probably try to milk you more in the future. Do you realise why broadcasters like Netflix and Disney+ won't go near VVC?

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.

[FranceBB]

Originally Posted by kurkosdr

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.

The pandemic and the economic crisis that followed afterwards didn't help either, so much so that the 8K productions are extremely rare and aside from a bunch of Japanese documentaries and a few Japanese movies, there's nothing actively produced in 8K. Some productions are shot in 6K and downscaled to 4K and in some rare cases like Das Boot (produced by our colleagues in Sky Deutschland) several shots were performed in 8K and then downscaled to 4K. Not only 8K TVs are nowhere near mass adoption (heck, they're basically nowhere to be seen), but aside from cinema productions and a few tv series, most countries are even finding the cost of making 4K live productions too high and are resorting to FULL HD 50p/60p log 10bit which is then converted to PQ (yes, PQ, not HLG) and upscaled to 4K to "trick" the customers (I'm looking at you FOX Sports). Anyway this is to say that we're nowhere near 8K, so the justification to move from H.265 HEVC to H.266 VVC, even for broadcasters, is really not there just yet considering that the gains at lower resolutions are fewer and that most TVs can't even decode it...

[kurkosdr]

Originally Posted by FranceBB

The pandemic and the economic crisis that followed afterwards didn't help either, so much so that the 8K productions are extremely rare and aside from a bunch of Japanese documentaries and a few Japanese movies, there's nothing actively produced in 8K. Some productions are shot in 6K and downscaled to 4K and in some rare cases like Das Boot (produced by our colleagues in Sky Deutschland) several shots were performed in 8K and then downscaled to 4K. Not only 8K TVs are nowhere near mass adoption (heck, they're basically nowhere to be seen), but aside from cinema productions and a few tv series, most countries are even finding the cost of making 4K live productions too high and are resorting to FULL HD 50p/60p log 10bit which is then converted to PQ (yes, PQ, not HLG) and upscaled to 4K to "trick" the customers (I'm looking at you FOX Sports). Anyway this is to say that we're nowhere near 8K, so the justification to move from H.265 HEVC to H.266 VVC, even for broadcasters, is really not there just yet considering that the gains at lower resolutions are fewer and that most TVs can't even decode it...

I know I will sound like a broken record, but 8K is a solution looking for a problem to solve. The thought process behind 8K is "Hey, you know we can skip cutting this motherglass into four pieces but sell it as one big expensive panel with quadruple the resolution of 4K instead, right? Now, is there a problem we can apply this solution to?" I mean, just how large is your TV and how close are your viewing distances that you need more than 8.3 megapixels of resolution for video?

VVC would have been useful to make 4K HDR bitrates more tolerable, but this is not happening because broadcasters are locked into HEVC for 4K HDR (in the same way broadcasters ignored Mpeg-4 Part 2 because they were locked into Mpeg-2 for SD) and streaming is going to AV1.

8K is useful in video production, since you can slice and dice a video (for example centering shots) and still end up with an effective resolution of at least 4K. So, 8K cameras do solve a real-world problem. But I doubt they capture VVC, cameras typically use their own formats like Redcode.

[benwaggoner]

Originally Posted by kurkosdr

Ok, you've saved a couple of cents in silicon costs and you've saved a couple of cents in bitrate costs. Now how much do you have to pay to the 20 different patent licensing entities in VVC? Didn't this undo your savings? Also, silicon costs will go down in time, and so will bitrate costs, but the VVC patent holders will probably try to milk you more in the future. Do you realise why broadcasters like Netflix and Disney+ won't go near VVC?

Yeah, exactly. Which is why we're seeing AV1 HW heading towards ubiquity and VVC largely missing in action, despite designs being completed a while ago.

Terrestrial and satellite Broadcasters could be a market for VVC, since they have to deal with fixed bitrate allocations decided at the government level and want all the bitrate savings they can get, but they are stuck with HEVC for HDR and 4K HDR content for compatibility reasons and that won't change any time soon.

You mean for backwards compatibility reason? Set top boxes create so much friction for codec updates, since they have to be replaced wholesale to deprecate an old codec type. We saw MPEG-2 lasting a lot longer in those than any other consumer tech.

[benwaggoner]

Originally Posted by kurkosdr

I know I will sound like a broken record, but 8K is a solution looking for a problem to solve. The thought process behind 8K is "Hey, you know we can skip cutting this motherglass into four pieces but sell it as one big expensive panel with quadruple the resolution of 4K instead, right? Now, is there a problem we can apply this solution to?" I mean, just how large is your TV and how close are your viewing distances that you need more than 8.3 megapixels of resolution for video?

To visually resolve >4K signal you need to sit so close to the screen that the viewing angle to the sides of the screen are terrible. Even with expert viewers with unusually acute vision. The average person couldn't tell the difference in an uncompressed A/B test, let alone just watching compressed video (with the same bitrate for 4K and 8K). For real-time encoding at reasonable bitrates, 4K would typically look better due to fewer compression artifacts.

VVC would have been useful to make 4K HDR bitrates more tolerable, but this is not happening because broadcasters are locked into HEVC for 4K HDR (in the same way broadcasters ignored Mpeg-4 Part 2 because they were locked into Mpeg-2 for SD) and streaming is going to AV1.

MPEG-4 part 2 also suffered from codec licensing ambiguity (which inspired AVC to get its act together, a lesson since forgotten) and was around a 20-30% bitrate reduction, not enough to go through the pain of switching codecs for. When AVC came out with sane, clear licensing and 50% reduction, that was worth it.

8K is useful in video production, since you can slice and dice a video (for example centering shots) and still end up with an effective resolution of at least 4K. So, 8K cameras do solve a real-world problem. But I doubt they capture VVC, cameras typically use their own formats like Redcode.

Yeah, oversampling on production can always find a use. But those are going to be in professional formats, in the high end some sort of Beyer pattern recording ala RAW.

Distributing >4K for moving image content that people will watch the entire frame of at once just doesn't make sense.

Digital signage and other applications can find uses.

[hajj_3]

mpeg association looks like they are taking in consideration market conditions not just technical merit for their next gen video codec. Maybe they want 1 patent pool, maybe lower cost, maybe a guarantee of no streaming costs.

Introduction
To help ensure that actions are in accordance with industry needs, MPEG is conducting a ‘market needs’ study for a next-generation video codec. Both MPEG and non-MPEG members are invited to submit comments using this template.

These ‘market needs’ studies have been performed by MPEG in the past for a diverse range of technical areas. The template below is written in a generic way and has not been specifically modified for the specific next-generation video work. Market needs are considered to be complementary information to technical requirements and use cases. This template is requested for work prior to an exploratory stage, and for new amendments, versions or profiles of an existing standard.

source: https://www.mpeg.org/wp-content/uplo...eva/w24910.zip

[Z2697]

Originally Posted by hajj_3

mpeg association looks like they are taking in consideration market conditions not just technical merit for their next gen video codec. Maybe they want 1 patent pool, maybe lower cost, maybe a guarantee of no streaming costs.



source: https://www.mpeg.org/wp-content/uplo...eva/w24910.zip

I think it's just "buzzword".
We thought they should've learned from the disaster of HEVC.
They gave us "EVC baseline" garbage and go "yay, free and better than AVC".

The reality is no one is eagerly asking for a next gen MPEG codec anymore.
I don't think AOM is the one and only 100% pure morally superior good-hearted godlike savior, but at least they are doing better than MPEG.

[ksec]

The reality is no one is eagerly asking for a next gen MPEG codec anymore.

It is not just MPEG video codec. It is any video codec. From internet companies POV.

The thing is, as I have been saying for many years and may be soon a decade. As bandwidth cost continue to drop, the incentive to invest into a new codec also continue to faint away. Because storage cost isn't dropping anywhere near as fast as bandwidth. This is specifically on the Internet and not broadcast. With the new explosion of AI, what the industry predicted bandwidth and port cost price falling would slow down turns out continue to accelerate.

The investment of encoding all videos file into a new Codec, and storing it, which cost CPU cycle and Storage does not make much of ROI to the saving of using less bandwidth. The wait and see approach means adoption continue to be slow. And this limit / slow down hardware maker from implementing it.

I think the bar for a next replacement codec needs to be 80% BD-Rate of x264 while being decoding and encoding efficient. That is a tall order. It may be somewhat counter- intuitive, what used to be internet / technology companies dictating video codec market acceptance in the past 10-15 years, will now leads to TV broadcasting leading or spear heading for changes. Because they are the ones with limited bandwidth, and they will need to pick and choose something that can be served in board casting and internet streaming. Something like Brazil TV 3.0. Hopefully VVC + LCEVC.

I really hope there is an JPEXG-XL moment that shows how something could be so much better.

[Z2697]

Originally Posted by ksec

It is not just MPEG video codec. It is any video codec. From internet companies POV.

The thing is, as I have been saying for many years and may be soon a decade. As bandwidth cost continue to drop, the incentive to invest into a new codec also continue to faint away. Because storage cost isn't dropping anywhere near as fast as bandwidth. This is specifically on the Internet and not broadcast. With the new explosion of AI, what the industry predicted bandwidth and port cost price falling would slow down turns out continue to accelerate.

The investment of encoding all videos file into a new Codec, and storing it, which cost CPU cycle and Storage does not make much of ROI to the saving of using less bandwidth. The wait and see approach means adoption continue to be slow. And this limit / slow down hardware maker from implementing it.

I think the bar for a next replacement codec needs to be 80% BD-Rate of x264 while being decoding and encoding efficient. That is a tall order. It may be somewhat counter- intuitive, what used to be internet / technology companies dictating video codec market acceptance in the past 10-15 years, will now leads to TV broadcasting leading or spear heading for changes. Because they are the ones with limited bandwidth, and they will need to pick and choose something that can be served in board casting and internet streaming. Something like Brazil TV 3.0. Hopefully VVC + LCEVC.

I really hope there is an JPEXG-XL moment that shows how something could be so much better.

JPEG-XL is so much better only because we've been stuck with JPEG-1 for decades, plus other "modern" choices we have today are basically some frankenstein weird stuff.

[nhw_pulsar]

Originally Posted by Z2697

JPEG-XL is so much better only because we've been stuck with JPEG-1 for decades, plus other "new" choices we have today are basically some frankenstein weird stuff.

Hi,

When you speak about these "new" choices that resemble frankenstein weird stuff, are you talking about the other new choices on the Internet that are WebP, AVIF, HEIC? If so, you think they're weird because they're based on a video codec (JPEG XL would be the evolution of JPEG image codec, and WebP is VP8 intra and AVIF and HEIC are VP10/HEVC intra)?

Or with these "new" choices, you make reference to other (weird) stuff?

Cheers,
Raphael

[Z2697]

Originally Posted by nhw_pulsar

Hi,

When you speak about these "new" choices that resemble frankenstein weird stuff, are you talking about the other new choices on the Internet that are WebP, AVIF, HEIC? If so, you think they're weird because they're based on a video codec (JPEG XL would be the evolution of JPEG image codec, and WebP is VP8 intra and AVIF and HEIC are VP10/HEVC intra)?

Or with these "new" choices, you make reference to other (weird) stuff?

Cheers,
Raphael

Yeah, I mean the video codec based format. It might be a bit exaggerated to call them frankenstein. But they are inferior to JPEG-XL in almost every aspect.

But the "distance" between JPEG-1 and JPEG-XL is the key. It's like comparing MPEG2 to HEVC.

[nhw_pulsar]

Originally Posted by Z2697

Yeah, I mean the video codec based format. It might be a bit exaggerated to call them frankenstein. But they are inferior to JPEG-XL in almost every aspect.

But the "distance" between JPEG-1 and JPEG-XL is the key. It's like comparing MPEG2 to HEVC.

Ok, thank you for your reply.


Cheers,
Raphael

[birdie]

JXL has not had its moment yet.

Zero major web browsers support it out of the box.

Firefox sort of supports it except it needs to be enabled via about:config that is hidden for mobile users and nearly impossible to discover for desktop users.

[GeoffreyA]

It's baffling that we've got a premier image format collecting dust, whilst the "Frankenformats" are strutting their feathers on the runway of the web.

[kurkosdr]

Originally Posted by GeoffreyA

It's baffling that we've got a premier image format collecting dust, whilst the "Frankenformats" are strutting their feathers on the runway of the web.

Because the "Frankenformats" were the first viable royalty-free successor to JPEG, so they captured the market. Also, the "Frankenformats" are widely used by Google, so if some NPE comes out of the woodwork and starts demanding content fees, they will realistically go after Google or some other big player first. Instead, if Google is not using JPEG XL and your website does, any NPEs will go after you.

It's what killed EVC too: VP9 and AV1 captured the market EVC was going for, and using EVC means painting a big red target on your back for NPEs looking to sue for content fees, since no big player uses EVC.

[GeoffreyA]

Originally Posted by kurkosdr

Because the "Frankenformats" were the first viable royalty-free successor to JPEG, so they captured the market. Also, the "Frankenformats" are widely used by Google, so if some NPE comes out of the woodwork and starts demanding content fees, they will realistically go after Google or some other big player first. Instead, if Google is not using JPEG XL and your website does, any NPEs will go after you.

It's what killed EVC too: VP9 and AV1 captured the market EVC was going for, and using EVC means painting a big red target on your back for NPEs looking to sue for content fees, since no big player uses EVC.

Good points. And well said.