Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th January 2019, 02:22   #1381  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 447
Quote:
Originally Posted by utack View Post
A new Speech on AV1 by Tim Terriberry
https://www.youtube.com/watch?v=qubPzBcYCTw
Kind of amusing that, for something about new fancy-pants video codecs, the video itself has telecine judder (it's been telecine'd from 25fps to 30fps).
__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       
Nintendo Maniac 64 is offline   Reply With Quote
Old 25th January 2019, 08:07   #1382  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Quote:
Originally Posted by utack View Post
A new Speech on AV1 by Tim Terriberry
https://www.youtube.com/watch?v=qubPzBcYCTw
Tim is a smart engineer, but engineers typically aren't well equipped to present legal opinions / advice (from 2 min to 9 min). Nobody pays for patents twice. If you license from both MPEG LA and HEVC Advance, the companies that are in both only get paid once. The patent chart Tim is using is out of date and inaccurate. Fraunhofer sold their patents to GE, which is why GE has HEVC patents. Canon licenses their patents through MPEG LA. Velos Media has told hundreds of companies what they charge, and they've signed many companies to their license program.

The truth is that every competitive device that supports video now supports HEVC in hardware. Billions of devices, with billions more sold each year. Most every TV, smartphone, tablet or connected set-top box (including Google Chromecast Ultra). If the patent situation were really untenable, Apple, Samsung, LG, Sony, Amazon, Google, GoPro, Roku and hundreds of other device OEMs wouldn't be incorporating HEVC in their devices. And we wouldn't be watching 4K HDR HEVC movies from Netflix, Amazon, Hulu, Vudu, and Apple. If you want another perspective, I gave a talk at the SF Video meetup on the topic... https://www.youtube.com/watch?v=vgE8-4rcXl0

The Alliance for Open Media isn't the only group working to deal with the difficulties of licensing patents for industry standards. MPEG is dealing with it. The Media Coding Industry Forum is dealing with it. And outside firms like Unified Patents, with their Video Codec Zone (specifically focused on HEVC) are dealing with it. It's an ongoing challenge (both for HEVC and for new standards in development), but it's being dealt with.

On the other hand, I'm blazing along at less than 1 frame per minute of 1080P with aomenc cpu-used=1 on a fast Core i7-7820X (with hyperthreading disabled, for the fastest possible single-threaded performance). The resulting videos are roughly on par with my HEVC encodes at identical bit rates (sometimes better, but very often worse). They're clean, but soft and lacking detail.

Last edited by TomV; 25th January 2019 at 08:11.
TomV is offline   Reply With Quote
Old 26th January 2019, 22:10   #1383  |  Link
TD-Linux
Registered User
 
Join Date: Aug 2015
Posts: 34
Quote:
Originally Posted by benwaggoner View Post
Do we have data on quality @ perf @ bitrate
Here's a link to AWCY as shown in Tim's presentation:

https://beta.arewecompressedyet.com/...y-525f981376bd

tl;dr it is better than x264 at every bitrate, but still worse that libvpx VP9. It is also currently about 10x slower than x264, which is blazing fast compared to libaom but still has a lot of room for improvement.
TD-Linux is offline   Reply With Quote
Old 26th January 2019, 22:56   #1384  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
AWCY doesn’t include any metrics that are well demonstrated to be able to finely discriminate between quality of different encoders and codecs. VMAF is the least-bad we’ve ever had, but can still be off quite a bit for individual clips, especially if the use codec features or psychovisual optimizations that weren’t included in their test clips. For example, VMAF is bad at rating effectiveness of low-Luna adaptive quant, I speculate because they didn’t include any clips that used different ways to do that in their testing. VMAF is a very impressive effort, but it is not magic. Like all machine learning aystems, it tried to predict what a human would answer given complex input, based on a. large set of example inputs and answers. But I’d it doesn’t have human input for some kinds of inputs, the validity of its predicted ratings for those inputs is unpredictable at best.

Also, the value of mean or even harmonic mean of per-frame scores is limited for clips much more than 10 seconds. A movie encoded in CBR and a VBR encode at the same ABR might up with the same mean score per frame, but the VBR would be strongly preferred by viewers as it offers consistent quality, with the worse sections being a lot better than the worst in a CBR encode.

Comparing psychovisual optimizations, rate control, and significantly different encoders tools requires subjective double-one testing before any confidence in objective meassures’ applicability.

Net-net: you can’t know how good video looks without real people looking at it when techniques are used that weren’t incorporated in an objective metric. If we see a high correlation between MOS and VMAF for a new technique/codec, then we can start trusting that metric.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 27th January 2019, 15:30   #1385  |  Link
utack
Registered User
 
Join Date: Apr 2018
Posts: 63
Does anyone know what the deal with Qualcomm is?
Is the YouTube rollout and Netfix talk pushing them towards making a hardware decoder, or do they have some interests in MPEG doing well and will try to delay AV1 support in phones for a while?
utack is offline   Reply With Quote
Old 28th January 2019, 18:25   #1386  |  Link
Djfe
Registered User
 
Join Date: May 2014
Posts: 24
A rendered 8k video on YouTube with lots of flickering (epilepsy warning), details like rain drops on a helmet etc.
https://youtu.be/fOWsamMv_v4
Maybe good for comparing av1 to vp9 on YouTube (up to 480p currently)
Once YouTube gets better encoders anyways

obvious already: more blurred but definitely less blocky and less obvious artifacts)
1:30min into the video is probably the best place to compare (and the hardest part for their av1 implementation so far at that bitrate)
Djfe is offline   Reply With Quote
Old 28th January 2019, 20:37   #1387  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Djfe View Post
A rendered 8k video on YouTube with lots of flickering (epilepsy warning), details like rain drops on a helmet etc.
https://youtu.be/fOWsamMv_v4
Maybe good for comparing av1 to vp9 on YouTube (up to 480p currently)
Once YouTube gets better encoders anyways

obvious already: more blurred but definitely less blocky and less obvious artifacts)
1:30min into the video is probably the best place to compare (and the hardest part for their av1 implementation so far at that bitrate)
Are you getting at AV1 encode at 480p and below somehow? It shows as VP9 for me at every bitrate (using Chrome).

That is a very interesting clip from a compression perspective. It'll really stress weighted prediction (all those strobes) and adaptive quantization (intense variation in frequency distribution). Tons of value from intraframe prediction.

The VP9 encode is not doing well; at 8K scaled down to my 4K monitor there's lots of blocking and banding issues on the guy. I don't have an immediate intuition for how much is limitations in VP9 versus libvpx. That's a kind of content not in the standard libraries of clips encoders get tuned against.

I would expect libaom to do pretty well against it at a slow preset, as libaom is doing a pretty broad mode search with its myriad tools. So it might find lots of oddball methods that work well with this clip. Probably a big gap between slower and faster modes.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th January 2019, 20:51   #1388  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 447
Quote:
Originally Posted by benwaggoner View Post
Are you getting at AV1 encode at 480p and below somehow? It shows as VP9 for me at every bitrate (using Chrome).
Don't you have to opt into using AV1 on YouTube?


Anyway, I can definitely confirm via youtube-dl that AV1 (listed as av01) encodes do in fact exist for that video at 480p resolution and lower:

__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       
Nintendo Maniac 64 is offline   Reply With Quote
Old 28th January 2019, 20:57   #1389  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Nintendo Maniac 64 View Post
Don't you have to opt into using AV1 on YouTube?


Anyway, I can definitely confirm via youtube-dl that AV1 (listed as av01) encodes do in fact exist for that video at 480p resolution and lower:
Yes, it can be set here: https://www.youtube.com/testtube

Now I need to figure out how to do side/by/side in different codecs. Worth comparing to the x264 encodes as well.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th January 2019, 21:51   #1390  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by utack View Post
Does anyone know what the deal with Qualcomm is?
Is the YouTube rollout and Netfix talk pushing them towards making a hardware decoder, or do they have some interests in MPEG doing well and will try to delay AV1 support in phones for a while?
I don’t know anything about Qualcomm specifically, but it can take quite a while to go from final spec to design to tape-out to samples to full-scale fab to products launching with a new SoC.

It was being generally discussed in the industry that AV1’s bitstream finalization delays caused chipmakers to miss the 2019 product design window. A HW accelerated decoder is a lot more flexible, but fixed-function decoder needs to be RIGHT. Small product flaws can wind up impact the entire industry for years. And the combination of video decode and DRM is complex with very high functional requirements.

And I’ve heard that implementing AV1 in hardware is more complex than anticipated, due to relatively low parallelism opportunities and how many discreet tools can get applied to any given pixel. Getting a decoder running on a low-power chip is quite diffeeenr than with 1-2 very fast x64 threads. Say what you will about the MPEG process, but it is good at constraining decode complexity for software and hardware.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th January 2019, 23:39   #1391  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 447
Quote:
Originally Posted by benwaggoner View Post
Now I need to figure out how to do side/by/side in different codecs.
Download each individual video stream via youtube-dl and then play them back in their own video player program window?
__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       
Nintendo Maniac 64 is offline   Reply With Quote
Old 29th January 2019, 18:28   #1392  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 56
libavfilter could be utilized to perform some sort of [0:v]crop=in_w/2:in_h:0:0[VL];[1:v]crop=in_w/2:in_h:in_w/2:0[VR];[VL][VR]hstack stuff
alex1399 is offline   Reply With Quote
Old 29th January 2019, 23:39   #1393  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 729
Quote:
Originally Posted by benwaggoner View Post
AWCY doesn’t include any metrics that are well demonstrated to be able to finely discriminate between quality of different encoders and codecs. VMAF is the least-bad we’ve ever had, but can still be off quite a bit for individual clips, especially if the use codec features or psychovisual optimizations that weren’t included in their test clips.
Isn't there also the possibility that there's a sort of implicit "training" for this metric included in one codec/encoder and not the other?

I don't know whether VMAF was used in some x265 tuning, but given the age of all the significant parts of x264 codebase, I am fairly sure that there has been no attempts to do this.

Meanwhile VMAF was IIRC used during development of Daala and AV1 and maybe AOMenc/Rav1e? In that case, there could be some inherent bias in the metric towards those codecs that would then add some imaginary advantage above their real compression quality into the numbers, when measured by VMAF. Simply because their output was implicitly tuned to get better VMAF, because VMAF was used to test new tools/analysis/RDO and so on.
mandarinka is offline   Reply With Quote
Old 30th January 2019, 00:20   #1394  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by mandarinka View Post
Isn't there also the possibility that there's a sort of implicit "training" for this metric included in one codec/encoder and not the other?
It is an inevitability. Generally the utility of a metric goes down once it is codified, because people start optimizing for that metric instead of the subjective quality that metric approximates. So the correlation of the metric with subjective ratings becomes weaker, as metric-specific tricks get implemented.

Quote:
I don't know whether VMAF was used in some x265 tuning, but given the age of all the significant parts of x264 codebase, I am fairly sure that there has been no attempts to do this.
x265 was around long before VMAF, and a VMAF useful for UHD has only been around a few months. Libaom seems have been getting a lot more tuning-by-VMAF.

Quote:
Meanwhile VMAF was IIRC used during development of Daala and AV1 and maybe AOMenc/Rav1e? In that case, there could be some inherent bias in the metric towards those codecs that would then add some imaginary advantage above their real compression quality into the numbers, when measured by VMAF. Simply because their output was implicitly tuned to get better VMAF, because VMAF was used to test new tools/analysis/RDO and so on.
VMAF wasn't around for most/all of Daala work. AV1 is really the first bitstream to have its practical implementations start in the VMAF era. This is one reason I'm suspicious of VMAF scores for AV1. Good analysis.

In particular I worry that VMAF is insufficiently sensitive to temporal shifts in video quality. A VMAF of 70, 65, 60, 60, 65, 60, 60 might come out as a nice "VMAF=65.3" but be a annoying to watch. Frame strobing was a weakness of libvpx.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book

Last edited by benwaggoner; 30th January 2019 at 00:21. Reason: Bad quoting
benwaggoner is offline   Reply With Quote
Old 30th January 2019, 01:00   #1395  |  Link
TD-Linux
Registered User
 
Join Date: Aug 2015
Posts: 34
Quote:
Originally Posted by TomV View Post
The patent chart Tim is using is out of date and inaccurate. Fraunhofer sold their patents to GE, which is why GE has HEVC patents. Canon licenses their patents through MPEG LA.
The chart is the one used in Leonardo's blog, though I think it's originally from streamingmedia.com. I've attached an updated version for future presentations.


Last edited by TD-Linux; 30th January 2019 at 01:45.
TD-Linux is offline   Reply With Quote
Old 30th January 2019, 02:14   #1396  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 56
I originally created the figure and presented it for the first time during Streaming Tech Sweden 2017. There have been some changes since then.

Here is an updated figure:



Please note that the figure is only based on public information available from ISO/IEC/ITU and from the patent pools. Please also note that not all of the MPEG LA patent holders are shown in the figure.
__________________
Jonatan Samuelsson
Co-founder and CEO at Divideon

www.divideon.com | xvc.io
jonatans is offline   Reply With Quote
Old 30th January 2019, 02:32   #1397  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,120
Quote:
Originally Posted by jonatans View Post
I originally created the figure and presented it for the first time during Streaming Tech Sweden 2017. There have been some changes since then.

Here is an updated figure:



Please note that the figure is only based on public information available from ISO/IEC/ITU and from the patent pools. Please also note that not all of the MPEG LA patent holders are shown in the figure.
I think i read that Franhaufer sold their HEVC patents to General Electric, if true you should remove Franhaufer from your diagram.
hajj_3 is offline   Reply With Quote
Old 30th January 2019, 02:51   #1398  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Quote:
Originally Posted by TD-Linux View Post
The chart is the one used in Leonardo's blog, though I think it's originally from streamingmedia.com. I've attached an updated version for future presentations.

It's originally from Jonatan Samuelsson, a.k.a. jonatans

Even with the updates, the main problem with this chart is that it's a bit misleading.. for 2 reasons. First, I think most people in the video industry assume that AVC patent licensing is and was perfectly clean and simple... that all necessary standard-essential patents were licenseable in the MPEG LA patent pool. That's not true. Nokia, Qualcomm, Broadcomm, Blackberry, Texas Instruments, MIT all hold standard-essential AVC patents outside the MPEG LA pool (although Qualcomm messed up and a judge ruled they can't assert them for AVC). Multiple legal battles have been fought over AVC patents, including some pretty big cases... Microsoft v Motorola, and Apple v Nokia. Today, everyone can agree that the patent licensing situation for AVC is much better than it is for HEVC. But it didn't start out that way, and it took some time for the situation to settle. Also, there are quite a few more patent holders in some of these HEVC pools than shown in this chart.

In his talk, Tim mentioned that patents are issued that may not be valid, and then said "and you could go around and try to invalidate them all, but they're really expensive to do that, and there's a lot of them". Well, if you're a multi-billion dollar company (Apple, Samsung, Google, Amazon, etc.), you have a lot of lawyers, and that's what they're paid to do. If you're being asked to pay tens or hundreds of millions of dollars a year in patent license fees, you have all the motivation in the world to spend whatever it takes on legal fees to right-size the problem. When multiple multi-billion dollar companies have this issue, collectively there is a lot of motivation. It turns out that when challenged in court, most patents don't hold up. They can be invalidated for many reasons... prior art, unpatentable claims, obviousness, the invention was anticipated, etc. This type of effort is being undertaken by Unified Patents (as a service to many large tech companies), and there is a relatively new law called the America Invents Act that provides a faster, less expensive way to get rid of bad patents, called an Inter Partes Review (IPR). Unified already filed an IPR against Velos Media, and you can expect more such filings under their Video Codec domain. But you don't even have to invalidate patents in order not to pay a fortune.

Again, keep in mind that no one is asking for patent license fees for content distribution (streaming, etc.), except for UHD-Blu-ray disc (a small per-disc fee to HEVC advance). Only hardware device manufacturers need to license HEVC patents, and they are dealing with that issue and they continue to support HEVC in every device they make that supports video. For video services, HEVC is free. Now that the majority of active end-user devices support HEVC, it makes a lot of financial sense for video services to make their VOD catalog, or the majority of their live channels available in both AVC and HEVC (not just 4K and HDR content... all content). The bandwidth savings and customer experience improvement far outweigh the additional cost of encoding and CDN storage.
TomV is offline   Reply With Quote
Old 30th January 2019, 02:58   #1399  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Quote:
Originally Posted by hajj_3 View Post
I think i read that Franhaufer sold their HEVC patents to General Electric, if true you should remove Franhaufer from your diagram.
That's true. I don't think it was ever announced, but I can assure you that I got confirmation from a very reliable source.
TomV is offline   Reply With Quote
Old 30th January 2019, 03:02   #1400  |  Link
TomV
VP Eng, Kaleidescape
 
Join Date: Jan 2018
Location: Mt View, CA
Posts: 51
Quote:
Originally Posted by jonatans View Post
I originally created the figure and presented it for the first time during Streaming Tech Sweden 2017. There have been some changes since then.

Here is an updated figure:



Please note that the figure is only based on public information available from ISO/IEC/ITU and from the patent pools. Please also note that not all of the MPEG LA patent holders are shown in the figure.
Hey... we were both responding at the same time (cross posting). Thanks for posting an update Jonatan. Your efforts on this, and in the MC-IF are really appreciated.
TomV is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:28.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.