Alliance for Open Media codecs

[IgorC]

Originally Posted by TomV

Again, the people who run worldwide video streaming services know exactly how many customers are bandwidth limited,

So do network engineers know (in fact even better)

Again, where is xHE-AAC adoption? It's as old as HEVC. HEVC was adopted, xHE-AAC wasn't.
If ultra low bitrates are so important why nobody hurries to adopt this codec?

Name me just one relatively large broadcasting company who has adopted it.
Name me just one developer team (not Fraunhofer themselves) who actually developing xHE-AAC encoder in this moment.

Can You do it?
Because if You can't I don't see a reason to keep this dicussion.

Originally Posted by TomV

The video service provider has logs of all of this activity, so they know what % of streams are at the lowest bit rate tier, and where/when this happens. Some have tiers as low as 100 kbps... (including about 10 kbps for mono audio).

Are You sure this number isn't very small? 0.1%, 1-2% ?
Can You present any statistics?

[Nintendo Maniac 64]

So um, as someone on the outside, I've got to ask - outside of DRM and satisfying "not invented here" syndrome, what benefit does xHE-AAC provide over something like Opus?

[benwaggoner]

Originally Posted by Nintendo Maniac 64

So um, as someone on the outside, I've got to ask - outside of DRM and satisfying "not invented here" syndrome, what benefit does xHE-AAC provide over something like Opus?

It supposedly offers somewhat better quality at very low bitrates. I've not seen a detailed double-blind listening test to validate that, though.

I think xHE-AAC is going to become more broadly supported on platforms, out of momentum. AAC licensees now get access to xHE-AAC for free so it's a trivial effort to roll in xHE-AAC support with platforms updates. And use of AAC in MPEG-4 streams is broadly understood and implemented. Opus does have a mapping, but I've not seem much use of it. MPEG-4 as a file format is more dominant than H.264 was; the Matroska based container formats aren't a significant player for commercially distributed content.

The biggest recent news is that Android Pie has xHE-AAC built in.

xHE-AAC also offers gapless switching between bitrates without having to do overlap decoding. Does Opus support that. I consider this a significant advantage of xHE-AAC over HE v1/v2 and LC, which can only do gapless switching within v1 or v2, but not between. xHE-AAC can switch from very low bitrates to very high quality, which wasn't feasible before.

And don't knock how critical DRM is for premium content. It's a Very Big Deal. And there are SoC reasons why mixing encrypted video and unencrypted audio can be problematic.

[Blue_MiSfit]

^ DRM is absolutely positively mandatory - no two ways about it. You simply will not get the rights to distribute content if you don't have approved DRM implementations, and this is extremely specific e.g hardware implementations of PlayReady, Widevine with progressive restrictions to unlock HD or UHD content.

I don't love DRM, but it's just table stakes when you're delivering premium content. There's no way around it, so the best we can do is make it as unobtrusive as possible!

Regarding the 200 - 300 Kbps scenario - another use case would be rural customers with satellite or very poor cell service. I grew up in a very small town and many of my friends live outside the city limits where there simply is no broadband. Not even DSL, though you could maybe get an ISDN or T1 line if you're a masochist

You might get one bar of LTE (or two if you stand in exactly the right place), and you're sharing this one solitary tower with your neighbors, so during peak times you're lucky to get 1 Mbps, assuming you're not over your data cap, at which point you drop down to under 500 Kbps. Satellite can be fast, but generally is very over-sold in these areas, and cannot keep up with demand during peak times. It also has extreme data caps, and the throttled speed is extremely slow.

Anyway - hoping we can get back on topic - AOM codec discussion

Maybe viewing through the ultra low bitrate lens, has anyone done very low bitrate 2 pass VBR tests with AV1? I'd be interested to see how that might fit into the above scenario regarding downloading content for offline playback. This is a neat scenario because you get the bonus of being able to skip all the compromises one must make when encoding for adaptive bitrate delivery and can use very large buffers (vbv-maxrate + vbv-bufsize) and longer adaptive keyframe intervals.

[benwaggoner]

Originally Posted by Blue_MiSfit

^ DRM is absolutely positively mandatory - no two ways about it. You simply will not get the rights to distribute content if you don't have approved DRM implementations, and this is extremely specific e.g hardware implementations of PlayReady, Widevine with progressive restrictions to unlock HD or UHD content.

Yeah. It is table stakes for anything that isn't piracy or user-generated content. If AV1 matters outside of social networks, it'll be because it gets 1st class DRM support, which requires HW decoders and other deep SoC integration. Weird little SoC DRM design decisions have kept a lot of amazing things from happening.

Maybe viewing through the ultra low bitrate lens, has anyone done very low bitrate 2 pass VBR tests with AV1? I'd be interested to see how that might fit into the above scenario regarding downloading content for offline playback. This is a neat scenario because you get the bonus of being able to skip all the compromises one must make when encoding for adaptive bitrate delivery and can use very large buffers (vbv-maxrate + vbv-bufsize) and longer adaptive keyframe intervals.

It's not so much a long buffer window, but having a maxrate>>bitrate. Maxrate and bufsize can be kept at the profile @ level maximums while ABR can be way way lower. Doing maximum 10 sec Open GOP is pretty reasonable.

I worry that AV1's interframe CABAC dependencies will impair random access enough to make the practical maximum GOP duration a lot smaller. Some of that could probably be addressed via encoder tweaks, at the loss of a little efficiency.

[Phanton_13]

I think that I misled wen I reference DRM, I was referencing to Digital Radio Mondiale where xhe-aac is one of the codecs used.

The lowest that I went with AV1 is q50 for sd anime and is surprisingly watchable (resulting in about 100 to 200 kbps of video bitrate, the audio was opus at 32kbps). Definitely CDEF is a great tool. Around 30MB per episode... I wish that this quality-bitrate ratio was available 20 years ago.

[IgorC]

Originally Posted by Nintendo Maniac 64

So um, as someone on the outside, I've got to ask - outside of DRM and satisfying "not invented here" syndrome, what benefit does xHE-AAC provide over something like Opus?

Both Opus and xHE-AAC are hybrid music/speech formats and have essentially same quality. xHE-AAC is slightly better than Opus at 16-32 kbps, quality at 48-64 kbps is the same for both and Opus is slightly better than all family LC-AAC/HE-AAC/xHE-AAC at higher bitrates.

Many people try to market a feature of xHE-AAC as switching between bitrates as something outstanding and so on. But in reality it's not a premium feature and Opus supports it from the very beginning. More here https://wiki.hydrogenaud.io/index.php?title=Opus

Also xHE-AAC is a high delay codec and it's not suitable for real time communications like VOIP calls etc. Company who wants real time communication should also adopt low-delay xHE-AAC fork (EVS). And if You want stereo EVS then this is just another extension. So we got mutltiple codecs and/or extensions:
1.xHE-AAC
2.low delay EVS
3.EVS stereo extension (aka IVAS)
...
It's sort of LC-AAC, HE-AAC, HE-AACv2, Low delay AAC (LD-AAC), Enhanced LD-AAC ( low delay HE-AAC) aka ELD-AAC, ELD-AAC v2 (HE-AACv2 low delay), xHE-AAC, EVS ( low delay xHE-AAC), IVAS (low delay stereo xHE-AAC)...

While Opus is just one single format for everything: high-,low-delay, stereo and multichannel codec.
So I'm not surprised that Opus is popular in internet community while xHE-AAC support is non-existent. Android 9 will get an xHE-AAC decoder but there is still no any encoder

[Tommy Carrot]

Originally Posted by Blue_MiSfit

Maybe viewing through the ultra low bitrate lens, has anyone done very low bitrate 2 pass VBR tests with AV1? I'd be interested to see how that might fit into the above scenario regarding downloading content for offline playback.

Very low bitrate is definitely the strong point of AV1 (more precisely aomenc). At around crf 28-30 compression rates, it's starting to get better than x265, and the lower the bitrate, the bigger the advantage. XVC, and especially VVC are still better though, the VVC reference encoder is seriously amazing at ultra low bitrate scenarios.

[benwaggoner]

Originally Posted by Tommy Carrot

Very low bitrate is definitely the strong point of AV1 (more precisely aomenc). At around crf 28-30 compression rates, it's starting to get better than x265, and the lower the bitrate, the bigger the advantage. XVC, and especially VVC are still better though, the VVC reference encoder is seriously amazing at ultra low bitrate scenarios.

Ooh, do you have any examples or command lines for the >x265 for very low bitrates? Low bitrate, low resolution video enables MUCH faster encode/evaluate/reencode cycles and would be a great place to evaluate AV1's potential strengths.

[Tommy Carrot]

Originally Posted by benwaggoner

Ooh, do you have any examples or command lines for the >x265 for very low bitrates? Low bitrate, low resolution video enables MUCH faster encode/evaluate/reencode cycles and would be a great place to evaluate AV1's potential strengths.

Nothing special, i pretty much use the default settings.

aomenc --end-usage=q --cq-level=xx --cpu-used=1 --kf-max-dist=250 -v -o av1.webm test.y4m

I haven't tried 2-pass bitrate mode though, only CQ mode with using the quantizer with the closest bitrate.

[Mr.Radar]

Originally Posted by Phanton_13

I think that I misled wen I reference DRM, I was referencing to Digital Radio Mondiale where xhe-aac is one of the codecs used.

For those who are unaware, Digital Radio Mondiale is the digital broadcasting standard used on the shortware radio bands. Due to the narrow bandwidth of shortwave broadcast channels and the high amount of error correction required to avoid dropouts the usable bitrates are very low. Here is a 2017 comparison between xHE-AAC and Opus at those bitrates. More recent versions of libopus should produce significantly more competitive results in the speech-focused samples (libopus 1.3 enabled wideband audio down to 9 kbps) but on music samples xHE-AAC would probably retain the edge since Opus at very low bitrates is effectively operating mostly as a speech codec in its SILK mode.

[Mr_Khyron]

https://github.com/OpenVisualCloud/SVT-AV1/pull/55

Over 50% reduction in memory requirements

[soresu]

It says "low core count (e.g. 4-core machines)" though, does that mean it doesnt apply at all to higher core counts, or that they alreeady have a similar optimisation implemented or in the pipe?

[nevcairiel]

It probably means that memory usage didn't scale properly to different core counts. The more cores you run, the more memory its going to need, that is not unexpected from any encoder.

[Mr_Khyron]

https://phoronix.com/scan.php?page=n...Speed-Progress

It was just a few weeks ago that Intel open-sourced the SVT-AV1 project as a CPU-based AV1 video encoder. In the short time since publishing it, there's already been some significant performance improvements.

Since the start of the month, SVT-AV1 has added multi-threaded CDEF search, more AVX optimizations, and other improvements to this fast evolving AV1 encoder. With having updated the test profile against the latest state as of today, here's a quick look at the performance of this Intel open-source AV1 video encoder.

[Nintendo Maniac 64]

Originally Posted by soresu

"low core count (e.g. 4-core machines)"

That's certainly amusing considering that Intel seemed to love branding quad cores as i7 up until recently.

Speaking of quad core i7 CPUs...

Originally Posted by Mr_Khyron

https://phoronix.com/scan.php?page=n...Speed-Progress

I'm going to go out on a limb and guess that, on the SVT-AV1 v2019-02-15 results, the lack of performance delta seen between the i7-7740X and Ryzen 2700X is due to the former's considerably faster AVX2 implementation even though the Ryzen has twice as many cores and threads as the i7?

[Selur]

btw. https://ci.appveyor.com/project/Open...uild/artifacts offers Windows binaries of the stv-av1 encoder (SvtAv1EncApp.exe and SvtAv1EncApp.dll are needed)

[Blue_MiSfit]

Very cool. I'm playing with this now.

Default settings are quite fast - it did 12 fps for 480p on my i7 7700k with 50% CPU usage and 1.2 GB RAM usage.

Here's the basic usage guide:
https://github.com/OpenVisualCloud/S..._user_guide.md

A sample command:
.\SvtAv1EncApp.exe -i .\beauty_480p.yuv -b out.ivf -w 848 -h 480 -fps 24

Yes, I had to encode at 848x480 and not 854 - comically this encoder requires mod 8 input

Results aren't too bad - the default CQ 50 setting produced a 623 Kbps file that looks marginally okay, and CQ 40 produced a 1.2 Mbps file that looks a lot better. There are some odd artifacts, almost like the edges of objects wiggle a bit every other frame.

However, I'm getting very jerky playback for some reason. I've tried a LAV nightly build in MPC-HC, latest VLC, and ffplay, and they all show the issue. I confirmed my input YUV is 24 fps and the output webm is 24 fps. When I step through it frame by frame it's all there, but for some reason it's jerky on playback. I don't recall seeing this with aomenc encodes, and the decoder isn't close to maxing out one core, so I'm not sure what's up with that...

[tnti]

Originally Posted by Blue_MiSfit

Very cool. I'm playing with this now.

Default settings are quite fast - it did 12 fps for 480p on my i7 7700k with 50% CPU usage and 1.2 GB RAM usage.

Here's the basic usage guide:
https://github.com/OpenVisualCloud/S..._user_guide.md

A sample command:
.\SvtAv1EncApp.exe -i .\beauty_480p.yuv -b out.ivf -w 848 -h 480 -fps 24

Yes, I had to encode at 848x480 and not 854 - comically this encoder requires mod 8 input

Results aren't too bad - the default CQ 50 setting produced a 623 Kbps file that looks marginally okay, and CQ 40 produced a 1.2 Mbps file that looks a lot better. There are some odd artifacts, almost like the edges of objects wiggle a bit every other frame.

However, I'm getting very jerky playback for some reason. I've tried a LAV nightly build in MPC-HC, latest VLC, and ffplay, and they all show the issue. I confirmed my input YUV is 24 fps and the output webm is 24 fps. When I step through it frame by frame it's all there, but for some reason it's jerky on playback. I don't recall seeing this with aomenc encodes, and the decoder isn't close to maxing out one core, so I'm not sure what's up with that...

https://github.com/OpenVisualCloud/SVT-AV1/issues/33

[TD-Linux]

Originally Posted by benwaggoner

I worry that AV1's interframe CABAC dependencies will impair random access enough to make the practical maximum GOP duration a lot smaller. Some of that could probably be addressed via encoder tweaks, at the loss of a little efficiency.

You don't need to worry - AV1 probability dependencies can only come from one of the reference frames, so it doesn't place any additional impairment on seekability.

(also note that CABAC is a misnomer, as it's not binary. The spec doesn't give it an acronym, but dav1d uses MSAC).