Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th May 2019, 13:38   #1681  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 957
Quote:
Originally Posted by stax76 View Post
Last time I tried rav1e 2019-04-30 and it was SLOW AS HELL, like < 1 fps.
1 fps is fast for av1
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is online now   Reply With Quote
Old 28th May 2019, 14:19   #1682  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 95
I'm just wondering if the quality regression in rav1e has been fixed yet, with activity masking coming from GSOC it should be in a pretty good place quality wise if nothing is still broken.
soresu is offline   Reply With Quote
Old 29th May 2019, 06:56   #1683  |  Link
VincAlastor
Registered User
 
Join Date: Sep 2009
Location: Berlin
Posts: 148
Quote:
Originally Posted by stax76 View Post
What is currently the fastest AV1 encoder?
Again thank you very much for staxrip!!!

i use in staxrip a custom cmd line for ffmpeg aomenc from streaming media experts.

-c:v libaom-av1 -crf 40 -b:v 0 -strict experimental -threads 24 -tiles 8x2 -cpu-used 5 -row-mt 1

https://www.streamingmedia.com/Artic...ls-130284.aspx

https://www.reddit.com/r/AV1/comment...s_mostly_with/


also new version of rav1e with tiles support brings up to 3 fps for fullhd content

https://blog.rom1v.com/2019/04/imple...ding-in-rav1e/
VincAlastor is offline   Reply With Quote
Old 29th May 2019, 13:35   #1684  |  Link
birdie
.
 
birdie's Avatar
 
Join Date: Dec 2006
Posts: 135
At this time shouldn't we consider AV1 a failure and move on to newer codecs, e.g AV2?
  • It doesn't have fast enough decoders to decode on mobile at 1080p on most devices (>80%).
  • It still doesn't have encoders which are anywhere fast enough to be usable by mere mortals.
  • Its hardware adoption is not there - the spec was finalized almost half a year ago, and AV1 is nowhere to be seen in Zen 2.0 (Ryzen 3000), Radeon RDNA 5700 or Intel Ice Lake. No word on its decoding acceleration even in the recently announced Arm's Cortex-A77/Mali-G77.
birdie is offline   Reply With Quote
Old 29th May 2019, 15:19   #1685  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,760
You do realize that a newer codec would end up even more complex, and thus even slower, and also slowing down hardware adoption even more?
All your points can be applied for any new codec.

- Mobile hardware, especially on the low-end, is inherently slow. The dav1d AV1 decoder can decode 1080p on mid-range and high-end mobile devices. But for mainstream roll out, thats not going to be used anyway, because it uses too much battery.
- Encoder development takes years. You must not have been around when any other codec was new. On top of that, any new codec is also always going to be slower then previous codecs. You pay for quality or compression with speed.
- Hardware development also takes years. Noone knolwedgeable ever realistically expected AV1 to show up in hardware before the end of 2020 or so. The turn-around times for hardware are really long. Hardware you see launch/announced today was long through the design process before AV1 was finished.

Any other new codec would be in the exact same situation a year or so after the spec was officially finalized. In fact this is already looking pretty good on adoption, most major browsers now include AV1 decoders, and YouTube is rolling out content.

Basically, do some more research on how codec lifetime has been in the past for any other codec.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 29th May 2019 at 15:24.
nevcairiel is online now   Reply With Quote
Old 29th May 2019, 16:50   #1686  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 95
Quote:
Originally Posted by nevcairiel View Post
You do realize that a newer codec would end up even more complex, and thus even slower, and also slowing down hardware adoption even more?
All your points can be applied for any new codec.

- Mobile hardware, especially on the low-end, is inherently slow. The dav1d AV1 decoder can decode 1080p on mid-range and high-end mobile devices. But for mainstream roll out, thats not going to be used anyway, because it uses too much battery.
- Encoder development takes years. You must not have been around when any other codec was new. On top of that, any new codec is also always going to be slower then previous codecs. You pay for quality or compression with speed.
- Hardware development also takes years. Noone knolwedgeable ever realistically expected AV1 to show up in hardware before the end of 2020 or so. The turn-around times for hardware are really long. Hardware you see launch/announced today was long through the design process before AV1 was finished.

Any other new codec would be in the exact same situation a year or so after the spec was officially finalized. In fact this is already looking pretty good on adoption, most major browsers now include AV1 decoders, and YouTube is rolling out content.

Basically, do some more research on how codec lifetime has been in the past for any other codec.
I wish I could upvote this, you pretty much laid it out nice and clear there.
soresu is offline   Reply With Quote
Old 29th May 2019, 16:54   #1687  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 95
To add to Nevcariel's reply to Birdie, I would not expect any ASIC decoders to appear on AMD CPU's as they are generally integrated into the GPU design, which also means they end up in APU's aswell - once they have an ASIC it will be on the nearest APU or GPU release from that point.
soresu is offline   Reply With Quote
Old 29th May 2019, 21:43   #1688  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,436
We'll have to wait and see when or if free AV1 encoders surface that can beat e.g. x265 at not more than 10x slow down. It's hard to please the doom9 crowd. VP9 never got to that point, maybe AV1 will be the same.
sneaker_ger is offline   Reply With Quote
Old 30th May 2019, 11:10   #1689  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,760
Quote:
Originally Posted by sneaker_ger View Post
We'll have to wait and see when or if free AV1 encoders surface that can beat e.g. x265 at not more than 10x slow down. It's hard to please the doom9 crowd. VP9 never got to that point, maybe AV1 will be the same.
Encoders are no longer being made for this crowd. The primary design goal is massive-scale cloud encoding for YouTube, Netflix, Amazon, and everyone else that fits the encode-once, download hundreds of thousands of times scenario.

In such a scenario, even the slowest encoder is acceptable if it saves enough bytes.

In that scenario, VP9 also didn't fail. It gets used for a lot of content on the web.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is online now   Reply With Quote
Old 30th May 2019, 11:32   #1690  |  Link
birdie
.
 
birdie's Avatar
 
Join Date: Dec 2006
Posts: 135
Quote:
Originally Posted by nevcairiel View Post
Encoders are no longer being made for this crowd. The primary design goal is massive-scale cloud encoding for YouTube, Netflix, Amazon, and everyone else that fits the encode-once, download hundreds of thousands of times scenario.

In such a scenario, even the slowest encoder is acceptable if it saves enough bytes.

In that scenario, VP9 also didn't fail. It gets used for a lot of content on the web.
This is what I was talking about.

AV1 is not a codec for masses. It's a very special codec for content delivery. That's it. And that makes it and its discussion kinda worthless.

And VVC is already miles better/faster/more effective than AV1.
birdie is offline   Reply With Quote
Old 30th May 2019, 11:36   #1691  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 90
Quote:
Originally Posted by birdie View Post
And VVC is already miles better/faster/more effective than AV1.
Whatever you think.
richardpl is offline   Reply With Quote
Old 30th May 2019, 12:25   #1692  |  Link
dapperdan
Registered User
 
Join Date: Aug 2009
Posts: 180
I'm not sure it's fair to say VP9 and AV1 were designed purely for those use cases, it's just that that's one of the easiest niches to win if you're a next-gen codec where encoding time is traded for smaller size.

That lets them use it profitably from day one while expanding further into other niches. It probably has impacts on how much effort goes into multithreading or other features that this use case doesn't need.

Libvpx seems to equal x265, subjectively, objectively and in encoding time in the recent MSU study (and both are near the head of the pack) The argument now seems to be that it's the rate control that makes it unsuitable for many users despite good showing in test scenarios. But libvpx having bad rate control is a rather different claim than the VP9 format being 10x slower than it should be and therefore a disaster.
dapperdan is offline   Reply With Quote
Old 30th May 2019, 12:35   #1693  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,436
Doom9 crowd can't exactly live on the potential of a spec. It needs (free/cheap) access to a well-rounded encoder implementation with a sweet spot on bitrate distribution/AQ/speed. x264 and x265 meet those demands. libvpx? Not so much.
sneaker_ger is offline   Reply With Quote
Old 30th May 2019, 13:02   #1694  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 122
Quote:
Originally Posted by sneaker_ger View Post
(free/cheap) access to a well-rounded encoder implementation with a sweet spot on bitrate distribution/AQ/speed.
I suspect that we won't see such encoders for VVC.
lvqcl is offline   Reply With Quote
Old 30th May 2019, 13:41   #1695  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,760
Quote:
Originally Posted by dapperdan View Post
I'm not sure it's fair to say VP9 and AV1 were designed purely for those use cases, it's just that that's one of the easiest niches to win if you're a next-gen codec where encoding time is traded for smaller size.
Its not a design target for the codec itself, because the codec really doesn't care. I'm talking about encoders. The huge open-source push that made x264 as great as it is for "personal" encodes is unlikely to repeat itself. Companies driving encoder development do not target doom9ers. You can already see this on x265 where the community involvement is pretty low, and this will only get worse as the computational complexity of codecs goes up and the "personal use" usecases get less attractive.

Quote:
Originally Posted by birdie View Post
This is what I was talking about.

AV1 is not a codec for masses. It's a very special codec for content delivery. That's it. And that makes it and its discussion kinda worthless.
This will not change with any future codec. Not with AV2, not with VVC, or anything that follows. The computational complexity increase in all those future codecs just makes it impractical for "hobbyist" use.

Quote:
Originally Posted by birdie View Post
And VVC is already miles better/faster/more effective than AV1.
Dream on. A MPEG reference encoder has never won any price in any of those categories.

Quote:
Originally Posted by sneaker_ger View Post
Doom9 crowd can't exactly live on the potential of a spec. It needs (free/cheap) access to a well-rounded encoder implementation with a sweet spot on bitrate distribution/AQ/speed. x264 and x265 meet those demands. libvpx? Not so much.
And unless the "Doom9 crowd" is going to develop their own encoder, noone is going to do that for them. Thats how x264 was ultimately born, it was made by video enthusiasts, not a company.

---

It really all comes down to the inherent complexity of newer codecs. As it gets more and more impractical to encode them due to the speed, less and less people and smaller companies are going to use them, and the entire ecosystem shifts over to only the bigger players that have the volume to host huge encoding farms. Even if there was a perfect free AV1 encoder, it would still be slow. There is no going fast without sacrificing quality or compression, at which point you eventually cross into the domain of already established codecs, and you lose your reason to even use the newer codec in the first place. Hence, development is no longer targeting individuals or small companies.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 30th May 2019 at 13:54.
nevcairiel is online now   Reply With Quote
Old 30th May 2019, 14:01   #1696  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,436
Quote:
Originally Posted by nevcairiel View Post
And unless the "Doom9 crowd" is going to develop their own encoder, noone is going to do that for them. Thats how x264 was ultimately born, it was made by video enthusiasts, not a company.
I'm not judging, just saying how it is. People see the advertisement for the new shiny codec and can't wait to profit from "50% less bitrate". They need to reduce their hopes. As you say AV1 today isn't for them and maybe never will. Of course I wouldn't mind to be proved wrong in the future.
sneaker_ger is offline   Reply With Quote
Old 30th May 2019, 22:20   #1697  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,922
Quote:
Originally Posted by nevcairiel View Post
Encoders are no longer being made for this crowd. The primary design goal is massive-scale cloud encoding for YouTube, Netflix, Amazon, and everyone else that fits the encode-once, download hundreds of thousands of times scenario.

In such a scenario, even the slowest encoder is acceptable if it saves enough bytes.
Oh, these things are always down to cost benefit. Google is just willing to subsidize VPx/AV1 to a huge degree, and likely has a lot of spot-idle capacity in data centers to do the encoding.

But high quality encoding doesn't work with chunks of a few seconds. Encoding longer sequences allows for IDRs and shot changes and more aggressive VBV use. YouTube can have quite a bit of keyframe strobing with difficult content for these reasons. YouTube quality wouldn't be acceptable for lots of premium content.

There has never been a VP9 encoder that offers sufficient quality OR performance for premium content, and there isn' one for AV1 yet either. I don't think this is because the VP9 bitstream wasn't capable of it, it's just that no one wrote an encoder with good psychovisual tuning, intra-frame parallelism, and other stuff.

Encoders are a real chicken-egg problem. There needs to be enough companies willing to pay for better encoders to create a competitive market so that companies work hard to make better encoders than each other. That market never emerged for VP9, so libvpx never saw the kind of quality and performance improvement of, say, the H.264 or HEVC reference encoders to the best available commercial encoders.

There is clearly more interest in AV1 than there ever was for VP9, and more quality innovation already than VP9 has had to date. Which is very promising.

Quote:
In that scenario, VP9 also didn't fail. It gets used for a lot of content on the web.
Other than YouTube?

VP9/s niche is user-generated non-DRM social media content. And in practice, H.264 would have offered at least equivalent quality at equal encoding time due to faster and more psychovisually tuned encoders. An x264 running at veryslow speed is going to be the same speed as a quite low-complexity VP9 encoder, especially on high-core systems. The quality comparison for high volume use are done at quality @ bitrate @ time.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 30th May 2019, 22:28   #1698  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,922
Quote:
Originally Posted by sneaker_ger View Post
I'm not judging, just saying how it is. People see the advertisement for the new shiny codec and can't wait to profit from "50% less bitrate". They need to reduce their hopes. As you say AV1 today isn't for them and maybe never will. Of course I wouldn't mind to be proved wrong in the future.
We have seen 50% improvements generation-to-generation with the MPEG codecs. But that's at the same point of development. An HEVC enocoder with seven years of refinement isn't going to be only half as good as a VVC reference implementation. But comparing HEVC encoders seven years after spec freeze to VVC seven years after code freeze probably will show ~50% bitrate savings. Some content will even be more. Film grain synthesis could result in 75% reduction in some of the most difficult content, for example.

AV1 wasn't ever promised to be more than 20% better, and even that was in mean PSNR, not psychovisually. AV1 encoders haven't demonstrated any advantage over HEVC with subjective quality, even with the reference encoders.

The VPx code base started VERY heavily PSNR-tuned, which may be way it does well with that today. I don't have any reason to think that is a limitation of the AV1 bitstream versus just the libaom encoder.

But the general case of "AV1 can deliver the same subjective quality at meaningfully lower bitrates than HEVC" has yet to be demonstrated.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 30th May 2019, 23:19   #1699  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,760
Quote:
Originally Posted by benwaggoner View Post
Other than YouTube?
Netflix uses it for mobile devices, with the EVE encoder I believe, which they have found to be equal to x265 quality at the time, if not slightly better in some cases (last blog on that was from December)
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 30th May 2019 at 23:25.
nevcairiel is online now   Reply With Quote
Old 30th May 2019, 23:46   #1700  |  Link
TD-Linux
Registered User
 
Join Date: Aug 2015
Posts: 32
Quote:
Originally Posted by stax76 View Post
Last time I tried rav1e 2019-04-30 and it was SLOW AS HELL, like < 1 fps.
You can try speeding it up with the --tile-cols-log2 and --tile-rows-log2 option if you have a lot of cores. I'm working on making these more automatic.
TD-Linux is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:12.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.