Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd April 2020, 17:08   #1  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 174
CRF use for content aware VOD streaming?

Hi,

So there are a lot of discussions regarding content aware/per-title streaming for VOD nowdays, were different types of methods are used to select what bitrate is appropriate when it comes to streaming.

old, but yeah basically this:
https://netflixtechblog.com/per-titl...n-7e99442b62a2

But it seems like a lot of streaming services (i guess at least most of the smaller local ones) still just use fixed bitrates for the ladder, cause I guess they dont have resources to implement complex workflows.

So I've been wondering, why isnt CRF used (or is it?)? Cant it be in used together with vbv restrictions to create a crude method to do this?

I'm a bit out of my realm of expertise here but. Lets say you have a 1Mbps connection target preset, why not set a crf value that is close to 500Kbps in avg bitrate for your average complexity material, then set --vbv-maxrate 750 --vbv-bufsize 1000. I'm I missing something or shouldn't this be a fairly good (or at least easy) way of doing it compared to fixed bitrates?

So a ladder would look something like this:

540p24 ~500Kbps (1Mbps safe) --crf 26 --vbv-maxrate 750 --vbv-bufsize 1000

720p24 ~1250Kbps (2Mbps safe) --crf 24 --vbv-maxrate 1750 --vbv-bufsize 2500

1080p24 ~3Mbps (5Mbps safe) --crf 22 --vbv-maxrate 4500 --vbv-bufsize 6000

1080p24 ~6Mbps (10Mbps safe) --crf 19 --vbv-maxrate 9000 --vbv-bufsize 12000

So, my question is basically: is this a good or a poor way of doing it? If good, is this already in widespread use, and if not, why?

Last edited by excellentswordfight; 22nd April 2020 at 17:28.
excellentswordfight is offline   Reply With Quote
Old 22nd April 2020, 19:13   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,816
This is a reasonable way of doing things. Some providers do exactly this.

Beamr did an interesting review of the landscape and found several content adaptation methods

http://media2.beamrvideo.com/pdf/Bea...Tech_Guide.pdf

Beamr's tech is cool, they do frame by frame optimization in loop with encoding, so they calculate a lot of metrics for an encoded frame and then re-encode the frame with fewer and fewer bits until they get the lowest data rate possible while maintaining the metric score. It's more complex than that, but it's a tight integration. I've looked at their stuff and it's quite good. It's not cheap tho

Netflix does extensive content optimization (the per scene convex hull approach).
Blue_MiSfit is offline   Reply With Quote
Old 22nd April 2020, 19:39   #3  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,644
Yeah, using CRF can definitely be a useful tool in this regard. One problem is that some older heuristics struggle with the variable bitrate, especially if they don't make an internal distinction between peak bitrate and actual bitrate. So you'll see this sort of technique used more by companies who either focus only on web or recent devices, or who are able to deliver their own player with custom heuristics. If one is trying to work with the build-in DASH client in older living room devices, things get fraught quickly.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 22nd April 2020, 20:15   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,644
Looks like someone at Amazon patented some related ideas . http://pat2pdf.org/patents/pat9712860.pdf
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 22nd April 2020, 20:25   #5  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,816
Makes sense. Smart TVs in general (especially older ones) are _ATROCIOUS_ platforms to develop for, especially when you have to live in the sandbox environment they give app developers. Netflix always gets special permission to run their own binary. Nobody else gets that level of integration, so it limits what's possible.
Blue_MiSfit is offline   Reply With Quote
Old 24th April 2020, 00:01   #6  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,644
Quote:
Originally Posted by Blue_MiSfit View Post
Makes sense. Smart TVs in general (especially older ones) are _ATROCIOUS_ platforms to develop for, especially when you have to live in the sandbox environment they give app developers. Netflix always gets special permission to run their own binary. Nobody else gets that level of integration, so it limits what's possible.
I wouldn't say nobody else. I'd expect it is available to the big five premium content services. They build players in a lot of different ways targeting different levels of abstraction.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 24th April 2020, 00:25   #7  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,816
That's true, I guess having worked for one of the big studios and still running into these limitations I assumed that would be the case for other platforms as well
Blue_MiSfit is offline   Reply With Quote
Old 24th April 2020, 16:36   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,644
Quote:
Originally Posted by Blue_MiSfit View Post
That's true, I guess having worked for one of the big studios and still running into these limitations I assumed that would be the case for other platforms as well
Other than Disney+, the big studios's official platforms don't have a significant market share. Here's a recent breakdown: https://www.mediapost.com/publicatio...=factoftheday1

Because only Disney has core brands customers are familiar with (Disney, Marvel, Pixar). People just don't know which shows and movies would be on Peacock versus CBS All Access.

Building and supporting premimum experience video players for all the devices people want to watch on (so many generations of OS software running on so many different SoCs!) and the content for those (different codecs, packaging, and DRM systems) takes a 1000+ person engineering org by my estimate.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 26th April 2020, 22:27   #9  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
We are actually doing this as we speak.
We dont have need to go as far as netflix/youtube are going. So we went the CRF + CAP way with a fixed ladder. Gives quite results as we are only allowing newer devices on native players..
TEB is offline   Reply With Quote
Old 27th April 2020, 23:20   #10  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,644
Quote:
Originally Posted by TEB View Post
We are actually doing this as we speak.
We dont have need to go as far as netflix/youtube are going. So we went the CRF + CAP way with a fixed ladder. Gives quite results as we are only allowing newer devices on native players..
By CAP do you mean capped VBV?

It is a delicious luxury to only have to worry about newer devices! Of course, the longer a service lasts, the more today's bright and shiny will eventually become tomorrow legacy deadweight holding you back .
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 28th April 2020, 09:39   #11  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
Quote:
Originally Posted by benwaggoner View Post
By CAP do you mean capped VBV?

It is a delicious luxury to only have to worry about newer devices! Of course, the longer a service lasts, the more today's bright and shiny will eventually become tomorrow legacy deadweight holding you back .
X264:
-crf 20 -maxrate 15M -bufsize 15M for the top profile
TEB is offline   Reply With Quote
Old 29th April 2020, 20:03   #12  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,816
In my experience, the DASH clients on older devices are insane. If those devices are important enough, then you need to make playback work well on them. It kind of doesn't matter what the problem really is.

This can ALSO be an issue with poorly implemented DRM, btw. I've definitely seen certain popular settop boxes blow up when fed streams that were 100% valid within the constraints of the Main10 profile @ level 5 (both vbv limits and MaxDpbSize). Reducing the reference frame count was the only solution. This was acknowledged as a firmware bug but I don't know if it's ever been fixed
Blue_MiSfit is offline   Reply With Quote
Old 30th April 2020, 09:42   #13  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
Quote:
Originally Posted by Blue_MiSfit View Post
In my experience, the DASH clients on older devices are insane. If those devices are important enough, then you need to make playback work well on them. It kind of doesn't matter what the problem really is.

This can ALSO be an issue with poorly implemented DRM, btw. I've definitely seen certain popular settop boxes blow up when fed streams that were 100% valid within the constraints of the Main10 profile @ level 5 (both vbv limits and MaxDpbSize). Reducing the reference frame count was the only solution. This was acknowledged as a firmware bug but I don't know if it's ever been fixed
I hear ya Luckily we have a higher minimum device req.
With that said, what we DO have seen as the major issue is actually Lvl1 WV in the combination with 1080p50 for linear sports broadcasts.
Alot of embedded devices running AOSP/ATV cant properly do this without dropping frames in the TEE environment..
Apple IOS/APPLE eats most of our content with little complaint as long as we have ios10+ as the minimum os level req
TEB is offline   Reply With Quote
Old 1st May 2020, 00:10   #14  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,816
Quote:
Originally Posted by TEB View Post
I hear ya Luckily we have a higher minimum device req.
With that said, what we DO have seen as the major issue is actually Lvl1 WV in the combination with 1080p50 for linear sports broadcasts.
Alot of embedded devices running AOSP/ATV cant properly do this without dropping frames in the TEE environment..
Apple IOS/APPLE eats most of our content with little complaint as long as we have ios10+ as the minimum os level req
Yeah, throughput for AES-CTR mode becomes a major issue on cost constrained devices. Apple was right to support only AES-CBCS since it' so much faster. Incidentally they were right for the wrong reasons Their devices are beefy enough to do CTR at high speeds! We've seen on certain cheap crap Smart TVs that support both formats that switching from CTR to CBCS almost doubled throughput.

Anyway, we're drifting off-topic here a bit, even if this is still relevant in the context of the risks associated with not using more traditional CBR / heavily capped VBR rate control modes for streaming
Blue_MiSfit is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.