Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
22nd April 2020, 17:08 | #1 | Link |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
CRF use for content aware VOD streaming?
Hi,
So there are a lot of discussions regarding content aware/per-title streaming for VOD nowdays, were different types of methods are used to select what bitrate is appropriate when it comes to streaming. old, but yeah basically this: https://netflixtechblog.com/per-titl...n-7e99442b62a2 But it seems like a lot of streaming services (i guess at least most of the smaller local ones) still just use fixed bitrates for the ladder, cause I guess they dont have resources to implement complex workflows. So I've been wondering, why isnt CRF used (or is it?)? Cant it be in used together with vbv restrictions to create a crude method to do this? I'm a bit out of my realm of expertise here but. Lets say you have a 1Mbps connection target preset, why not set a crf value that is close to 500Kbps in avg bitrate for your average complexity material, then set --vbv-maxrate 750 --vbv-bufsize 1000. I'm I missing something or shouldn't this be a fairly good (or at least easy) way of doing it compared to fixed bitrates? So a ladder would look something like this: 540p24 ~500Kbps (1Mbps safe) --crf 26 --vbv-maxrate 750 --vbv-bufsize 1000 720p24 ~1250Kbps (2Mbps safe) --crf 24 --vbv-maxrate 1750 --vbv-bufsize 2500 1080p24 ~3Mbps (5Mbps safe) --crf 22 --vbv-maxrate 4500 --vbv-bufsize 6000 1080p24 ~6Mbps (10Mbps safe) --crf 19 --vbv-maxrate 9000 --vbv-bufsize 12000 So, my question is basically: is this a good or a poor way of doing it? If good, is this already in widespread use, and if not, why? Last edited by excellentswordfight; 22nd April 2020 at 17:28. |
22nd April 2020, 19:13 | #2 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
This is a reasonable way of doing things. Some providers do exactly this.
Beamr did an interesting review of the landscape and found several content adaptation methods http://media2.beamrvideo.com/pdf/Bea...Tech_Guide.pdf Beamr's tech is cool, they do frame by frame optimization in loop with encoding, so they calculate a lot of metrics for an encoded frame and then re-encode the frame with fewer and fewer bits until they get the lowest data rate possible while maintaining the metric score. It's more complex than that, but it's a tight integration. I've looked at their stuff and it's quite good. It's not cheap tho Netflix does extensive content optimization (the per scene convex hull approach). |
22nd April 2020, 19:39 | #3 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Yeah, using CRF can definitely be a useful tool in this regard. One problem is that some older heuristics struggle with the variable bitrate, especially if they don't make an internal distinction between peak bitrate and actual bitrate. So you'll see this sort of technique used more by companies who either focus only on web or recent devices, or who are able to deliver their own player with custom heuristics. If one is trying to work with the build-in DASH client in older living room devices, things get fraught quickly.
|
22nd April 2020, 20:15 | #4 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Looks like someone at Amazon patented some related ideas . http://pat2pdf.org/patents/pat9712860.pdf
|
22nd April 2020, 20:25 | #5 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
Makes sense. Smart TVs in general (especially older ones) are _ATROCIOUS_ platforms to develop for, especially when you have to live in the sandbox environment they give app developers. Netflix always gets special permission to run their own binary. Nobody else gets that level of integration, so it limits what's possible.
|
24th April 2020, 00:01 | #6 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
|
|
24th April 2020, 16:36 | #8 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Because only Disney has core brands customers are familiar with (Disney, Marvel, Pixar). People just don't know which shows and movies would be on Peacock versus CBS All Access. Building and supporting premimum experience video players for all the devices people want to watch on (so many generations of OS software running on so many different SoCs!) and the content for those (different codecs, packaging, and DRM systems) takes a 1000+ person engineering org by my estimate. |
|
26th April 2020, 22:27 | #9 | Link |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
We are actually doing this as we speak.
We dont have need to go as far as netflix/youtube are going. So we went the CRF + CAP way with a fixed ladder. Gives quite results as we are only allowing newer devices on native players.. |
27th April 2020, 23:20 | #10 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
It is a delicious luxury to only have to worry about newer devices! Of course, the longer a service lasts, the more today's bright and shiny will eventually become tomorrow legacy deadweight holding you back . |
|
28th April 2020, 09:39 | #11 | Link | |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Quote:
-crf 20 -maxrate 15M -bufsize 15M for the top profile |
|
28th April 2020, 09:48 | #12 | Link |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
So for us, we (ISP and Operator) finally got to the point that we had the opportunity to move from a classic static CBR ladder til something better.
We had to decide on how far we wanted to take this into the constant quality domain and we ended up with the conclusion that we wanted to do something between what we have today and what Netflix is doing! We did a lot of tests of 2pass VBR, vs CRF "free" vs CRF Capped, vs CBR both on x264, x265 and commercial encoders like Titan File We ended up with a static ladder based on CRF with a bitrate CAP which gives us a lot of what Netflix is doing, but not going all the way with regards to scene based encoding, title tuning and multiple pre-passes to find the sweet spot etc.. To give an example: Our top HQ 1080p profile/rung is basically CRF19@15mbps CAP with a fallback of CRF22@8mbps CAP etc.. |
28th April 2020, 18:56 | #13 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Oh, if that was the only hard part! But what about the rare cases where the first NAL unit is larger than the following, which is allowed by the spec, but not by some SoC decoders? Or the many, many permutations of DRM encryption that can be fussy with some devices. Or Qualcomm DRM carveouts where you need to allocate on boot enough memory for the maximum resolution & max reference frames. Note that it's the worst case of both; supporting 320x180 with 6 reference frames and 3840x2160 with 3 reference frames requires 3840x2160x6 memory carveout.
And then there are all the heuristics that will treat a 15 Mbps peakrate as all fragments of that stream index being really 15 Mbps, so it won't go to that even when that top stream is at 3 Mbps and there's 12 Mbps of bandwidth. |
28th April 2020, 18:58 | #14 | Link | ||
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Quote:
Note that CRF itself IS scene-based encoding. There are other flavors of it, but the most basic is not to spend bits where they aren't needed. |
||
29th April 2020, 10:06 | #15 | Link | ||
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
Quote:
Last edited by excellentswordfight; 29th April 2020 at 10:59. |
||
29th April 2020, 20:03 | #16 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
In my experience, the DASH clients on older devices are insane. If those devices are important enough, then you need to make playback work well on them. It kind of doesn't matter what the problem really is.
This can ALSO be an issue with poorly implemented DRM, btw. I've definitely seen certain popular settop boxes blow up when fed streams that were 100% valid within the constraints of the Main10 profile @ level 5 (both vbv limits and MaxDpbSize). Reducing the reference frame count was the only solution. This was acknowledged as a firmware bug but I don't know if it's ever been fixed |
30th April 2020, 09:37 | #17 | Link | |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Quote:
1. Using some kind of ML process to detect the scenes, separation 2. Encoding each scene seperatly with different encoding parameters (sweet spot), maybe different gop lengths also?? 3. Use loops of CRF tests against a VMAF threshold to find the baseline for encoding with static/capped VBR. |
|
30th April 2020, 09:42 | #18 | Link | |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Quote:
With that said, what we DO have seen as the major issue is actually Lvl1 WV in the combination with 1080p50 for linear sports broadcasts. Alot of embedded devices running AOSP/ATV cant properly do this without dropping frames in the TEE environment.. Apple IOS/APPLE eats most of our content with little complaint as long as we have ios10+ as the minimum os level req |
|
30th April 2020, 09:43 | #19 | Link | |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Quote:
We dont go past 1080p50 today and have relatively modern devices mentioned in other replies. |
|
1st May 2020, 00:10 | #20 | Link | |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
Quote:
Anyway, we're drifting off-topic here a bit, even if this is still relevant in the context of the risks associated with not using more traditional CBR / heavily capped VBR rate control modes for streaming |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|