Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th May 2009, 18:42   #1  |  Link
floz23
Registered User
 
Join Date: Feb 2007
Posts: 14
x264 VBV -- Progressive download and Streaming

Hi everyone,

I've been a longtime user of megui and x264, and I have to admit that the work that everyone has put into it is quite amazing! From the programmers to the support staff on all levels, thanks!

But, I have a question that I've only been able to get partial answers to, and nothing totally conclusive. On the issue of VBV and progressive http streaming, and real "rtsp:" streaming.

I'm basically trying to figure out how to set the VBV settings to restrict the maximum bitrate of my .mp4 files. In the case of JWPlayer, i can set, for instance, a 3 second buffer. Should I set the vbv-bufsize to match that 3 seconds worth of buffer?

I understand that VBV is generally used for specific hardware requirements, but what about in the cases of progressive and rstp streaming?

Thanks,
-Adam
floz23 is offline   Reply With Quote
Old 17th May 2009, 19:08   #2  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
For progressive download, which isn't really streaming, you don't need VBV.

For streaming, set bufsize equal to (maxrate) * (number of seconds the player is set to buffer).
Dark Shikari is offline   Reply With Quote
Old 17th May 2009, 20:17   #3  |  Link
floz23
Registered User
 
Join Date: Feb 2007
Posts: 14
Excuse my ignorance, but why would I not need VBV for progressive download? If I don't need VBV for progressive download, how could I ensure that my bitrate wont spike too high for a given buffer length?

Regards,
-Adam
floz23 is offline   Reply With Quote
Old 17th May 2009, 20:25   #4  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by floz23 View Post
Excuse my ignorance, but why would I not need VBV for progressive download? If I don't need VBV for progressive download, how could I ensure that my bitrate wont spike too high for a given buffer length?

Regards,
-Adam
Generally, for progressive download, you are treating the video like a download: fully variable bitrate, ideally with the average bitrate well below the downloader's connection speed. You're hoping that by the time you've reached any such a spike, you've buffered far enough ahead that this won't be a problem.

With streaming, you can't do this, since the client can't buffer arbitrarily far ahead.

But wait, you ask. What if there's a spike at the beginning? Well then the user has to buffer for another second or two . Neither Youtube nor Facebook use VBV in their progressive download.

The reason for this is twofold:

1. The impact of CBR on quality is tremendous.
2. You could modify an encoder to model the interaction described here (effectively an infinite-size VBV that starts empty), but that takes effort.

Last edited by Dark Shikari; 17th May 2009 at 20:30.
Dark Shikari is offline   Reply With Quote
Old 18th May 2009, 01:32   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,109
Quote:
Originally Posted by Dark Shikari View Post
Generally, for progressive download, you are treating the video like a download: fully variable bitrate, ideally with the average bitrate well below the downloader's connection speed. You're hoping that by the time you've reached any such a spike, you've buffered far enough ahead that this won't be a problem.
Well, we can do better than just hope, can't we. With long form content, an unbound peak can give you cases where the current bitrate is several times that of average for a sustained period. Progressive download's really an intermediate between pure download (where you just care about file size) and pure streaming (where you just care about peak bitrate).

You can also think of VBR as a CBR with some really easy parts and without padding bits. There's no reason not to set a VBV as well as ABR/PBR, where the VBV is PBR * buffer duration.

When doing progressive download to devices with hardware decoders, you need to respect the VBV as well.

This is fine for streaming as well if you don't want to spend the bandwidth budget making the easy bits higher than a certain quality. Thus, you could make sure it's streamable with 1000 Kbps of bandwidth, but the utilized bandiwidth could be lower much of the time.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th May 2009, 12:13   #6  |  Link
JohannesL
AviSynth/x264 user
 
JohannesL's Avatar
 
Join Date: Jan 2009
Posts: 149
So maybe no VBV and a low qcomp is ideal for progressive downloads.
__________________
archlinux
JohannesL is offline   Reply With Quote
Old 18th May 2009, 13:29   #7  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by benwaggoner View Post
When doing progressive download to devices with hardware decoders, you need to respect the VBV as well.
Of course, but that's a different VBV. There are really two VBVs here: one for the connection and one for the decoder. This thread is about the connection one.
Dark Shikari is offline   Reply With Quote
Old 18th May 2009, 16:56   #8  |  Link
floz23
Registered User
 
Join Date: Feb 2007
Posts: 14
thanks everyone. What is the VBV_maxrate limited by? Frame, gop, or seconds?

I use this program to examine the bitrate of my resulting .mp4's
http://www.winhoros.de/docs/bitrate-viewer/

Even if I set a VBV_maxrate it seems that the maxrate is being respected only in regards to the GOP, and it seems to be about 200k higher than the maxrate. So for instance, if I set a maxrate of 1500k and a buffer of 4500k, when I use the bitrate viewer, it will show a rate of 1700k maximum bitrate when I put it on GOP mode. If I select frame or seconds mode, i still see huge spikes on the I frames.

Thanks for all the input, right now my website is running on purely progressive download, and I have to make a decision very soon to switch to a more advanced progressive solution like this one:
http://h264.code-shop.com/trac

Or go for a full-on wowza RTSP solution.

Regards,
-Adam

PS- I'm currently using MEGUI as a front-end to my encoding, and using automated 2-pass mode, using the VBV settings that are exposed in the GUI. I check the resulting command lines and they SEEM ok, but could MEGUI be giving me problems in my VBV experiments?
floz23 is offline   Reply With Quote
Old 18th May 2009, 17:01   #9  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by floz23 View Post
thanks everyone. What is the VBV_maxrate limited by? Frame, gop, or seconds?
Second.
Quote:
Originally Posted by floz23 View Post
I use this program to examine the bitrate of my resulting .mp4's
http://www.winhoros.de/docs/bitrate-viewer/Even if I set a VBV_maxrate it seems that the maxrate is being respected only in regards to the GOP, and it seems to be about 200k higher than the maxrate. So for instance, if I set a maxrate of 1500k and a buffer of 4500k, when I use the bitrate viewer, it will show a rate of 1700k maximum bitrate when I put it on GOP mode. If I select frame or seconds mode, i still see huge spikes on the I frames.
That's because the bitrate viewer isn't using the correct bufsize.
Dark Shikari is offline   Reply With Quote
Old 18th May 2009, 18:31   #10  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,109
Quote:
Originally Posted by Dark Shikari View Post
Of course, but that's a different VBV. There are really two VBVs here: one for the connection and one for the decoder. This thread is about the connection one.
Ah. In which case we're in agreement .
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th October 2009, 18:50   #11  |  Link
Riki
Registered User
 
Join Date: Dec 2007
Posts: 152
Hi Dark, I was hoping for a little more info on what you said here:

Quote:
Originally Posted by Dark Shikari View Post
For streaming, set bufsize equal to (maxrate) * (number of seconds the player is set to buffer).
Im using wowza with streaming and also the dual buffering technique where you set 2 buffering values on the flash player (2,10) and the first value is the initial buffer before the video starts, and the 2nd value is what it will buffer in the background while the video is playing or paused.

So would it be a good idea to set the buffer size *2, *10, or somewhere in the middle. Also, what is the harm in setting this too high.

Thanks

Last edited by Riki; 20th October 2009 at 18:50. Reason: spelling
Riki is offline   Reply With Quote
Old 21st October 2009, 12:43   #12  |  Link
Riki
Registered User
 
Join Date: Dec 2007
Posts: 152
Quote:
Originally Posted by Dark Shikari View Post
Second.
That's because the bitrate viewer isn't using the correct bufsize.
Does anyone know a program that will show the average and peak bitrates of a mp4 video? Bitrate viewer doesnt seem to be accurate at all. :/
Riki is offline   Reply With Quote
Old 21st October 2009, 19:31   #13  |  Link
RunningSkittle
Skittle
 
RunningSkittle's Avatar
 
Join Date: Mar 2008
Posts: 539
http://forum.doom9.org/showthread.php?t=144409
RunningSkittle is offline   Reply With Quote
Old 21st October 2009, 20:28   #14  |  Link
Riki
Registered User
 
Join Date: Dec 2007
Posts: 152
ok purchased, when I get my activation Ill test it out and post my results.
Riki is offline   Reply With Quote
Old 23rd October 2009, 19:51   #15  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,109
Anyway, in general I'm fan of having the peak bitrate at least 1.5x the average, with a buffer duration at least as big as the GOP length (so the VBR model can allocate bits for at least one full GOP).

With our modern byte-range access progressive download mechanisms, using a GOP and buffer duration around 4-6 seconds seems pretty optimal in practice. Much higher and random access can get slow, and much less and compression efficiency can be badly impacted.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th October 2009, 16:27   #16  |  Link
aapp
Registered User
 
Join Date: Oct 2009
Posts: 5
Quote:
Originally Posted by Dark Shikari View Post
Neither Youtube nor Facebook use VBV in their progressive download.
Are you sure this is still the case? I have seen this recently on Facebook videos:

Code:
Video Encoder (Settings) -> cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / 
subme=7 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 
8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / 
mbaff=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / keyint=150 / 
keyint_min=30 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=1316 / ratetol=1.0 / 
qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=1579 / 
vbv_bufsize=6581 / ip_ratio=1.40 / aq=1:1.00 / cabac=1 / ref=3 / deblock=1:0:0 / 
analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / 
chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / 
nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / 
wpredb=1 / keyint=150 / keyint_min=30 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / 
bitrate=1316 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / 
qblur=0.5 / vbv_maxrate=1579 / vbv_bufsize=6581 / ip_ratio=1.40 / aq=1:1.00

Date of Original Encoding -> UTC 2009-10-17 00:09:26
aapp is offline   Reply With Quote
Old 28th October 2009, 17:11   #17  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by aapp View Post
Are you sure this is still the case? I have seen this recently on Facebook videos:
Yes, I made that change myself, for throttling-related purposes
Dark Shikari is offline   Reply With Quote
Old 28th October 2009, 17:20   #18  |  Link
aapp
Registered User
 
Join Date: Oct 2009
Posts: 5
Quote:
Originally Posted by Dark Shikari View Post
Yes, I made that change myself, for throttling-related purposes
Ah =)

Are the VBV values based on anything other than the target bitrate? I'm wondering what the logic is behind ~120% for maxrate and ~420% as reasonable values for capping bitrate spikes, or am I extrapolating too much from one video?
aapp is offline   Reply With Quote
Old 28th October 2009, 17:28   #19  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by aapp View Post
Ah =)

Are the VBV values based on anything other than the target bitrate?
Nope.
Dark Shikari is offline   Reply With Quote
Old 28th October 2009, 17:33   #20  |  Link
aapp
Registered User
 
Join Date: Oct 2009
Posts: 5
Thanks!
aapp is offline   Reply With Quote
Reply

Tags
bitrate, download, rstp, streaming, vbv

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.