Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th June 2012, 08:30   #1  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 601
x264: bit rate on second pass too low - any way to force "minimum bitrate"?

I'm encoding a documentary to go out to Blu-ray, which features very smooth, denoised, low motion video footage (interviews with still camera), with scenes from a grainy film intercut.

I specified 14mbps as the bit rate, 40mbps as the max (as per the BD spec) and a VBV buffer of 30mbps.

Code:
set brate=14000
set speed=slow
set tune=film

x264 --bitrate %brate% --preset %speed% --tune %tune% --bluray-compat
--vbv-maxrate 38000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop
--slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1
-o "%of%" "%if%"

On the first pass, x264 gives a fairly constant bit rate. The result looks good.

On the second pass, during some of the low motion video scenes, it applies a bit rate which is too low - even as low as 1.5mbps at times. The resulting video, while outstandingly impressive for that low a bit rate, has visible artefacts. And since I'm going to optical disc and not download, I'd rather have the better quality.

I got around this by just using the first pass encode. Alternatively I had success using --zones and forcing a bitrate multiplier on the affected areas, but this was pretty time consuming.

I did try setting a max quantizer (--qpmax), but that seems unpredictable; I started getting buffer underflows.

I realise x264 has no minimum bitrate setting. Could a ratecontrol expert fill me in on why that is? And possibly, give a suggestion on how to get around the issue of it applying too low a bitrate without resorting to manual adjustment?

For comparison, I used Cinema Craft SP3 to do the PAL DVD version, and it nailed exactly what I wanted: 8-9mbps for the movie scenes, and 3-4mbps for the video inserts. (The visual quality is consistent). The bitrate graph looked like an up-down-up-down wave. Is there any way to get similar behaviour out of x264?

And also, out of curiosity - why would it be doing this when A) the bit rate was clearly specified as 14mbps, and the rate it used for these scenes - 1/10th of that - was inadequate? It seems to have been fooled by the low amount of motion?

Last edited by Lyris; 25th June 2012 at 08:36.
Lyris is offline   Reply With Quote
Old 25th June 2012, 12:44   #2  |  Link
DeeGee
Registered User
 
DeeGee's Avatar
 
Join Date: May 2004
Location: Finland
Posts: 102
If I'm understanding correctly a new commit in x264 sources, your problem might be already fixed. You just need to find a new enough x264 build to test if it helps.
https://github.com/DarkShikari/x264-...bab6c2b4610454

[edit] And as it seems, I didn't understand it correctly... I guess I'll go back to lurking mode

Last edited by DeeGee; 25th June 2012 at 18:04.
DeeGee is offline   Reply With Quote
Old 25th June 2012, 13:14   #3  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,230
Quote:
Originally Posted by DeeGee View Post
If I'm understanding correctly a new commit in x264 sources, your problem might be already fixed. You just need to find a new enough x264 build to test if it helps.
https://github.com/DarkShikari/x264-...bab6c2b4610454
This commit is about VBV, so it's not specific to 2-Pass encoding.

Also all that VBV does (if enabled) is ensuring that the VBV buffer won't underflow, i.e. it effectively limits the maximum bitrate.

Though it has to be noted that VBV allows (short) bitrate spikes above VBV Maxrate...

BTW: I don't think VBV is the problem here. If VBV effects quality, you would rather see this in "high complexity" (high bitrate) scenes.

Quote:
Originally Posted by Lyris View Post
I'm encoding a documentary to go out to Blu-ray, which features very smooth, denoised, low motion video footage (interviews with still camera), with scenes from a grainy film intercut.

I specified 14mbps as the bit rate, 40mbps as the max (as per the BD spec) and a VBV buffer of 30mbps.

[code]On the first pass, x264 gives a fairly constant bit rate. The result looks good.
On the second pass, during some of the low motion video scenes, it applies a bit rate which is too low - even as low as 1.5mbps at times.
Well, this sounds exactly like what one does expect from a 2-Pass encode.

The first pass is basically 1-Pass ABR encode, so no big surprise that the resulting bitrate is relatively constant.

In the second pass, 2-Pass rate-control is applied to distribute the available bits over the complete video. And of course "low complexity" scenes will get fewer bits than "high complexity" scenes. If you think too few bits are assigned to the "low complexity" scenes, you'll have to tweak x264's 2-Pass RC algorithm.

I think "--qcomp" might be one parameter you could try to tweak here...

Quote:
Originally Posted by Lyris View Post
I did try setting a max quantizer (--qpmax), but that seems unpredictable; I started getting buffer underflows.
If you restrict "--qpmax" too much with VBV enabled, at some point the VBV algorithm won't be able to do its job anymore.

Imagine the VBV algorithm realizes that we need a higher quantizer ("stronger" compression) to avoid an underflow at a certain position, but your settings simply don't allow an even higher quantizer. VBV won't be able to react to the impending underflow. The result will be an actual underflow...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 25th June 2012 at 13:39.
LoRd_MuldeR is offline   Reply With Quote
Old 25th June 2012, 15:52   #4  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,048
Quote:
Originally Posted by Lyris View Post
denoised
then add some noise to bias bitrate.
pandy is offline   Reply With Quote
Old 25th June 2012, 16:36   #5  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by DeeGee View Post
If I'm understanding correctly a new commit in x264 sources, your problem might be already fixed. You just need to find a new enough x264 build to test if it helps.
https://github.com/DarkShikari/x264-...bab6c2b4610454
This commit most likely has nothing to do with the problem.
Quote:
I did try setting a max quantizer (--qpmax), but that seems unpredictable; I started getting buffer underflows.
Setting QP-max will cause buffer underflows because it prevents VBV from doing its job.

If you have a small sample of the video that triggers the problem, please post it and a set of encoding settings that consistently produce the problem (along with pointing me towards whatever frame numbers have the issue).

If you really need a constant bitrate video at the expense of quality, try using CBR mode.

Let me clear up some misconceptions here, because the previous post was extremely misleading and largely incorrect:

Quote:
Originally Posted by LoRd_MuldeR View Post
Though it has to be noted that VBV allows (short) bitrate spikes above VBV Maxrate...
Only if you incorrectly define VBV maxrate.
Quote:
Originally Posted by LoRd_MuldeR View Post
BTW: I don't think VBV is the problem here. If VBV effects quality, you would rather see this in "high complexity" (high bitrate) scenes.
This is not necessarily true.
Quote:
Originally Posted by LoRd_MuldeR View Post
The first pass is basically 1-Pass ABR encode, so no big surprise that the resulting bitrate is relatively constant.
This is not necessarily true. 1-pass bit distribution may be suboptimal, but it is not "relatively constant".
Quote:
Originally Posted by LoRd_MuldeR View Post
In the second pass, 2-Pass rate-control is applied to distribute the available bits over the complete video. And of course "low complexity" scenes will get fewer bits than "high complexity" scenes. If you think too few bits are assigned to the "low complexity" scenes, you'll have to tweak x264's 2-Pass RC algorithm.
The part of x264's RC algorithm that affects bit distribution, macroblock-tree, affects both passes identically. It has absolutely nothing to do with 1-pass or 2-pass.
Quote:
Originally Posted by LoRd_MuldeR View Post
I think "--qcomp" might be one parameter you could try to tweak here...
This affects both passes identically.

Last edited by Dark Shikari; 25th June 2012 at 16:43.
Dark Shikari is offline   Reply With Quote
Old 25th June 2012, 16:42   #6  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Isnt it an idea to use one-pass crf mode instead of 2-pass bitrate mode ?
G_M_C is offline   Reply With Quote
Old 25th June 2012, 16:42   #7  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by G_M_C View Post
Isnt it an idea to use one-pass cq mode instead of 2-pass bitrate mode ?
CRF and 2-pass use the exact same bit distribution algorithm. If you do a CRF encode and a 2-pass encode at the same bitrate, the results will be nearly identical.

Hypothetically, there could be a bug in the 2-pass code that didn't show up in CRF, but outside of a hypothetical bug workaround, this is probably not a good suggestion.
Dark Shikari is offline   Reply With Quote
Old 25th June 2012, 16:54   #8  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Quote:
Originally Posted by Dark Shikari View Post
CRF and 2-pass use the exact same bit distribution algorithm. If you do a CRF encode and a 2-pass encode at the same bitrate, the results will be nearly identical.

Hypothetically, there could be a bug in the 2-pass code that didn't show up in CRF, but outside of a hypothetical bug workaround, this is probably not a good suggestion.
Ok, I thought crf is comparable to a constant quality mode.

My idea is that if you set an high enough quality (low enough setting of --crf, say --crf 12), based on TS' example, the high detail and the denoised scenes would come out with comparable quality (both crf 12); while still maintaining BD specs, if other options arent changed. This is what TS' actually wants isnt it ?
G_M_C is offline   Reply With Quote
Old 25th June 2012, 16:57   #9  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,667
Quote:
Originally Posted by G_M_C View Post
Ok, I thought crf is comparable to a constant quality mode.
It is. 2-pass is also constant quality. You just ask x264 for a bitrate instead of a quality.

Again, they use the same bit distribution algorithm.
Dark Shikari is offline   Reply With Quote
Old 25th June 2012, 17:59   #10  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Hmmm, i see. Thx for the clarification DS !
G_M_C is offline   Reply With Quote
Old 25th June 2012, 18:16   #11  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 601
Quote:
Originally Posted by pandy View Post
then add some noise to bias bitrate.
Hmm, I shouldn't have to change the source on a technical or aesthetic level to fool the rate control algorithm. (I don't mean that to sound ungrateful, BTW).

Quote:
If you have a small sample of the video that triggers the problem, please post it and a set of encoding settings that consistently produce the problem (along with pointing me towards whatever frame numbers have the issue).
Thanks for looking into this, DS - PM incoming.
Lyris is offline   Reply With Quote
Old 25th June 2012, 20:00   #12  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,485
I make actually test on x264 Rate Control and optimized profil for Blu Ray encoding. You can try that and make little feedback please:

x264.exe --slow-firstpass --bframe 1 --b-pyramid "strict" --b-adapt 2 --weightp 1 --ref 4 --direct auto --deblock -1:-3 --bitrate 14000 --pass 1 --stats "test.log" --no-mbtree --qcomp 0.25 --ipratio 1.00 --pbratio 1.00 --min-keyint 1 --partitions "all" --me "hex" --subme 7 --no-fast-pskip --no-dct-decimate --deadzone-inter 2 --deadzone-intra 2 --trellis 1 --psy-rd 1.0:0.25 --aq-mode 1 --aq-strength 0.5 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --vbv-init 0.9 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --psnr --ssim -o test.264 Lossless.avs

x264.exe --bframe 1 --b-pyramid "strict" --weightp 1 --ref 4 --direct auto --deblock -1:-3 --bitrate 14000 --pass 3 --stats "test.log" --no-mbtree --qcomp 0.25 --ipratio 1.00 --pbratio 1.00 --min-keyint 1 --partitions "all" --me "umh" --subme 10 --no-fast-pskip --no-dct-decimate --deadzone-inter 2 --deadzone-intra 2 --trellis 2 --psy-rd 1.0:0.25 --aq-mode 1 --aq-strength 0.5 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --vbv-init 0.9 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --psnr --ssim -o test.264 Lossless.avs
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 26th June 2012, 12:49   #13  |  Link
jethro
Registered User
 
Join Date: Mar 2006
Posts: 101
Min bitrate is a heavy-handed tool to enforce a minimal level of quality. Maybe it is possible to set minimal byte size for individual macroblocks to prevent them being quantized to oblivion?
jethro is offline   Reply With Quote
Old 26th June 2012, 16:31   #14  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 601
Quote:
Min bitrate is a heavy-handed tool to enforce a minimal level of quality.
I understand that there may be arguments against it in that it's not a "neat" solution, but the fact is, it works very well to avoid losing quality in CCE when I'm using it to produce DVDs. Keep in mind the goal is to fill both (or one) layer of an optical disc, any savings beyond that are nearly irrelevant when it comes to physical media. If there's a potentially wasteful but more consistent choice, that's the one that makes most sense in this area

Haven't had any luck with other encoding parameters yet. It looks like I might need to go through and manually assign a higher bit rate to these scenes in order to fit this on the disc.

Last edited by Lyris; 26th June 2012 at 17:07.
Lyris is offline   Reply With Quote
Old 26th June 2012, 17:18   #15  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,485
even with really low qcomp ... ???
With qcomp = 0 you have really constant bitrate.
Try my profil with lower qcomp.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 1st July 2012, 19:00   #16  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 601
I did try some of your settings. That resulted in the opposite situation: the grainy film clips degraded in quality, whereas the interviews looked better (in line with what DS told me to expect).

What actually helped in the end was using the --psnr setting. Probably not ideal, but the closest thing I could find which worked in time for submission.
Lyris is offline   Reply With Quote
Old 1st July 2012, 22:41   #17  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,485
Quote:
Originally Posted by Lyris View Post
I did try some of your settings. That resulted in the opposite situation: the grainy film clips degraded in quality, whereas the interviews looked better (in line with what DS told me to expect).

What actually helped in the end was using the --psnr setting. Probably not ideal, but the closest thing I could find which worked in time for submission.
Well try to find the best value of qcomp for your eyes.
Anyway in your case I think that the average must be higher to have acceptable quality in complete area.
--pnsr command line don't mean that the profil is optimised for psnr but just that psnr is calculate.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 1st July 2012, 23:04   #18  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,230
If he indeed means that using "--psnr" solved his problem, then this would be a typical placebo. Adding "--psnr" does not change the output at all.

If, however, he means that he is using "--tune psnr" (which optimizes the output for "--psnr"), then it's not very surprising that he is seeing a significant difference.

That's because "--tune psnr" essentially disables all the Psy optimizations of x264! But completely disabling the Psy optimizations usually is a very bad choice

(Instead I would keep the Psy optimizations enabled and instead try to tweak the Psy parameters a little, e.g. "--psy-rd" or "--aq-strength")
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 1st July 2012 at 23:42.
LoRd_MuldeR is offline   Reply With Quote
Old 2nd July 2012, 12:58   #19  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 601
I didn't know that - I was confusing it with the --tune psnr you mentioned. Thanks for the clarification.

In that case, it'd have to be the extra zones (with bitrate multipliers) I specified which made the difference.
Lyris is offline   Reply With Quote
Old 11th July 2012, 00:02   #20  |  Link
nibus
Telewhining
 
Join Date: Mar 2010
Posts: 272
You might also try disabling mbtree (--no-mbtree). This will give more bitrate to those denoised portions and limit qcomp swinging the bitrate so much.
nibus is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.