Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th June 2010, 12:36   #1  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,464
Still severe bitrate loss with aq-mode 2

Okay, I'm still experiencing severe bitrate loss with aq-mode 2.

Yesterday I re-encoded Blade Runner -- The Final Cut again (not having made the same mistake of throwing away the old encode first).

Old Encode: CRF 16, aq=1 = 16.2G
New Encode: CRF 14, aq=2 = 11.5G

As you can tell from the MediaInfo below, for the New Encode I lowered CRF by 2 (which ought to add ~25% filesize/bitrate). And still, the bitrate under aq=2 drops drastically below that of the Old Encode (by ca. 30%).

So, simple question, what's causing this? Is the newer x264 that much more efficient? I just can't explain this huge bitrate loss (overall nearly 6 Mbps).

-----
Old Encode:
Code:
Video
Bit rate                         : 19.7 Mbps
Maximum bit rate                 : 35.0 Mbps
Stream size                      : 16.2 GiB (71%)
Writing library                  : x264 core 66 r1093M 1df50b9
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=tesa / subme=9 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40(pre) / rc=crf / crf=16.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / vbv_maxrate=35000 / vbv_bufsize=35000 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
New Encode:
Code:
Video
Bit rate                         : 14.0 Mbps
Maximum bit rate                 : 40.0 Mbps
Stream size                      : 11.5 GiB (64%)
Writing library                  : x264 core 98 r1649 c54c47d
Encoding settings                : cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x133 / me=tesa / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=6 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / vbv_maxrate=60000 / vbv_bufsize=70000 / crf_max=0.0 / ip_ratio=1.40 / aq=2:1.00 / nal_hrd=non
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 29th June 2010, 12:54   #2  |  Link
me7
Registered User
 
Join Date: Mar 2007
Posts: 217
Do you see a loss of quality? Try to compare the encodes with AviSynth:

Code:
a = FFVideoSource("clip1").Subtitle("A")
b = FFVideoSource("clip2").Subtitle("B")

Interleave(a, b).AssumeFPS(1)
me7 is offline   Reply With Quote
Old 29th June 2010, 13:06   #3  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,251
Asarian, as you are using CRF mode, so you cannot make any assumption about the output size, except that a lower CRF value yields better quality (at the cost of bigger file) and a higher CRF value yields a smaller file (at the cost of reduced quality). Moreover you cannot expect that the same CRF value still gives the same file size, after you changed other options, such as AQ mode! CRF isn't an exact "constant quality" mode. Such mode doesn't exist! The same CRF value will give roughly the same quality for different source (of the same nature), as long as you don't change any other options. But that's it! It's also wrong to believe that using "slower" settings will necessarily result in a smaller file at the same CRF value. Actually the opposite might happen.

If you want to know if AQ mode 1 or AQ mode 2 works better for you, you must visually(!) compare them in 2-Pass mode. And if you do a visual comparison, always make sure you compare files of the same size. Then, after you have decided between AQ mode 1 or AQ mode 2, you can start using your preferred AQ mode in CRF mode. The final step is re-adjusting the CRF value to your needs...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 29th June 2010 at 13:11.
LoRd_MuldeR is offline   Reply With Quote
Old 29th June 2010, 13:23   #4  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,464
Quote:
Originally Posted by LoRd_MuldeR View Post
Asarian, as you are using CRF mode, so you cannot make any assumption about the output size, except that a lower CRF value yields better quality (at the cost of bigger file) and a higher CRF value yields a smaller file (at the cost of reduced quality).
Thanks for answering.

I understand what you're saying. I'm just surprised that even lowering CRF by two, for aq-mode 2, still yields a file which is so much smaller. For another movie (Evangelion 1.11) I had tested and calculated that I needed to go from CRF 16 to 14.2, in order to compensate for the bitrate loss under aq=2. Seems for this Blade Runner Blu-Ray I need to lower CRF even significantly further. So, predicting a 'general-compensation factor' for aq=2 proves more difficult than I thought.

For the record: I'm not judging aq=2 (yet). At this stage I'm just baffled why it appears to be needing/wanting to use so much less bitrate.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 29th June 2010, 13:26   #5  |  Link
rack04
Registered User
 
Join Date: Mar 2006
Posts: 1,538
Quote:
Originally Posted by asarian View Post
For the record: I'm not judging aq=2 (yet). At this stage I'm just baffled why it appears to be needing/wanting to use so much less bitrate.
My question for you is why do you think it is necessary to have more bitrate? I think once you start visually inspecting the encodes this will be more clear.
rack04 is offline   Reply With Quote
Old 29th June 2010, 13:40   #6  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,251
Quote:
Originally Posted by asarian View Post
I understand what you're saying. I'm just surprised that even lowering CRF by two, for aq-mode 2, still yields a file which is so much smaller. For another movie (Evangelion 1.11) I had tested and calculated that I needed to go from CRF 16 to 14.2, in order to compensate for the bitrate loss under aq=2. Seems for this Blade Runner Blu-Ray I need to lower CRF even significantly further. So, predicting a 'general-compensation factor' for aq=2 proves more difficult than I thought.
IMHO trying to hit a specific target bitrate by iteratively adjusting CRF isn't the intended (convenient) way of using CRF mode. If you want to hit a specific (average) bitrate, then 2-Pass mode is for you. But if you want to go the CRF route, you should adjust the CRF value for visual quality, not for bitrate! So first decide between AQ1/AQ2 and all the other options (using 2-Pass mode for visual comparisons), then find the highest possible CRF for your settings that still gives acceptable quality. And then use that CRF value without looking at the bitrate...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊
LoRd_MuldeR is offline   Reply With Quote
Old 29th June 2010, 14:00   #7  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,464
Quote:
Originally Posted by rack04 View Post
My question for you is why do you think it is necessary to have more bitrate?
Well, for one, because everyone keeps telling me I need to compare files of (roughly) equal bitrates in order to judge the effects of aq=2 properly. :)

Quote:
Originally Posted by LoRd_MuldeR View Post
IMHO trying to hit a specific target bitrate by iteratively adjusting CRF isn't the intended (convenient) way of using CRF mode. If you want to hit a specific (average) bitrate, then 2-Pass mode is for you.
I don't really target for a specific filesize/bitrate. The only time when I'm concerned about it, is times like these, when going to aq-mode 2 makes the output file so much smaller, even at lower CRF. If someone were to come out and say: "Don't worry, aq-2 is supposed to use less bitrate!", then I'd be happy too. Could simply be that auto-variance truly is that much more efficient; but I'm not knowledgeable enough in these matters to convince myself of that to the point of not worrying about it. :P
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 29th June 2010, 14:11   #8  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
In lossy video compression there's no 'number' that works better than your eye. Look closely with your eyes and decide if the bitrate is enough to obtain your standard of 'good quality'. If not, raise the bitrate/change settings/etc until you're satisfied.
mp3dom is offline   Reply With Quote
Old 29th June 2010, 14:21   #9  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,464
Quote:
Originally Posted by mp3dom View Post
In lossy video compression there's no 'number' that works better than your eye. Look closely with your eyes and decide if the bitrate is enough to obtain your standard of 'good quality'. If not, raise the bitrate/change settings/etc until you're satisfied. :)
Unlike the Evangelion 1.11 disc, with (as you know) its very specific gradient/dither issues, here aq=2 actually, visually at least, looks better. ;) Dunno how that's possible, but here aq=2 definitely seems to produce superior visual quality.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 29th June 2010, 14:23   #10  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,251
Quote:
Originally Posted by asarian View Post
If someone were to come out and say: "Don't worry, aq-2 is supposed to use less bitrate!", then I'd be happy too.
I wouldn't say AQ2 is generally supposed to use less bitrate.

But it's important to understand that CRF mode never was supposed the give the same filesize/bitrate for the same CRF value when other influential options are changed.

Or in other words: After you switched form AQ1 to AQ2, your old CRF value became meaningless. Hence you might have to re-adjust your CRF value.


Quote:
Originally Posted by asarian View Post
Could simply be that auto-variance truly is that much more efficient; but I'm not knowledgeable enough in these matters to convince myself of that to the point of not worrying about it. :P
You cannot make any assumptions about compression efficiency by looking at the filesize of a CRF encode. As said before, CRF mode isn't a "constant quality" mode. Especially NOT when you change other influential options. So if you compare files of the same CRF value, but with different AQ mode, you might very well be comparing files of different quality! If AQ mode 2 produces a smaller file (at the same CRF), but looks worse at the same time, you cannot say that it is more efficient. Nor can you say the opposite. It's simply impossible to draw conclusions from such a test.

If you want to draw useful conclusions about compression efficiency, you need either:
(a) Two files that have identical quality (hard to produce, because "identical quality" is difficult to find)
(b) Two files that have identical size/bitrate (easy to produce, thanks to 2-Pass mode)

In case (b) you will either notice that one file has significant better/worse quality than the other one (in this case it's quite clear which setting compressed more/less efficient) or you will notice that you cannot clearly tell which of the two files looks better/worse. In the latter case the conclusion would be that the impact on efficiency of the settings under test is negligible.
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 29th June 2010 at 14:36.
LoRd_MuldeR is offline   Reply With Quote
Old 29th June 2010, 15:28   #11  |  Link
Rumbah
Registered User
 
Join Date: Mar 2003
Posts: 481
You changed too many settings to compare. Your loss in bitrate might also be connected to more b-frames, more refs, better subme, higher trellis, mbtree, weightp, etc.
Rumbah is offline   Reply With Quote
Old 29th June 2010, 15:52   #12  |  Link
nurbs
Registered User
 
Join Date: Dec 2005
Posts: 1,460
I didn't notice that there are more than 500 x264 revisions between the encodes. MB-Tree would be a good candidate. Lots of people complained about much smaller filesizes at the same RF when MB-Tree was added.

edit:
Anyway, I still agree with everything Lord Mulder said in this thread.
nurbs is offline   Reply With Quote
Old 29th June 2010, 17:43   #13  |  Link
Soichiro
Registered User
 
Join Date: May 2007
Posts: 75
Seriously... If you raise all those settings and update x264 by 500 revisions, of course you're going to get a smaller filesize (or rather, better compression). A year and a half of x264 updates is definitely going to make a difference. On top of that, crf has been adjusted at least a few times over that period. Try again using the same x264 revision (and same settings except for AQ-mode) for both encodes and tell us if you still get the same results.

Last edited by Soichiro; 29th June 2010 at 17:47.
Soichiro is offline   Reply With Quote
Old 29th June 2010, 18:11   #14  |  Link
Brazil2
Registered User
 
Join Date: Jul 2008
Posts: 549
Quote:
Originally Posted by nurbs View Post
Lots of people complained about much smaller filesizes at the same RF when MB-Tree was added.
Yes, it's well known that MBTree had an impact on CRF behaviour.

And there are also some important differences between the settings you have used:
Code:
Old:			New:
ref=3			ref=4
trellis=1		trellis=2
bframes=3		bframes=8
b_pyramid=1		b_pyramid=2
Brazil2 is offline   Reply With Quote
Old 29th June 2010, 18:34   #15  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,229
Also the newer version has weighted p-frames.

Other settings which were changed that affects a comparisona are (albeit very slightly) p-skip, subme, and the me-range chosen.

A real comparison determining the difference between aq-modes can only be done with the same version, same settings except having one set to 2 and the other set to 1

If you want to compare just the versions, feature independent, you'd have to disable the new features (mb-tree, weighted p-frames etc), and use the same settings.

Its not the filesize that matters, its what it actually looks like if the new one looks better (even with the higher settings), its irrelevant whether its smaller or not !

Changing ME from TESA to UMH may make encoding a little less slow
burfadel is offline   Reply With Quote
Old 29th June 2010, 19:08   #16  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,464
Okay, I'm convinced. :) Indeed, too much settings have changed since the previous encode. I should do a new re-encode, with the same settings, only with aq=1 this time.

Thank you all for your constructive ideas and input!
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Reply

Tags
aq-mode, bitrate, crf, x264

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.