Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th April 2019, 08:43   #41  |  Link
infoeater
Registered User
 
Join Date: May 2007
Posts: 53
Maybe you could lower qcomp value? 0, will produce nearly constant bitrate without any peaks, 0.3 may be enough to prevent such high bitrate peaks. It will be suboptimal, but still better then adding artificial noise (providing both better compression/quality ratio and similarity to original).
infoeater is offline   Reply With Quote
Old 20th April 2019, 13:38   #42  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
It can be a workaround, but only if the average bitrate is close to the maximum bitrate allowed.
It doesn't make sense to have a "constant bitrate" at 20 Mbps when I can have bitrates up to 40 Mbps according to the scene complexity.
The risk is to have a way worse visible quality, since this "quality drop" problem I get affects the first second/gop of a scene, the time needed for the encoder to "recover" its quality.
mp3dom is offline   Reply With Quote
Old 21st April 2019, 10:20   #43  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
As far as i know, all the necessary data have been provided to the devs to reproduce and analyse the issue, months ago.
If any dev is reading this thread, is it possible to have any feedback on the status of this issue ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 21st April 2019, 12:10   #44  |  Link
infoeater
Registered User
 
Join Date: May 2007
Posts: 53
Quote:
Originally Posted by mp3dom View Post
It can be a workaround, but only if the average bitrate is close to the maximum bitrate allowed.
It doesn't make sense to have a "constant bitrate" at 20 Mbps when I can have bitrates up to 40 Mbps according to the scene complexity..
First default qcomp=0.6 provides some bitrate variability. Then if it exceeds limits, it's limited by broken bitrate management. Probably not only in places where you see it, but many more, where quality drop is not so noticeable.

When you lower qcomp for whole movie, bitrate variability before broken bitrate management is stabilized, so it's not used that much. qcomp=0.0 will give you nearly constant bitrate, but it will be indeed needed only if your average bitrate is close to maximum bitrate allowed.

When you lower qcomp, to say qcomp=0.3, bitrate is stabilized in quite smart way. It's taken much more from places with bitrate above allowed maximum (places with highest bitrate), and it's taken first from places where it will be less noticeable. This bitrate is not totally wasted, but distributed to places with low bitrate. Compare it to adding artificial noise, which will change distribution of bitrate, but also just waste a lot of it (encoding noise is very bitrate-demanding, in fact it works just because it is so bitrate demanding, that complexity of rest of the video becomes less important) to lower quality.

While lowering qcomp is suboptimal (because most bitrate is taken from scenes with bitrate 80 Mbps, but some (but less) also from scenes with 40 Mbps, it is designed to change overall bitrate distribution, not enforce maximum bitrate), the loss in quality is not big.

Quote:
Originally Posted by mp3dom View Post
The risk is to have a way worse visible quality, since this "quality drop" problem I get affects the first second/gop of a scene, the time needed for the encoder to "recover" its quality
qcomp when lowered just as needed for this particular encode will probably not do that. The optimal value depends on particular video (higher difference between maximum allowed and average bitrate -> higher qcomp, more problematic scenes -> lower qcomp). But there is a great chance you will find value which is ok for 99% videos, or you will learn how to adjust it. You can always check on two levels faster preset (which will speed up many times). You can encode video without bitrate restrictions first to see level of bitrate before broken bitrate management. Give it a chance

Other way is limiting bitrate of high complexity scene before problem with zones. If you have time to find optimal bitrate multiplier<1 and length of scene for each scene for each encode it will be nearly optimal. It's just what working bitrate management should do. Note that in fact you need to limit it's bitrate only a bit, but you may need very low bitrate multiplier to accomplish this, because it affect not actual bitrate, but bitrate before broken bitrate management.

Also note, that perfect quality even without this problem might not be sometimes possible without removing details from source, because maximum allowed bitrate might be just not enough.

Last edited by infoeater; 21st April 2019 at 12:15.
infoeater is offline   Reply With Quote
Old 21st April 2019, 14:47   #45  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
I checked a whole full 2hrs movie, on a frame by frame basis also with debuggin tools, to spot this issue, to see when and where it happens and created, also thanks to jpsdr, a video sequence that can easily trigger the issue.
The problem refers mainly to the vbv management and, by reflection, the bitrate. When there's a very complex and lengthy scene (mainly "particles effects", dust or a "grainy" scene that easily eats bitrate, when both previous and next scenes doesn't have grain or this kind of complexity) the bitrate go well over the allowed max bitrate thanks to the vbv. This is fine and expected, but the problem is that the vbv reach its very maximum, it hits its fullness and so, when the complex scene ends, the next one (simpler) is used by x264 to flush the buffer. In that circumstance, the vbv go from 30 Mbps (its maximum on bluray) to almost 10-12 Mbps in one gop/1 second, thanks to an extreme bitrate drop. From the next gop, both vbv and bitrate regain their right values. In that 1 second/gop quality drop, the qp values are very high (in the 24-25 range) whereas otherwise it would have been 10-12, so compression artifacts are very visible.
The qpcomp can't be changed by zones, I can't encode different segments either (on a 2 hrs movie I risk to have 6-7 segments ). Lowering bitrate using zones for the complex scenes may be not enough either and it's not always working. Raising the bitrate for the next simpler scene doesn't work at all (seems to just ignore the zone, probably because the vbv status doesn't allow this).

I'll try to encode with a more lower qcomp and see if this happens anyway or not.

It's a serious problem especially for animation, when you go from simple to complex scene very ofter... quite unlikely this happens on standard movies.

Last edited by mp3dom; 21st April 2019 at 16:40.
mp3dom is offline   Reply With Quote
Old 22nd April 2019, 04:18   #46  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Did you try rc-lookahead 250 ?
StvG is offline   Reply With Quote
Old 22nd April 2019, 14:36   #47  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
With bluray encoding, the max value of rc-lookahead is one second (24).

This is the vbv/bitrate ratio:

Last edited by mp3dom; 22nd April 2019 at 14:56.
mp3dom is offline   Reply With Quote
Old 23rd April 2019, 00:08   #48  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by mp3dom View Post
The qpcomp can't be changed by zones, I can't encode different segments either (on a 2 hrs movie I risk to have 6-7 segments ). Lowering bitrate using zones for the complex scenes may be not enough either and it's not always working. Raising the bitrate for the next simpler scene doesn't work at all (seems to just ignore the zone, probably because the vbv status doesn't allow this).

I'll try to encode with a more lower qcomp and see if this happens anyway or not.

It's a serious problem especially for animation, when you go from simple to complex scene very ofter... quite unlikely this happens on standard movies.
Maybe use the qpfile to raise the QP of the difficult region so it doesn't exhaust the VBV so much? Manually lowering QP for the following easy region might also help.

I'm curious if the quality issues are due to the first IDR of the lower quality region being specifically low; IDR is very important in animation! Just setting that frame's QP might change things a bunch.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 23rd April 2019, 00:19   #49  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Quote:
Originally Posted by mp3dom View Post
With bluray encoding, the max value of rc-lookahead is one second (24)...
... rc-lookahead is not necessarily capped to keyint, the mb-tree lookahead is...

vbv-lookahead can actually be even higher than keyint depending on other settings

Last edited by StvG; 23rd April 2019 at 00:28.
StvG is offline   Reply With Quote
Old 23rd April 2019, 09:15   #50  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
It will be capped because vbv_buffer_size=30000, maxrate=40000 => bufsize < 1 => X264_MAX( h->param.i_keyint_max, bufsize*fps ) = keyint_max (in our case of Blu-Ray encode where fps=24 and keyint_max=24) => X264_MIN( h->param.rc.i_lookahead,keyint_max) = keyint_max = 24.
Unless... I'm assuming wrong thinking vbv_buffer_size is also the value of vbv_max_rate...
What happens if we don't redure rc-lookahead and/or vbv-lookahead ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 23rd April 2019, 11:01   #51  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
Every rc-lookahead value over 24 is automatically adjusted to 24 by x264. Tried 60 and got 24.
Also noticed that IDR frames are almost not used by x264. Yes, first frame of a stream is an IDR frame, but almost all the other ones are canonical I frames (not IDR), even at complete scene change. This according to both DGDecNV and other tools as well. All the other bluray encoders I saw so far, do exactly the opposite (anyway this clearly is not the reason of the problem).
mp3dom is offline   Reply With Quote
Old 23rd April 2019, 14:44   #52  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
My "question" was not clear : Is there someone who knows what may happen if the code is changed to prevent rc-lookahead to be reduced ?

Are not canonical I frame a little "less bitrate" consuming (so, in a way, "better" because preserving bitrate bandwith) than IDR frames ? And finaly IDR frames are realy truly needed for places you want to create chapters point.
In that case, the x264 way would be more bitrate efficiant. IDR frames would be created only on the specific point asked in the "qpfile".
Just my two cents thoughts, but maybe i'm wrong.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 23rd April 2019, 20:28   #53  |  Link
infoeater
Registered User
 
Join Date: May 2007
Posts: 53
Quote:
Originally Posted by mp3dom View Post
Yes, first frame of a stream is an IDR frame, but almost all the other ones are canonical I frames (not IDR), even at complete scene change. This according to both DGDecNV and other tools as well. All the other bluray encoders I saw so far, do exactly the opposite (anyway this clearly is not the reason of the problem).
Looks like bug which was fixed once. From thread https://forum.doom9.org/showthread.php?t=164083:
Quote:
Originally Posted by Lorax2161 View Post
To follow up on poisondeathray's reference to the commit, this was in the x264 Development Newsletter: Vol 27:
Quote:
Blu-ray compliance fix: Bluray doesn't allow referencing across I-frames, that is, it treats I-frames like IDR -- so just don't force --keyint-min 1.

I doubt this caused any issues with real playback hardware, but the spec mandates it for some insane reason. Thanks to Pegasys for the bug report.
Maybe adding option --keyint-min 1 could help?
infoeater is offline   Reply With Quote
Old 23rd April 2019, 21:22   #54  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by jpsdr View Post
And finaly IDR frames are realy truly needed for places you want to create chapters point.
Chapters (seeking) don't need IDR.
sneaker_ger is offline   Reply With Quote
Old 23rd April 2019, 21:24   #55  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by infoeater View Post
Maybe adding option --keyint-min 1 could help?
--bluray-compat enforces --keyint-min 1.
sneaker_ger is offline   Reply With Quote
Old 24th April 2019, 09:53   #56  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
The thing i know for sure, it's that if i don't specify in the qpfile that i want "I" frames where i intend to put chapters, Scenarist will complain that chapters are not set on IDR frames. It will still allow me to author the Blu-Ray, but it will throw a warning about a no compliancy.
Well, but this is begining to be off-topic...
__________________
My github.
jpsdr is offline   Reply With Quote
Old 8th May 2019, 17:05   #57  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Stupid idea...
Quote:
Originally Posted by mp3dom View Post
...
In that 1 second/gop quality drop, the qp values are very high (in the 24-25 range) whereas otherwise it would have been 10-12, so compression artifacts are very visible.
...
Can't just something like --qpmax 15 (i set 15, but it was just for exemple, don't know what realy would be the proper value), or --qpmax 15:1000:1000 to just limit qp on I frames if better, solve this issue ? Or are values of 24-25 "legitimates" qp values for others scenes somewhere else ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 9th May 2019, 11:16   #58  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
The command is mostly ignored, doesn't abide with the specified zone setting and there's still the quality drop (tried with qp=8, just for reference)

Last edited by mp3dom; 9th May 2019 at 11:19.
mp3dom is offline   Reply With Quote
Old 23rd May 2019, 09:22   #59  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
@mp3dom
Have you tried to post here ?
Never tried, don't know if it's possible, but maybe worth to try...

Edit : I've opened an issue on their repository.
Maybe you can add a comment saying that you've allready provided data to some devs.
__________________
My github.

Last edited by jpsdr; 23rd May 2019 at 09:47.
jpsdr is offline   Reply With Quote
Old 23rd May 2019, 20:56   #60  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
This is where Cinemacraft outperforms x264- quality balance between different frame types.
kolak is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.