Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd July 2010, 19:03   #1  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
How to deal with this severe color banding?

I wonder if some of the more experienced folks here could give me some settings advice on how to get rid of some severe color banding on THIS small 19MB Blade Runner clip? I use 'MCTemporalDenoise(settings="medium")' on it as only filter. And encode as follows:

cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=tesa / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=6 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / vbv_maxrate=60000 / vbv_bufsize=70000 / crf_max=0.0 / ip_ratio=1.40 / aq=2:1.00 / nal_hrd=non

The quality of the resultant file, overall, though very small, is surprisingly good; but in certain scenes things have really GONE BAD.

I've been trying to get rid of the severe color banding; but, alas, to no avail. :( I tried adding things like '--deadzone-inter 0 --deadzone-intra' and/or '--no-dct-decimate', etc. But the output is still pretty bad. I can achieve a somewhat bearable look by adding '--qpmin 0' and lowering CRF all the way to 5. But that makes the output file nearly twice the size of the original. Needless to say, that's not ideal.

So, I was hoping someone can give me some better advice than the stuff I could come up with myself (which was, pretty much, not much more than rasising bitrate insanely).

Thanks.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 3rd July 2010, 19:24   #2  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,576
Don't know what you expecting to see but it looks great. If you want it to be visually identical to the source set lower crf so resulting bitrate will be 2-3 times higher.
However current crf behavior also seems strange to me. maybe It was rebalanced and I didn't noticed... Оо
Some time ago with higher crf values bitrates was higher and so result was closer to the source. There was no need to touch qpmin. Now we need to set qpmin and crf lower...

Last edited by Keiyakusha; 3rd July 2010 at 19:27.
Keiyakusha is offline   Reply With Quote
Old 3rd July 2010, 19:44   #3  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
Quote:
Originally Posted by Keiyakusha View Post
Don't know what you expecting to see but it looks great.
Well, it looks okay when seen on my PC with, say, GOM Player. :) On my Sony 40" Bravia TV, however, the banding (to the left of Rachel's face) becomes painfully obvious. As in being unwatchable.

Quote:
If you want it to be visually identical to the source set lower crf so resulting bitrate will be 2-3 times higher.
In this case, 'closer the original' has no real meaning, as I MCTemporalDenoise the source first before feeding it to x264.

To be fair, the original, denoised input actually already has some color banding (viewed in GOM Player with just the output of the AVS script, sans x264); x264 just aggrevates the issue. So the object is to tell x264 to not do that.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 3rd July 2010, 19:52   #4  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,576
asarian
Well, I mean closer to the input.
As for your TV, this is expected. Sometimes you even need to add dither with strong noise/grain to make it look good on TV :P
EDIT: and yes, in this case any kind or denoising is not your friend.

Last edited by Keiyakusha; 3rd July 2010 at 19:57.
Keiyakusha is offline   Reply With Quote
Old 3rd July 2010, 21:29   #5  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
Quote:
Originally Posted by Keiyakusha View Post
However current crf behavior also seems strange to me. maybe It was rebalanced and I didn't noticed... ??
Some time ago with higher crf values bitrates was higher and so result was closer to the source. There was no need to touch qpmin. Now we need to set qpmin and crf lower...
I've pretty much given up on understanding the intricacies on the new CRF meanings. :) For this test, for example, I lowered CRF to as low as 1 (from 14,9,7,5,3); and in the various stages the output would sometimes be smaller even on a lower CRF than its predecessor; go figure. :) Indeed, only also lowering qcomp seems to really make a significant difference.

For the current encode, I'm looking at an average bitrate of ~43 Mbp/s now, with a peak at ~80 Mbp/s; all of which I can still stream easily, but is, objectively, too high a bitrate for this source. I'm thinking it would be uebercool if you could set like 'zones' in x264: areas (of known, problematic frame-ranges or some such) for which you want special override bitrate settings. Who knows? Maybe in a distant future. :)
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 3rd July 2010, 21:34   #6  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
Quote:
Originally Posted by asarian View Post
Well, it looks okay when seen on my PC with, say, GOM Player. On my Sony 40" Bravia TV, however, the banding (to the left of Rachel's face) becomes painfully obvious. As in being unwatchable.
Most probably, the Bravia TV is more well calibrated than your PC monitor. To be precise, the banding is on the left (as you said) but even on the right of Rachel's face. It's less visible because it has some subtle grain but it's present. If you don't want to raise bitrate (or lower crf) you can:
- try to increase the AQ values (1.2~1.5)
- Apply gradfun2db prior to encoding as this add a bit of subtle dithering
- Add a bit of noise with Blockbuster. You can add noise only on uniform areas avoiding to add noise to the whole picture.
mp3dom is offline   Reply With Quote
Old 3rd July 2010, 21:37   #7  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 602
Quote:
Originally Posted by mp3dom View Post
Most probably, the Bravia TV is more well calibrated than your PC monitor.
I highly doubt that. PC monitors don't do half of the goofy bit-shifting nonsense that TVs are set up to do usually. It's almost certainly the TV's fault - what model of BRAVIA is it? I have experience with some of those.

What are the TV settings? Make sure any "Dynamic Contrast" modes are shut off, for starters.
Lyris is offline   Reply With Quote
Old 3rd July 2010, 21:41   #8  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
Quote:
Originally Posted by asarian View Post
I'm thinking it would be uebercool if you could set like 'zones' in x264: areas (of known, problematic frame-ranges or some such) for which you want special override bitrate settings. Who knows? Maybe in a distant future.
welcome to 5 years ago:
see the documentation on --zones
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 3rd July 2010, 21:43   #9  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
Quote:
Originally Posted by mp3dom View Post
Most probably, the Bravia TV is more well calibrated than your PC monitor. To be precise, the banding is on the left (as you said) but even on the right of Rachel's face. It's less visible because it has some subtle grain but it's present. If you don't want to raise bitrate (or lower crf) you can:
- try to increase the AQ values (1.2~1.5)
- Apply gradfun2db prior to encoding as this add a bit of subtle dithering
- Add a bit of noise with Blockbuster. You can add noise only on uniform areas avoiding to add noise to the whole picture.
As usual, you came up with some very useful tips! :) Thank you very much!

Man, there's a lot to learn about this stuff. It's only recently that I actually started paying attention to per-job settings. Once you do that, you'll learn soon enough that there really is no 'one-for-all' setings profile.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 3rd July 2010, 21:45   #10  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 1,135
Quote:
Originally Posted by Lyris View Post
I highly doubt that. PC monitors don't do half of the goofy bit-shifting nonsense that TVs are set up to do usually. It's almost certainly the TV's fault - what model of BRAVIA is it? I have experience with some of those.
I don't know which monitor asarian have, nor the quality of the Bravia TV in general, but if it's a TN LCD monitor there's no surprise. TN are 6 bpc monitors and to achieve 16.7M colors they use dithering. Also they suffer from color shifts, reduced viewing angle etc etc.
Anyway I've watched the result posted by asarian on an Eizo ColorEdge hardware-calibrated monitor and the encoded file indeed suffers from color banding on the left and on the right of the Rachel's face as I stated above.

zones are good feature as it allows to change settings in particular areas of the video, but this would require twice the time as you need to do 1st CRF pass, watch the video and then redo a full 2nd CRF pass

Last edited by mp3dom; 3rd July 2010 at 21:50.
mp3dom is offline   Reply With Quote
Old 3rd July 2010, 22:10   #11  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
Quote:
Originally Posted by mp3dom View Post
I don't know which monitor asarian have, nor the quality of the Bravia TV in general, but if it's a TN LCD monitor there's no surprise. TN are 6 bpc monitors and to achieve 16.7M colors they use dithering. Also they suffer from color shifts, reduced viewing angle etc etc.
Anyway I've watched the result posted by asarian on an Eizo ColorEdge hardware-calibrated monitor and the encoded file indeed suffers from color banding on the left and on the right of the Rachel's face as I stated above.
I quickly had to look up the model nr. again, but it's actually a Sony Bravia KDL-40W4000 LCD TV.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 3rd July 2010, 22:44   #12  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
That's not the TV's fault. It's simply the very old story of 8-bit colorspace limitations in general, and YV12 in particular.

Listen to those that advised to not denoise, or to add dithering after the denoising. Dithering works very good during playback ("Deband" in ffdshow). When using dithering before encoding, you need rather slow settings and/or high bitrate in order to keep the dithering in the compressed result.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 3rd July 2010, 22:45   #13  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
I can only advise to not denoise at all. Let x264's PsyRDO do it's magic.
Atak_Snajpera is offline   Reply With Quote
Reply

Tags
bitrate, color banding, settings, x264

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:54.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.