AutoAdjust [v2.60 - Update 2015/11/15]

[LaTo]

  +---------------------------------------------------------------------------------+
  |                                                                                 |
  |                    Automatic adjustement plugin: AutoAdjust()                   |
  |                                                                                 |
  |                               v2.60 by "LaTo INV."                              |
  |                                                                                 |
  |                                 15 November 2015                                |
  |                                                                                 |
  +---------------------------------------------------------------------------------+



      I.    DESCRIPTION
      II.   DISCLAIMER
      III.  INSTALLATION
      IV.   REQUIREMENTS
      V.    MULTITHREADING
      VI.   USAGE
      VII.  GENERAL SETTINGS
      VIII. AUTOGAIN SETTINGS
      IX.   AUTOBALANCE SETTINGS
      X.    CHANGELOG



 I. DESCRIPTION :
 ================

  AutoAdjust is an automatic adjustement filter. It calculates statistics of clip,
  stabilizes them temporally and uses them to adjust luminance gain & color balance.

  AutoAdjust have a smoothing & dithering algorithm to avoid banding issue.
  Calculations are made in 32bits float to avoid rounding errors and
  are internally multithreaded and SSE2/AVX2 optimized.

  For manual adjustments, see all plugins in "SmoothAdjust" package.



 II. DISCLAIMER :
 ================

  This program is free for non-commercial use only and must be distributed
  in its original form without any modification, including this documentation file.
  This program is provided as-is in the hope it will be useful but 
  without any warranty, including any hypothetical damages or issues.

  The official home page is "http://forum.doom9.org/showthread.php?t=167573",
  it always contains the latest version of this program.
  This page acts as the official support center, also the author is reachable
  through his member page: "http://forum.doom9.org/member.php?u=131032".

  This plugin has required a lot of work and much coding hours, so if you like
  it and feel the need to reward or inspire the author then please consider donating.
  Donations are a fast, easy and secure operation through Paypal: simply follow the
  link into the "Donate.url" shortcut (you do not need a Paypal account to donate).
  All donations will be very useful and I thank in advance all the generous people,
  also if you are on the doom9's forum please specify your nickname when donating.



 III. INSTALLATION :
 ===================

  There are different versions of AutoAdjust to suit all installations of Avisynth.
   - x86 builds are for Avisynth 32bits
   - x64 builds are for Avisynth 64bits



 IV. REQUIREMENTS :
 ==================

  - Avisynth v2.6.x or Avisynth Plus
  - YUV Planar (Y8, YV411, YV12, YV16, YV24)
  - 8bits or 16bits (MSB/LSB)



 V. MULTITHREADING :
 ===================
  AutoAdjust is internally multithreaded with official Avisynth version. 
  If AvisynthPlus or AvisynthMT is detected, internal multithreading is automatically disabled
  even if external multithreading isn't used (this can be reactivated in parameters).

  AutoAdjust is designed to work with external multithreading, so fastest MT mode can be used.
  With AvisynthPlus, this plugin is a MT_NICE_FILTER (automatic register, no need to call SetFilterMtMode).
  With AvisynthMT, SetMtMode(1) must be called before AutoAdjust.



 VI. USAGE :
 ===========

    AutoAdjust ( [clip], external_clip,
                 auto_gain,
                 dark_limit, bright_limit, gamma_limit, 
                 dark_exclude, bright_exclude, 
                 gain_mode, chroma_process, avg_safety,
                 input_tv, output_tv,
                 auto_balance,
                 chroma_limit, balance_str,
                 scd_threshold, temporal_radius, change_status,
                 use_interp, use_dither,
                 high_quality, high_bitdepth, 
                 threads_count, asm_opt,
                 debug_view )

Last version (2015/11/15): AutoAdjust v2.60

Old version (for Avs2.5 or WinXP or Wine): AutoAdjust v1.02


If you really like this filter, please consider donating...
It will help me a lot to continue development and support.
Thanks for all! (link in the .7z package)


v2.60:
updated for Avs v2.6.0 final
fixed AVX2 crash with Win7 without SP1

v2.50:
fixed crash with external_clip
fixed chroma processing bug
removed temporal_radius=-1
reintroduced internal caching

v2.40:
optimized frame access & frame cache
fixed potential freeze with very slow source filter

v2.20:
optimized memory usage with very long clip
fixed crash with ffdshow_source() and external MT

v2.00:
dropped Avs v2.5.x support
added AVX2 code (40% faster)
modified plugin to be thread-safe (MT_NICE_FILTER)
removed internal caching
added frame cache hints
improved debug_view

v1.90:
changed name of plugin & parameters
added color balance
added C version
fixed regression from v1.75 (bug at scenechange)

v1.75:
added avg_safety parameter
fixed a minor bug in temporal averaging
tweaked multi-threading code

v1.60:
split 'tv_range' into 'input_tv' & 'output_tv'
fixed possible freeze in MT code
added explicit error messages

v1.45:
fixed bugs inside multi-threading code
removed user32 dependency
added AVX2 detection (for future opt)

v1.40:
used new C++11 features
rewritten multi-threading (native)
speed optimization (10% faster)
exception-safe code
added a high quality interpolation mode
improved core algorithm
fixed various little things

v1.02:
fixed minor things & bugs
dropped ICL builds support

v1.00:
first stable release


_

[LaTo]

Screenshot of AutoAdjust(debug_view=true):


* Click the image to enlarge *

[LaTo]

Early alpha development discussion is available on the SmoothAdjust's thread, development and support will now continue here...

[LaTo]

/* Post from 01 april 2013 on SmoothAdjust's thread */


New AutoGain update: AutoGain-beta1.

Big update, any inputs is appreciated

[LaTo]

Update: AutoGain-beta2

More robust scenechange detector, less fail on fade in/out

[LaTo]

New update: AutoGain-beta3.

The plugin seems pretty mature now... One thing to add to be complete is a debug view, but later

[asarian]

Originally Posted by LaTo

Screenshot of AutoAdjust(debug_view=true):


* Click the image to enlarge *

My, AutoAdjust is real sweet! Thx!

[asarian]

"With AvisynthMT, SetMtMode(1) must be called before AutoAdjust."

Why SetMtMode(1) per se? SetMtMode(3), which I usually use, no good?!

[Groucho2004]

Originally Posted by asarian

"With AvisynthMT, SetMtMode(1) must be called before AutoAdjust."

Why SetMtMode(1) per se? SetMtMode(3), which I usually use, no good?!

I explained this to you before - mode 3 is for source filters. I suggest you read SEt's comments in the AVSMT thread.

[asarian]

Originally Posted by Groucho2004

I explained this to you before - mode 3 is for source filters. I suggest you read SEt's comments in the AVSMT thread.

And as I explained in that thread, MTMode(1) simply isn't stable enough for the things I do. Nor MTMode(2), for that matter. I may not understand all the math behind it, but, after years of experimenting, the following incantation simply works for me:

SetMemoryMax(768)

SetMTMode(5,4)

FFVideoSource("f:\jobs\whatever.mkv")

SetMTMode(3)

the rest of the filters

MTMode(3) may be slightly slower, but anything below it, and things simply start crashing.

Now, there may be one or two filters out there that could, stand-alone, run stable with MTMode(1), but in the togetherness of what I usually do (lots of temporal denoising), MTMode(3) is the only mode for me that is guaranteed not to crash my job.

And I've read as much about MT AviSynth as I could find, but still found no reason AutoAdjust *must* work within MTMode(1) per se. Hence my question.

[LaTo]

Originally Posted by asarian

"With AvisynthMT, SetMtMode(1) must be called before AutoAdjust."

Why SetMtMode(1) per se? SetMtMode(3), which I usually use, no good?!

AutoAdjust is a heavy temporal filter (it requests a lot of frames, temporal radius is 20 by default).
MtMode=1 was recommended (but not mandatory) in v2.00 because of this.

Now it is less significant, but if you do not use MtMode=1 AutoAdjust will be slower and will use more memory because:
- identical calculations will be performed in each instance
- identical data will be stored in each instance
- avisynth's cache will be more stressed
- frames will not be requested in linear order

MtMode=1 is not a mandatory, but it works and is better... So why don't use it?

[asarian]

Originally Posted by LaTo

AutoAdjust is a heavy temporal filter (it requests a lot of frames, temporal radius is 20 by default).
MtMode=1 was recommended (but not mandatory) in v2.00 because of this.

Now it is less significant, but if you do not use MtMode=1 AutoAdjust will be slower and will use more memory because:
- identical calculations will be performed in each instance
- identical data will be stored in each instance
- avisynth's cache will be more stressed
- frames will not be requested in linear order

MtMode=1 is not a mandatory, but it works and is better... So why don't use it?

Thx for the explanation! As soon as this process is done (just another 6 hours), I will try and rerun it with Mode 1. I will let you know what it does.

Originally Posted by LaTo

SetMemoryMax(1024)
SetMTMode(5,4)
ffdshow_source()
SetMTMode(1,4)
AutoAdjust(auto_gain=true, auto_balance=true, balance_str=1.0)
GetMTMode(false) > 0 ? distributor() : last

Not sure the '4' in SetMTMode(1,4) has any effect there, really; but it looks like you're saying you can just use Mode 1 for your filter alone, which is good.

[Reel.Deel]

Hi LaTo, I was wondering if you changed the defaults after Beta5 or something internally? By looking at the readme's I see that the defaults are still the same. But, beta5 compared to 6 and 7 behave differently.
Here's the comparison, I used all defaults and for beta 6 and 7 I used chroma_process=0. I like beta5 better.

Also, do you think it would be worthwhile to have U=xx, V=xx parameters instead of chroma_process=xx? I've come across footage where one of the chroma channels seems to be more "saturated" than the other.
Maybe independent parameters can aid those types of situations.

One last thing, AutoGain works fine with 8bit YV16/24 and with 16bit YV12 but with 16bit YV16/24 I get some weird artifacts which seems to only affect the MSB/top half. Here's an example after dithering to 8bit.


Kudos on the debug view. It looks very good!
Thanks again for all your awesome work!

[LaTo]

Originally Posted by Reel.Deel

Hi LaTo, I was wondering if you changed the defaults after Beta5 or something internally? By looking at the readme's I see that the defaults are still the same. But, beta5 compared to 6 and 7 behave differently.
Here's the comparison, I used all defaults and for beta 6 and 7 I used chroma_process=0. I like beta5 better.

Also, do you think it would be worthwhile to have U=xx, V=xx parameters instead of chroma_process=xx? I've come across footage where one of the chroma channels seems to be more "saturated" than the other.
Maybe independent parameters can aid those types of situations.

One last thing, AutoGain works fine with 8bit YV16/24 and with 16bit YV12 but with 16bit YV16/24 I get some weird artifacts which seems to only affect the MSB/top half. Here's an example after dithering to 8bit.


Kudos on the debug view. It looks very good!
Thanks again for all your awesome work!

I made a mistake with dark_exclude/bright_exclude, default should be "0.5" and not "0.05"... I will change it in next version, for now use "dark_exclude=0.5,bright_exclude=0.5".

And for 16bits / YV24, this seems a chroma bug (maybe I forgot a shift somewhere... )

[steptoe]

If I click that link, or any other to try and download autogain beta's I just get the '404 error' and had done in the past


Ignore that, must have been in the process of updating it while I was typing this. Just tried searching for another link came back to this one, beta 8 was available like 2 minutes later

Thanks

[LaTo]

Originally Posted by steptoe

If I click that link, or any other to try and download autogain beta's I just get the '404 error'

Yeah, I was updating

[ganymede]

Your plugins makes wonders on dark scenes with footage from my Canon DSLR. Thank you very much LaTo !

[LaTo]

Update: AutoGain-beta8

[Motenai Yoda]

just a question, how dark_exclude/bright_exclude works?
by percentile of excluded pixel or percentile of dynamic range?
also can be added a parameter that control how heavy and light correctons weighs?
ie if i want that light corrections remains unthouched, and lighting heavy corrections, or viceversa, lighting light corrections and leave heavy corrections unthouched.
I think to an "S" function that, for positive values relieve light corrections and, for negative values relieve heavy corrections (or increase lights ones).

[LaTo]

Originally Posted by Motenai Yoda

just a question, how dark_exclude/bright_exclude works?
by percentile of excluded pixel or percentile of dynamic range?

By percentile of excluded pixel

Originally Posted by Motenai Yoda

also can be added a parameter that control how heavy and light correctons weighs?
ie if i want that light corrections remains unthouched, and lighting heavy corrections, or viceversa, lighting light corrections and leave heavy corrections unthouched.
I think to an "S" function that, for positive values relieve light corrections and, for negative values relieve heavy corrections (or increase lights ones).

You can do that in an avisynth script with lut/mask/merge, but for general purpose I don't see the point to remove small corrections and leave big corrections untouched.
A sort of color vibrance's effect but on luminance?