Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th October 2016, 21:24   #61  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
OK, I need to get off the sidelines and get into this game. Harry Potter magic doesn't begin to describe what I'm seeing. Amazing stuff, StainlessS!

I'm working on a complete re-write of the software for my shutterless projector transfer system, and I need to integrate this into my new workflow.

Last edited by johnmeyer; 7th October 2016 at 21:25. Reason: added second sentence.
johnmeyer is offline   Reply With Quote
Old 7th October 2016, 23:03   #62  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
All, but all magic is down to Videofred, i' m just a grunt.

Videofred ain't no muggle. [EDIT: Yoda had some say so too, yo dude, Yoda for king, or maybe Pope or something].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 8th October 2016 at 00:27.
StainlessS is offline   Reply With Quote
Old 8th October 2016, 04:37   #63  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
I know what VideoFred can do, and it is beyond remarkable. However, I've followed this development, and you came up with the code that makes everything work.

I hope to get to this next week, and I'll report back on what I'm able to do with it.
johnmeyer is offline   Reply With Quote
Old 8th October 2016, 12:52   #64  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Post #1 of 2
GamMac v1.06 Beta6, New Version:- LINK REMOVED
(~998KB, Included Puppy.png, LennaRed.png, and GreenChurch.png)

Updates mainly cosmetic.
Increased number of Verbosity levels to 5 (default=2, 5=Show version info + ALL metrics, 0=show metrics Flags line only).
A few mods to docs.
Added Subs to 4 window output, optional coords shown (included avs script).
EDIT: The DC clip (detect clip) coords now changed from all white dotted lines to alternate white and black dots.

Docs
Code:
GamMac(), [Gamma Machine] An extraordinary Idea by VideoFred (the gent from Gent). Coded by StainlessS.

    Home Thread:- http://forum.doom9.org/showthread.php?p=1774281#post1774281
    Idea:-        http://forum.doom9.org/showthread.php?t=173683

RGB Only.

    Useful to correct color cast on old 8mm films.
    Alters channel pixel average to match LockChan using Gamma correction. (By default alters Red and Blue channels to match Green).
    Additional tweaking via RedMul, GrnMul and BluMul multipliers.

    What it does(roughly):-
        Firstly, RAW input channel Ranges are measured for all three channels (see RngLim).
        If ALL THREE raw input ranges are less than RngLim (single color frame), then for current frame,
          there is no scaling nor gamma estimation, and only linear rendering is done to output range omin -> omax.
          [Channels multipliers RedMul, GrnMul and BluMul still applied though.]
        OtherWise,
            If ANY ONE channel input range is less than RngLim and Scale==2, then Scale is (for current frame) knocked down to Scale=1.
            Get Channel averages, minimums, and maximums (using loTh for minimums and hiTh for maximums).
            if(Scale==0 OR (loTh<0.0 AND hiTh<0.0)) then
                No rescaling.
            if(Scale == 1 AND (loTh>=0.0 OR hiTh>=0.0)) then
                rescales averages using combined dynamic range of r,g,b ie 0.0 -> (max(redMax,grnMax,bluMax) - min(redMin,grnMin,bluMin)).
            else if(Scale == 2 AND (loTh>=0.0 OR hiTh>=0.0)) then
                rescales averages using separate dynamic ranges ie 0.0->(redMax-redMin), 0.0->(grnMax-grnMin), 0.0->(bluMax-bluMin).
            For each channel, estimate gamma function that will remap (scaled channel average * channel multiplier) to match a particular
            LockVal (chosen via LockChan) when rendered to the chosen output range specified by omin and omax.
            Then renders frame using the output averages from estimated gamma with output channel minimums at omin, and maximums at omax.

GamMac(Clip c,int "LockChan"=1,int "Scale"=2,
      \ Float "RedMul"=1.0,Float "GrnMul"=1.0, Float "BluMul"=1.0,
      \ Float "Th"= 0.04,Float "loTh"=Th,Float "hiTh"=Th,
      \ Float "LockVal"=128.0,int "RngLim"=11,Float "GamMax"=10.0,
      \ Clip "dc",
      \ int "x"=20,int "y"=20,int "w"=-20,int "h"=-20,
      \ int "omin"=0, int "omax"=255,
      \ Bool "Show"=True,int "Verbosity"=2,Bool "Coords"=false
      \ )

    LockChan Default 1(Grn).    Channel for lock to Average. [range -3 -> 2]
                                 0 ] LockVal = Scaled(RedAve)
                                 1 ] LockVal = Scaled(GrnAve)
                                 2 ] LockVal = Scaled(BluAve)
                                 -1] LockVal = Use explicit LockVal arg (see below).
                                 -2] LockVal = (Scaled(RedAve)+Scaled(GrnAve)+Scaled(BluAve))/3.0. [Mean]
                                 -3] LockVal = Median(Scaled(RedAve),Scaled(GrnAve),Scaled(BluAve))
                                Where Scaled(Channel Average) depends upon RngLim, Scale, and loTh, and hiTh.

    Scale, default 2           Range 0 -> 2.
                                There is NO SCALING DONE if ALL THREE channels range is less than RngLim, see RngLim, linear render only.
                                If ANY ONE channel input range is less than RngLim and Scale==2, then Scale is (for current frame) knocked down to Scale=1.

                                    where some described for Red Channel only:-
                                         redMin = RedChanMin(ignorePerc=loTh)        # Pixel minimum for red channel, ignoring up to loTh%, ie noise.
                                         redMax = RedChanMax(ignorePerc=hiTh)        # Pixel maximum for red channel, ignoring up to hiTh%, ie noise.
                                         redAve = RedChanAve()                       # Pixel average for red Channel.
                                         redRng = redMax - redMin
                                         inMin  = min(redMin,grnMin,bluMin)          # Min of minimums
                                         inMax  = max(redMax,grnMax,bluMax)          # Max of maximums

                                     0 (Scale==0 || (loTh==-1.0 && hiTh==-1.0))      # No Effect on scale.
                                         scaledAveR = redAve
                                         scaledAveG = grnAve
                                         scaledAveB = bluAve
                                     1) Scales input channel average maximum dynamic range of R,G,B, to 0.0->(ChanAve-inMin)*255.0/(inMax-inMin)
                                         scaler = 255.0 / (inMax - inMin)
                                         scaledAveR = min(max((RedAve - inMin) * scaler,0.0),255.0)
                                         scaledAveG = min(max((GrnAve - inMin) * scaler,0.0),255.0)
                                         scaledAveB = min(max((BluAve - inMin) * scaler,0.0),255.0)
                                     2) Scales input channel average dynamic range of R & G & B, Individually, to 0.0->(ChanAve-Chan_min)*255.0/(ChanMax-ChanMin)
                                         scalerR = 255.0 / (redMax-redMin)
                                         scalerG = 255.0 / (grnMax-grnMin)
                                         scalerB = 255.0 / (bluMax-bluMin)
                                         scaledAveR = min(max((redAve - redMin) * scalerR,0.0),255.0)
                                         scaledAveG = min(max((grnAve - grnMin) * scalerG,0.0),255.0)
                                         scaledAveB = min(max((bluAve - bluMid) * scalerB,0.0),255.0)

    RedMul, default 1.0         Red channel multiplier adjustment.   [0.1 <= RedMul <= 10.0]
    GrnMul, default 1.0         Green channel multiplier adjustment. [0.1 <= GrnMul <= 10.0]
    BluMul, default 1.0         Blue channel multiplier adjustment.  [0.1 <= BluMul <= 10.0]
                                Scaled averages are multiplied by their multiplier then given as args to the gamma estimator.
                                Allow tweaking of R,G,B channels.
                                   Above Multipliers only shown in metrics when at least one is != 1.0 (Always shown when Verbosity=3=FULL).

    Th, Default 0.04           Sets Default for loTh and hiTh. Suggest Default, 0.04(percent).  [-1.0(OFF) , or 0.0 -> 1.0]
    loTh, Default Th           As for Ignore_low in AutoLevels, or Threshold in YPlaneMin.  [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding minimum R, G or B channel values.
                               -1.0 is OFF, input channel minimum is set to 0 as for levels(0,gamma,input_max, ... ).
                               If loTh >=0.0, then will scan frame looking for lowest pixel value whose cumulative sum
                               [including all pixels counts of lower value pixels] is greater than loTh%.
                               loTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).
    hiTh, Default Th           As for Ignore_high in AutoLevels, or Threshold in YPlaneMax. [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding maximum R, G or B channel values.
                               -1.0 is OFF, input channel maximum set to 255, as in levels(input_min,gamma,255, ... ).
                               If hiTh >=0.0, then will scan frame looking for highest pixel value whose cumulative sum
                               [including all pixels counts of higher value pixels] is greater than hiTh%.
                               hiTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).

    LockVal, default 128.0     Only used if LockChan = -1. [0.0 < LockVal < 255.0] (set via LockChan if LockChan != -1)
                               There is no restricted range on this (other than 0.0 < LockVal < 255.0), so if you set a stupid value,
                               you will likely get stupid results.

    RngLim, default 11         [1 <= RngLim <= 32]
                               If ALL THREE RAW input channel ranges ie (ChannelMax(max(hiTh,0.0))-ChannelMin(max(loTh,0.0))) are less than RngLim then
                               all scaling is disabled, and remapping is linear without gamma estimation, to range omin -> omax,
                               ie avoid remapping of Black, White frames, or single color frames.

    GamMax, default 10.0       Upper value for guess gamma [2.0 <= GamMax <= 10.0]
                                   Starting guess upper range and limit for gamma estimator (probably best left alone).
                                   The lower guess range and limit will be set to 1.0 / GamMax, by default 0.1.

    dc, default clip c.        Detection clip, Must be same ColorSpace and FrameCount as source clip, no other similarities enforced.
                               (can be different size, denoised etc).

    x,y, Both default 20.      Area of dc Detect clip frame to sample when getting averages and estimating Gamma function, allows to ignore rubbish at frame edges.
    w,h, Both default -20.     Specified as for crop eg x=10,y=20,w=-30,h=-40, as in crop(10,20,-30,-40).

    omin, default 0.           Output limits for all three R, and G, and B channels. [Range 0 -> 16]
    omax, default 255.         [Range 239 -> 255] (extremes 16->239 allow for Studio RGB output).
                               May want to give yourself a little head/foot room by setting eg omin=5, omax=250, so that you leave a little room for
                               further manual color tweaking.

    Show, default true        True, show metrics info on frame.
    Verbosity, default 2       0 = Only upper frame metrics Flags line only
                               1 = Upper frame metrics
                               2 = Upper + important ones. (default)
                               3 = Nearly Full metrics.
                               4 = Full Metrics except version info
                               5 = Full Metrics including version info
                               Upper frame metrics shown as eg:- (when Verbosity=5=FULL)

                                  nnnnn] Flags:- 1SR
                                             R         G         B
                                  RAW:     10,253    10,253    10,253
                                  IN:      10,253    10,253    10,253
                                  IN_AVE:  78.466    88.552    78.767
                                  SCALED:  71.847    82.431    72.162
                                  GAMMA:    1.135     1.000     1.123
                                  OUTAVE:  82.431    82.422    82.451

                               where,
                                  nnnnn, is the frame number.
                                  Flags:- (Specific to current frame, can change frame to frame)
                                      '1' = LockChan, as above, channel '1'[ScaleAveG].
                                            Can be, '0', '1', '2' [ScaleAve Channel number LockChan=-3(median) assigns '0', '1' or '2' as appropriate]
                                                    'A'[LockChan=-2, (ScaleAveR+ScaleAveG+ScaleAveB)/3.0]
                                                    'V'[LockChan=-1, Explicit LockVal]
                                      'S' = Scale, mode signified by color.
                                            Greyed out. Scale = 0(No Effect). May be Greyed out if all channels Min/Max are 0,255.
                                            White.  Scale = 1[Scales input channel average maximum dynamic range of R,G,B]
                                            Orange. Scale = 2[Scales input channel average dynamic range of R and G and B, Individually]
                                      'R' = Limited by RngLim, mode signfied by color.
                                            Greyed out. Not Range Limited.
                                            Red, at least 1 channel has remapping disabled.
                                  RAW:    Shows RAW comma separated channel minimum and maximum, eg ChannelMin(max(loTh,0.0)) and (ChannelMax(max(hiTh,0.0)),
                                          only shown if Verbosity>=3 or, if any RAW input range is less than RngLim AND any of the RAW inputs are different
                                          to the equivalent standard input.
                                  IN:     Shows comma separated channel minimum and maximum (dependent upon Scale, loTh, hiTh).
                                  IN_AVE: Input channel averages.
                                  SCALED: Scaled input averages, (dependent upon Scale, loTh, hiTh, channel minimums and maximums).
                                  GAMMA:  Estimated gamma to achieve lockval for channel. (dependent upon pretty much everything).
                                  OUTAVE: Output channel average ie rendered result.

                               ALL metrics derived from the detection clip dc (Including OutAve's).

    Coords, default False.  If True, then shows DC clip with dotted lines showing the x,y,w,h coords plotted on frame. (All other functionality disabled).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th June 2018 at 14:42.
StainlessS is offline   Reply With Quote
Old 8th October 2016, 12:53   #65  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Post #2 of 2

Included script showing Coords

Code:
Imagesource("GreenChurch.png",end=0)
#Imagesource("Puppy.png",end=0)
#Imagesource("lennaRed.png",end=0)

ConvertToRGB24.KillAudio
#Spline36Resize(512,384)
O=Last
DC=Last

#DC=DC.Blur(1.0)           # Detection Clip (uses source clip if dc not supplied, Denoised or whatever)

LockChan = 1            # Chan for lock to Ave, 0=R, 1=G, 2=B
                        #    -1 = Use LockVal below.
                        #    -2 = LockVal=(RedAve+GrnAve+BluAve)/3.0 for LockVal.
                        #    -3 = LockVal=Median(RedAve,GrnAve,BluAve)
Scale=2
RedMul    = 1.0         # Required Ave multiplier for Red Channel, applied when requesting GuessGamma(reqAve*RedMul), Even applied when LockChan=-1.
GrnMul    = 1.0         #    Same for Green channel. GrnMul even applies when LockChan is Green Channel, etc for chans.
BluMul    = 1.0         #    Same for Blue channel.  Allows tinkering/fine tuning.
Th   	  = 0.04
LockVal   = 128.0       # Only valid if LockChan == -1
RngLim    = 11
GamMax    = 10.0
Show      = True        # Metrics
Verb      = 1           # Verbocity FULL
SHOWCOORDS= True        # Show Original with Coords

x =5                    # Coords (for dc Detection Clip)
y =5
w=-5
h=-5

omin=5                  # Output channels minimum (footroom for manual editing).
omax=250                # Output channels maximum (headroom for manual editing).

#Return GamMac(DC,x=x,y=y,w=w,h=h,Coords=True)    # Show Coords only

Scale=0
A_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
A=GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)


Scale=1 BluMul=1.05
B_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
B=GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)

Scale=2 BluMul=0.95
C_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
C=GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)

COORDS=GamMac(DC,x=x,y=y,w=w,h=h,Coords=True)
TOP=StackHorizontal(SHOWCOORDS?Sub(COORDS,"Coords Original"):Sub(O,"Original"),Sub(A,A_TEXT))
BOT=StackHorizontal(Sub(B,B_TEXT),Sub(C,C_TEXT))
StackVertical(TOP,BOT)
return Last

Function Sub(clip c,string tit) {StackVertical(c.BlankClip(height=20).ScriptClip("""Subtitle(RT_String("%d] %s",current_frame,tit))""",Args="tit",Local=true),c)}
Result


EDIT: The titles added above each frame include frame number.
EDIT: Script requires Grunt() (c) Gavino, for Args arg in Scriptclip [used by Function Sub()].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th June 2018 at 14:43.
StainlessS is offline   Reply With Quote
Old 8th October 2016, 21:33   #66  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
GamMatch() v0.00 NEW PLUGIN [based on GamMac()] :- LINK REMOVED

Code:
GamMatch(), [Gamma Matcher] A daft Idea, based upon an extraordinary Idea by VideoFred (the gent from Gent). (Daft idea and code by StainlessS).

    The Idea behind the daft idea(GamMac):-  http://forum.doom9.org/showthread.php?p=1774281#post1774281
    Original Idea:-                          http://forum.doom9.org/showthread.php?t=173683

RGB Only.

    Useful to correct bad color clip where there is a better color source of perhaps lower rez available.
    Additional tweaking via RedMul, GrnMul and BluMul multipliers.

    What it does(roughly):-
         Alter bad color clip c channel minimums, maximums and averages to match dc (good color clip) by 
         varying Gamma

GamMatch(Clip c,clip dc,int "Scale"=2,
      \ Float "RedMul"=1.0,Float "GrnMul"=1.0, Float "BluMul"=1.0,
      \ Float "Th"= 0.04,Float "loTh"=Th,Float "hiTh"=Th,
      \ int "RngLim"=11,Float "GamMax"=10.0,
      \ int "x"=20,int "y"=20,int "w"=-20,int "h"=-20,
      \ Bool "Show"=True,int "Verbosity"=2,Bool "Coords"=false
      \ )

    c, no default.              Bad Color clip.

    dc, no default.             Detection clip (good color clip). Must be same ColorSpace and FrameCount as source clip, no other similarities enforced.

    Scale, default 2            Range 0 -> 2.

    RedMul, default 1.0         Red channel multiplier adjustment.   [0.1 <= RedMul <= 10.0]
    GrnMul, default 1.0         Green channel multiplier adjustment. [0.1 <= GrnMul <= 10.0]
    BluMul, default 1.0         Blue channel multiplier adjustment.  [0.1 <= BluMul <= 10.0]

    Th, Default 0.04           Sets Default for loTh and hiTh. Suggest Default, 0.04(percent).  [-1.0(OFF) , or 0.0 -> 1.0]
    loTh, Default Th           As for Ignore_low in AutoLevels, or Threshold in YPlaneMin.  [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding minimum R, G or B channel values.
                               -1.0 is OFF, input channel minimum is set to 0 as for levels(0,gamma,input_max, ... ).
                               If loTh >=0.0, then will scan frame looking for lowest pixel value whose cumulative sum
                               [including all pixels counts of lower value pixels] is greater than loTh%.
                               loTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).
    hiTh, Default Th           As for Ignore_high in AutoLevels, or Threshold in YPlaneMax. [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding maximum R, G or B channel values.
                               -1.0 is OFF, input channel maximum set to 255, as in levels(input_min,gamma,255, ... ).
                               If hiTh >=0.0, then will scan frame looking for highest pixel value whose cumulative sum
                               [including all pixels counts of higher value pixels] is greater than hiTh%.
                               hiTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).

    RngLim, default 11         [1 <= RngLim <= 32]
                               If ALL THREE RAW input channel ranges ie (ChannelMax(max(hiTh,0.0))-ChannelMin(max(loTh,0.0))) are less than RngLim then
                               all scaling is disabled, and remapping is linear without gamma estimation, to range omin -> omax,
                               ie avoid remapping of Black, White frames, or single color frames.

    GamMax, default 10.0       Upper value for guess gamma [2.0 <= GamMax <= 10.0]
                                   Starting guess upper range and limit for gamma estimator (probably best left alone).
                                   The lower guess range and limit will be set to 1.0 / GamMax, by default 0.1.


    x,y, Both default 20.      Area of dc Detect clip frame to sample when getting averages and estimating Gamma function, allows to ignore rubbish at frame edges.
    w,h, Both default -20.     Specified as for crop eg x=10,y=20,w=-30,h=-40, as in crop(10,20,-30,-40).

    Show, default true        True, show metrics info on frame.
    Verbosity, default 2       0 = Only upper frame metrics Flags line only
                               1 = Upper frame metrics
                               2 = Upper + important ones. (default)
                               3 = Nearly Full metrics.
                               4 = Full Metrics except version info
                               5 = Full Metrics including version info
                               Upper frame metrics shown as eg:- (when Verbosity=5=FULL)

                                  nnnnn] Flags:- SR
                                             R         G         B
                                  DC:       4,255     3,255     2,255
                                  DC_AVE:  92.123   128.543   132.890
                                  RAW:     10,253    10,253    10,253
                                  IN:      10,253    10,253    10,253
                                  IN_AVE:  78.466    88.552    78.767
                                  SCALED:  71.847    82.431    72.162
                                  GAMMA:    1.135     1.000     1.123
                                  OUTAVE:  82.431    82.422    82.451   <<< OUTAVE will read about same as DC_AVE (made up numbers)

                               where,
                                  nnnnn, is the frame number.
                                  Flags:- (Specific to current frame, can change frame to frame)
                                      'S' = Scale, mode signified by color.
                                            Greyed out. Scale = 0(No Effect). May be Greyed out if all channels Min/Max are 0,255.
                                            White.  Scale = 1[Scales input channel average maximum dynamic range of R,G,B]
                                            Orange. Scale = 2[Scales input channel average dynamic range of R and G and B, Individually]
                                      'R' = Limited by RngLim, mode signfied by color.
                                            Greyed out. Not Range Limited.
                                            Red, at least 1 channel has remapping disabled.
                                  DC:     Shows comma separated channel minimum and maximum of DC clip(dependent upon loTh, hiTh).
                                  DC_AVE: DC Clip channel averages.
                                  RAW:    Shows RAW comma separated channel minimum and maximum, eg ChannelMin(max(loTh,0.0)) and (ChannelMax(max(hiTh,0.0)),
                                          only shown if Verbosity>=3 or, if any RAW input range is less than RngLim AND any of the RAW inputs are different
                                          to the equivalent standard input.
                                  IN:     Shows comma separated channel minimum and maximum (dependent upon Scale, loTh, hiTh).
                                  IN_AVE: Input channel averages.
                                  SCALED: Scaled input averages, (dependent upon Scale, loTh, hiTh, channel minimums and maximums).
                                  GAMMA:  Estimated gamma to achieve lockval for channel. (dependent upon pretty much everything).
                                  OUTAVE: Output channel average ie rendered result.

                               DC and DC_AVE derived from DC (Good color clip), all others from clip c (BAD color clip, Incl OUTAVE).

    Coords, default False.  If True, then shows DC clip with dotted lines showing the x,y,w,h coords plotted on frame. (All other functionality disabled).
Simulated Demo
Code:
# Some good color clip
Avisource("1941 Flint Michigan Parade [Low, 360p].mp4.AVI")
ConvertToRGB24.KillAudio
DC=Last

# Simulate bad color, Input clip
RED=ShowRed(Pixel_Type="Y8").Levels(  0,0.9,255, 8 , 247,coring=false)
GRN=ShowGreen(Pixel_Type="Y8").Levels(0,1.5,255, 12, 223,coring=false)
BLU=ShowBlue(Pixel_Type="Y8").Levels( 0,2.5,255, 18, 239,coring=false)
MergeRGB(RED,GRN,BLU,pixel_Type="RGB24")

O=Last

Scale=2
RedMul   = 1.0          # Required Ave multiplier for Red Channel, applied when requesting GuessGamma(reqAve*RedMul), Even applied when LockChan=-1.
GrnMul   = 1.0          #    Same for Green channel. GrnMul even applies when LockChan is Green Channel, etc for chans.
BluMul   = 1.0          #    Same for Blue channel.  Allows tinkering/fine tuning.
Th   	 = 0.04
RngLim   = 11
GamMax   = 10.0
Show     = True         # Metrics
Verb     = 5            # Verbocity FULL

x =5                   # Coords (for dc Detection Clip)
y =5
w=-5
h=-5

Scale=0
A_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
A=GamMatch(Last,DC,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,RngLim=RngLim,GamMax=GamMax,
    \ x=x,y=y,w=w,h=h,
    \ Show=Show,Verbosity=Verb)


Scale=1
B_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
B=GamMatch(Last,DC,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,RngLim=RngLim,GamMax=GamMax,
    \ x=x,y=y,w=w,h=h,
    \ Show=Show,Verbosity=Verb)

Scale=2
C_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
C=GamMatch(Last,DC,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,RngLim=RngLim,GamMax=GamMax,
    \ x=x,y=y,w=w,h=h,
    \ Show=Show,Verbosity=Verb)

COORDS=GamMatch(Last,DC,x=x,y=y,w=w,h=h,Coords=True)

TOP=StackHorizontal(Sub(O,"Original (Simulated bad clip)"),Sub(COORDS,"Detect with Coords"),Sub(A,A_TEXT))
BOT=StackHorizontal(Sub(DC,"DC Detect (Good color clip)"),Sub(B,B_TEXT),Sub(C,C_TEXT))
StackVertical(TOP,BOT)
return Last

Function Sub(clip c,string tit) {StackVertical(c.BlankClip(height=20).ScriptClip("""Subtitle(RT_String("%d] %s",current_frame,tit))""",Args="tit",Local=true),c)}
result



without metrics



EDIT: IN: Red values a little wonky in metrics.

EDIT: Both clips should be in sync.

EDIT: The Scale=0 result does not alter minimums and maximums to match DC, hence the slightly reddish white lines on road.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th June 2018 at 14:44.
StainlessS is offline   Reply With Quote
Old 18th October 2016, 23:09   #67  |  Link
super8boy
Registered User
 
Join Date: May 2016
Posts: 1
Overscans

That's seriously great work. Well done. Really impressive.
How well does it work on film for which an overscan has been done?

Would the system get confused by extra information on the top and bottom of the frame + left (perforations) and right?

If that was the case, I guess one could only process say only x by y amount of pixels (say 75% of the image) correct?
If not, the only solution would be not to overscan so much, to not overscan at all or to recrop the video to 1440x1080 for example before applying Gammac.

Trying to figure out what the best workflow would be.
super8boy is offline   Reply With Quote
Old 18th October 2016, 23:33   #68  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by super8boy View Post
That's seriously great work. Well done. Really impressive.
How well does it work on film for which an overscan has been done?
I haven't yet had the chance to use this, but for similar situations you merely create a cropped clip and use that. You can then apply the modifications to the uncropped clip, or apply the modifications to the cropped clip and then add borders. Either way, you end up with the same sized clip as the original.
johnmeyer is offline   Reply With Quote
Old 18th October 2016, 23:55   #69  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
You can set args x,y,w,h and coord=true, to show current coords on screen for DC detect clip. Set cords =false for use.
Cropping DC would also work fine.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 19th October 2016, 01:00   #70  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Now not mobile.

The above windowed demos did not I think cope with DC detect clip of different size to source (just for the windowed display),
I have done updated scripts some time ago, but there seemed to be little interest and so did not bother to update scripts.
Also, the published GamMatch() plug, suffered from Access Violations if not given two clips (1 was supposed to be optional),
I dont know why this occurred, from examining source I think there should have been no problem, but there was. Anyway,
made second clip non optional and problem has disappeared. (I had been testing with AVS+, perhaps problem was in some way related,
it seemed to crash before getting 'into' the plugin, so perhaps something in AVS+ was accessing optional clip when not supplied).
I shall try to update GamMatch() plug, and also demo windowed routines for both GamMac() and GamMatch(), soon.

GamMac(), lets you alter Channel averages to match one of the other channels (or optional specific value), eg faded color on aged film.

GamMatch, Allows to correct a clip that has had Levels + Gamma badly applied, to one of more channels, by using an in sync version
with good un-messed with color, but perhaps of inferior resolution.
GamMatch is unlikely to be of any use where eg Tweak or some other non RGB levels type mod has been applied, it is yet though not certain
(as nobody has responded) as to whether or not a faded hi rez clip could be fixed via a non faded DC detect clip.
(Actually, Tweak does not seem to give valid ColorSpace's in v2.6 AVS docs, just tested, does not support RGB).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 20th October 2016, 06:47   #71  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
OK, Here modded Windowed Demo script that supports different sized DC detect clip.

Code:
Avisource("1941 Flint Michigan Parade [Low, 360p].mp4.AVI")

ConvertToRGB24.KillAudio
O=Last
DC=Last

# Test DC can be different size
DC=DC.Blur(1.0).BilinearResize(320,240)           # Detection Clip Can be different size(uses source clip if dc not supplied, Denoised or whatever)

LockChan = 1            # Chan for lock to Ave, 0=R, 1=G, 2=B
                        #    -1 = Use LockVal below.
                        #    -2 = LockVal=(RedAve+GrnAve+BluAve)/3.0 for LockVal.
                        #    -3 = LockVal=Median(RedAve,GrnAve,BluAve)
Scale=2
RedMul   = 1.0          # Required Ave multiplier for Red Channel, applied when requesting GuessGamma(reqAve*RedMul), Even applied when LockChan=-1.
GrnMul   = 1.0          #    Same for Green channel. GrnMul even applies when LockChan is Green Channel, etc for chans.
BluMul   = 1.0          #    Same for Blue channel.  Allows tinkering/fine tuning.
Th       = 0.01
LockVal  = 128.0        # Only valid if LockChan == -1
RngLim   = 11
GamMax   = 10.0
Show     = True         # Metrics
Verb     = 4            # Verbocity FULL
SHOWCOORDS=True         # Show DC detect clip instead of original

x =5                   # Coords (for dc Detection Clip)
y =5
w=-5
h=-5

omin=5                 # Output channels minimum
omax=250               # Output channels maximum

#Return GamMac(DC,x=x,y=y,w=w,h=h,Coords=True)    # Show Coords only

Scale=0
A_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
A=O.GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)


Scale=1
B_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
B=O.GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)

Scale=2
C_TEXT = RT_String("Scale=%d rMul=%.2f gMul=%.2f bMul=%.2f",Scale,RedMul,GrnMul,BluMul)
C=O.GamMac(LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)

COORDS=O.GamMac(dc=DC,x=x,y=y,w=w,h=h,Coords=True)
ODC=((SHOWCOORDS)?COORDS.Spline36Resize(width,Height):O)                    # Resize COORDS (Not necessarily the same sizse as source clip)
ODC_TEXT=(SHOWCOORDS)?"Detect Clip with Coords":"Original"

TOP=StackHorizontal(Sub(ODC,ODC_TEXT),Sub(A,A_TEXT))
BOT=StackHorizontal(Sub(B,B_TEXT),Sub(C,C_TEXT))
StackVertical(TOP,BOT)
return Last

Function Sub(clip c,string tit) {StackVertical(c.BlankClip(height=20).ScriptClip("""Subtitle(RT_String("%d] %s",current_frame,tit))""",Args="tit",Local=true),c)}
The script as above has created a dummy DC detect clip by Blurring(1.0) and BilinearResize(320,240)
[EDIT: Resize to test DC different size in window display, Blur for no particular reason other than to throw a spanner in the works]
The source clip is 490x360 in size.
Also, SHOWCOORDS=True, to show DC and not the original clip frame.
The Coords shown are relative to the downsize DC clip which was upsized again for display
to match the source clip size.
oMin and oMax set at 5 and 250 respectively (foot and head room for further tweaking).
Below, DC clip shows coords as dotted lines. Pixels outside of the dotted line coords are not considered when
processing the DC frame.
The Blur/Resize test line can be commented out.

__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th June 2018 at 14:47.
StainlessS is offline   Reply With Quote
Old 26th October 2016, 00:04   #72  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
I guess that the nearest existing thing to GamMatch() [posted and linked a few posts earlier] is MatchHistogram() by LaTo:- http://forum.doom9.org/showthread.ph...ight=histogram
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 27th October 2016, 16:57   #73  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
New version GamMatch v0.02 (NOTE GamMatch, not this thread plugin, but a spinnoff from it).
See MediaFire in my sig below this post.

Code:
GamMatch(), [Gamma Matcher] An daft Idea, based upon an extraordinary Idea by VideoFred (the gent from Gent). (Daft idea and code by StainlessS).

    The Idea behind the daft idea(GamMac):-  http://forum.doom9.org/showthread.php?p=1774281#post1774281
    Original Idea:-                          http://forum.doom9.org/showthread.php?t=173683

RGB Only.

    Useful to correct bad color clip where there is a better color source of perhaps lower rez available.
    Additional tweaking via RedMul, GrnMul and BluMul multipliers.
      GamMatch, Allows to correct a clip that has had Levels + Gamma badly applied, to one of more channels, by using an in
    sync version clip with good un-messed with color, but perhaps of inferior resolution.
    GamMatch is unlikely to be of any use where eg Tweak or some other non RGB levels type mod has been applied.

    What it does(roughly):-
         Alter bad source color clip channel minimums, maximums and averages to match dc detect (good color) clip.

GamMatch(Clip c,clip dc,
      \ Float "RedMul"=1.0,Float "GrnMul"=1.0, Float "BluMul"=1.0,
      \ Float "Th"= 0.04,Float "loTh"=Th,Float "hiTh"=Th,
      \ int "RngLim"=11,Float "GamMax"=10.0,
      \ int "x"=20,int "y"=20,int "w"=-20,int "h"=-20,
      \ Bool "Show"=True,int "Verbosity"=2,Bool "Coords"=false
      \ )

    c, no default.              Bad Color source clip.

    dc, no default.             Detection clip (good color clip). Must be same ColorSpace and FrameCount as source clip, no other similarities enforced.

    RedMul, default 1.0         Red channel multiplier adjustment.   [0.1 <= RedMul <= 10.0]
    GrnMul, default 1.0         Green channel multiplier adjustment. [0.1 <= GrnMul <= 10.0]
    BluMul, default 1.0         Blue channel multiplier adjustment.  [0.1 <= BluMul <= 10.0]

    Th, Default 0.04           Sets Default for loTh and hiTh. Suggest Default, 0.04(percent).  [-1.0(OFF) , or 0.0 -> 1.0]
    loTh, Default Th           As for Ignore_low in AutoLevels, or Threshold in YPlaneMin.  [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding minimum R, G or B channel values.
                               -1.0 is OFF, input channel minimum is set to 0 as for levels(0,gamma,input_max, ... ).
                               If loTh >=0.0, then will scan frame looking for lowest pixel value whose cumulative sum
                               [including all pixels counts of lower value pixels] is greater than loTh%.
                               loTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).
    hiTh, Default Th           As for Ignore_high in AutoLevels, or Threshold in YPlaneMax. [-1.0, or 0.0 -> 1.0]
                               Percent, amount of extreme pixels (eg noise) to ignore when finding maximum R, G or B channel values.
                               -1.0 is OFF, input channel maximum set to 255, as in levels(input_min,gamma,255, ... ).
                               If hiTh >=0.0, then will scan frame looking for highest pixel value whose cumulative sum
                               [including all pixels counts of higher value pixels] is greater than hiTh%.
                               hiTh, only shown in metrics if greater or equal to 0.0 ie switched ON (Always shown when Verbosity=3=FULL).

    RngLim, default 11         [1 <= RngLim <= 32]
                               If ALL THREE RAW input channel ranges ie (ChannelMax(max(hiTh,0.0))-ChannelMin(max(loTh,0.0))) are less than RngLim then
                               all scaling is disabled, and remapping is linear without gamma estimation, to range omin -> omax,
                               ie avoid remapping of Black, White frames, or single color frames.

    GamMax, default 10.0       Upper value for guess gamma [2.0 <= GamMax <= 10.0]
                                   Starting guess upper range and limit for gamma estimator (probably best left alone).
                                   The lower guess range and limit will be set to 1.0 / GamMax, by default 0.1.


    x,y, Both default 20.      Area of dc Detect clip frame to sample when getting averages and estimating Gamma function, allows to ignore rubbish at frame edges.
    w,h, Both default -20.     Specified as for crop eg x=10,y=20,w=-30,h=-40, as in crop(10,20,-30,-40).

    Show, default true        True, show metrics info on frame.
    Verbosity, default 2       0 = Only upper frame metrics Flags line only
                               1 = Upper frame metrics
                               2 = Upper + important ones. (default)
                               3 = Nearly Full metrics.
                               4 = Full Metrics except version info
                               5 = Full Metrics including version info
                               Upper frame metrics shown as eg:- (when Verbosity=5=FULL)

                                  nnnnn] Flags:- SR
                                             R         G         B
                                  DC:       0,255     0,255     0,255
                                  DC_AVE: 110.423    98.284    97.997
                                  RAW:      8,247    12,223    18,239
                                  IN:       8,247    12,223    18,239
                                  IN_AVE: 104.545   120.037   149.821
                                  SCALED: 103.008   130.566   152.102
                                  GAMMA:    1.099     0.659     0.495
                                  OUTAVE: 110.416    98.282    97.986

                               where,
                                  nnnnn, is the frame number.
                                  Flags:- (Specific to current frame, can change frame to frame)
                                      'S' = Scale, mode signified by color.
                                            Greyed out. Scale = 0(No Effect). May be Greyed out if all channels Min/Max are 0,255.
                                            White.  Scale = 1[Scales input channel average maximum dynamic range of R,G,B]
                                            Orange. Scale = 2[Scales input channel average dynamic range of R and G and B, Individually]
                                      'R' = Limited by RngLim, mode signfied by color.
                                            Greyed out. Not Range Limited.
                                            Red, at least 1 channel has remapping disabled.
                                  DC:     Shows comma separated channel minimum and maximum of DC clip(dependent upon loTh, hiTh).
                                  DC_AVE: DC Clip channel averages.
                                  RAW:    Shows RAW comma separated channel minimum and maximum, eg ChannelMin(max(loTh,0.0)) and (ChannelMax(max(hiTh,0.0)),
                                          only shown if Verbosity>=3 or, if any RAW input range is less than RngLim AND any of the RAW inputs are different
                                          to the equivalent standard input.
                                  IN:     Shows comma separated channel minimum and maximum (dependent upon Scale, loTh, hiTh).
                                  IN_AVE: Input channel averages.
                                  SCALED: Scaled input averages, (dependent upon Scale, loTh, hiTh, channel minimums and maximums).
                                  GAMMA:  Estimated gamma to achieve lockval for channel. (dependent upon pretty much everything).
                                  OUTAVE: Output channel average ie rendered result.

                               DC and DC_AVE derived from DC (Good color clip), all others from clip c (BAD color clip, Incl OUTAVE).

    Coords, default False.  If True, then shows DC clip with dotted lines showing the x,y,w,h coords plotted on frame. (All other functionality disabled).
New Demo in AVS folder, copes with different sized DC clip.
Got rid of Scale argument, does not really make sense if trying to match to another clip.

No image at the moment, Photobucket seems to be having DNS problems.

EDIT: Managed to kick my way into PhotoBucket via another route


EDIT: The Demo.avs
Code:
# Some good color clip
#Avisource("F:\v\StarWars.AVI")
Avisource("1941 Flint Michigan Parade [Low, 360p].mp4.AVI")
ConvertToRGB24.KillAudio

DC=Last

# Simulate bad Input clip
RED=ShowRed(Pixel_Type="Y8").Levels(  0,0.8,255, 8,247,coring=false)
GRN=ShowGreen(Pixel_Type="Y8").Levels(0,1.5,255,12,223,coring=false)
BLU=ShowBlue(Pixel_Type="Y8").Levels( 0,2.0,255,32,239,coring=false)
MergeRGB(RED,GRN,BLU,pixel_Type="RGB24")

# Simulate DC Low res clip with good color
DC=DC.Blur(1.58).Blur(1.58).Spline36Resize(320,240)   # Fake change of size for detect

O=Last

RedMul    = 1.0          # Required Ave multiplier for Red Channel, applied when requesting GuessGamma(reqAve*RedMul), Even applied when LockChan=-1.
GrnMul    = 1.0          #    Same for Green channel. GrnMul even applies when LockChan is Green Channel, etc for chans.
BluMul    = 1.0          #    Same for Blue channel.  Allows tinkering/fine tuning.
Th        = 0.04
RngLim    = 11
GamMax    = 10.0
Verb      = 5            # Verbocity FULL
SHOWCOORDS= False

x =5                   # Coords (for dc Detection Clip)
y =5
w=-5
h=-5

A_TEXT = RT_String("OUT: rMul=%.2f gMul=%.2f bMul=%.2f",RedMul,GrnMul,BluMul)
A=O.GamMatch(DC,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,RngLim=RngLim,GamMax=GamMax,
    \ x=x,y=y,w=w,h=h,
    \ Show=True,Verbosity=Verb)

B_TEXT = RT_String("OUT No Metrics: rMul=%.2f gMul=%.2f bMul=%.2f",RedMul,GrnMul,BluMul)
B=O.GamMatch(DC,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,RngLim=RngLim,GamMax=GamMax,
    \ x=x,y=y,w=w,h=h,
    \ Show=False)


COORDS=O.GamMatch(DC,x=x,y=y,w=w,h=h,Coords=True).Spline36Resize(width,Height)          # Resize Coords to same as bad clip for window display
ODC=((SHOWCOORDS)?COORDS:DC).Spline36Resize(width,Height)                               # Resize COORDS (Not necessarily the same sizse as source clip)
ODC_TEXT=(SHOWCOORDS)?"Detect Clip with Coords":"DC Detect (Match Original to this)"

TOP=StackHorizontal(Sub(O,"Original (Simulated bad clip)"),Sub(A,A_TEXT))
BOT=StackHorizontal(Sub(ODC,ODC_TEXT),Sub(B,B_TEXT))
StackVertical(TOP,BOT)
return Last

Function Sub(clip c,string tit) {StackVertical(c.BlankClip(height=20).ScriptClip("""Subtitle(RT_String("%d] %s",current_frame,tit))""",Args="tit",Local=true),c)}
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 3rd June 2018 at 18:20.
StainlessS is offline   Reply With Quote
Old 20th January 2017, 06:20   #74  |  Link
juanitogan
Long-time Reader
 
juanitogan's Avatar
 
Join Date: Jan 2017
Location: New Mexico
Posts: 5
Thank you, everyone, for this incredible GamMac tool. I was doing something very similar with ImageMagick to recover overexposed scenes, but it was generally too aggressive and lacked temporal smoothing. What GamMac accomplishes is much more sane than what I had done so far. Far faster too. In fact, your results are very close to the autoleveling tool in Paint.NET (what I consider, personally, to be the gold standard in autoleveling -- you should see what Paint.NET does with GamMac's test files).

I seem to have uncovered, however, some occasional strange sensitivity in GamMac. That, or I'm just doing it wrong. GamMac does an incredible job with much of the film I'm working with... but with some scenes, however, GamMac creates a flicker where there is otherwise none. From what I can tell, GamMac appears to be too sensitive in determining the channel min and max.

Here's a link to a 7zip of 32 frames from one of the problem scenes:

https://www.dropbox.com/s/m1p2kccy3shaixk/f1a.7z?dl=0

Note the variations in brightness and values in this mosaic:

https://www.dropbox.com/s/fl42fpzctl92qvg/f1a.jpg?dl=0

In this particular case (a 1949 film), changing the lock channel from green to red stabilizes things quite a bit... but even the red channel is not stable (see frame 18 in my test frames, t003962.png, for example). I tried most of the settings GamMac has to offer but found little effect on the stability of the channel extents -- not in any automatic way, anyhow. Thus, setting Scale=0 (to use the channel averages, which are fairly stable) is the only automatic way I have found thus far to eliminate the flicker when using GamMac. Scale=0, however, seemed less effective overall than the other settings.

One point of suspicion is that the frames with heavy dirt appear to be more at risk for wild min numbers. But, then, there are other frames with wild numbers without heavy dirt. I then suspected maybe something unseen so I looked at these frames in Paint.NET with its Levels tool. Paint.NET autolevels the frames without much variance in channel values (after you strip off the borders like I do in the AVS script; set canvas size: 704 x 472, anchor middle). Furthermore, the autoleveling I was doing through ImageMagick was stable as well. Thus, this is why I suspect the procedures used to find the channel effective extents are too sensitive in GamMac (or something like that).

I've been using johnmeyer's excellent adaption of videoFred's amazing scripts. johnmeyer's script produces far less artifacts in the wake of moving objects with my source. I've added my current test version of his script below. (Ignore the autogamma stuff--it's just something I was playing with to bring out the shadows more in home movies.)

You will see that I was testing GamMac in two places: up front just after border cropping (partly to avoid an extra color conversion), and near the end just before border application.

I must say, putting it up front is a particularly bad idea because it makes all the later filters less effective. It does, however, reveal the issue I'm getting at much more vividly up front. In other words, adding GamMac after all the other filtering tames down the flickering quite a bit but, in my opinion, this is only somewhat hiding the issue and is not a fix to it.

[[[
I can't paste the script inline--too many characters for a single post--so here it is:
https://www.dropbox.com/s/85nvmy9t29el12x/ft.avs?dl=0
]]]


p.s. My source material is from films we sent to just8mm.com. I feel the quality is fair for the price but I will be more than happy to pay twice as much for a place like CinePost next time (which I hadn't yet discovered before). Just8mm didn't frame one of the reels correctly (seen in the samples here) and the source they sent me was a DV AVI -- which would have been okay except that their telecine was running at a very bizarre 19.3-19.8-ish fps. They are obviously asleep at the wheel. I wasted a few days of my life trying to pullup the video back into source frames for processing. Hint:

Code:
ffmpeg -y -i 1.avi -vf "pullup,yadif" -vsync 0 -q:v 1 f\t%06d.png
This resulted in some barely-perceptible loss in quality in the frames "pullup" couldn't identify and that "yadif" had to correct. The real trick here was "-vsync 0"... that part took forever to find versus trying to calculate a strange fps (which always led to dupe frames here and dropped frames there) and/or trying various other pullup methods.
juanitogan is offline   Reply With Quote
Old 20th January 2017, 10:56   #75  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Terneuzen, Zeeland, the Netherlands, Europe, Earth, Milky Way,Universe
Posts: 689
Quote:
Originally Posted by juanitogan View Post
I seem to have uncovered, however, some occasional strange sensitivity in GamMac.
Hello Juanitogan and welcome to this forum

The flickering is caused by the dirt spots and emulsion damage on your source files. Frame 3972 for example. GamMac is detecting a lot of almost pure black pixels on this one.
You can set LoTh pretty high (above 0.40) and the flickering will be almost gone.

You can also use a precleaned clip (detect) for the GamMac analysis, for example with removedirt().

Or you can clean the clip itself, before using GamMac.

PS: You need a better transfer, there is much more detail in those films than what I see on your examples.

many greetings,
Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 21st January 2017, 00:48   #76  |  Link
juanitogan
Long-time Reader
 
juanitogan's Avatar
 
Join Date: Jan 2017
Location: New Mexico
Posts: 5
Hi videoFred, thank you for the welcome. I'm a fan of your digitizers. Even before seeing your work, I was tempted to build my own film scanner. I just can't justify the time with how little I would likely use it. Thank you for sharing your knowledge on that.

Thank you, also, for the loTh hint. I had tested the Th setting but had only gone 2x to 3x of default, if I recall correctly. Setting an extreme loTh does indeed work as you said it would. Apparently, the default is adequate for lighter scenes, dirt or no dirt, but darker scenes like this one require greater measures to account for dirt and other variance. Makes sense. I'll work with it more next week and see where it goes.

Yes, frame 3972 is an obvious example of how dirt affects this. That is the most extreme I have and why I included it in my sample. Yet, frame 3955 appeared to be an anomaly with no visible dirt (and I had many others like 3955). Hence, my suspicions that an algorithm somewhere can maybe be improved. Furthermore, dirt or no dirt, other applications appear to do a better job finding effective extents automatically. But that's just my two cents worth and I'll leave it to others more expert in these filters to figure that out. I hope my sample may be useful for such testing.

Yeah, I need a better transfer -- why I mentioned my experience with just8mm.com -- so others know what to expect from them (and their sister sites: 8mmtodvd.com and dvdmemories.com). This was our first trial with transferring our collection and we sent a few reels to just8mm.com. We'll be trying somewhere with better equipment as budget allows. Supposedly, most of our collection is footage of architecture in Europe, post WWII, because my grandfather was a builder with marble. Not sure when we'll get to that stuff, if ever.
juanitogan is offline   Reply With Quote
Old 21st January 2017, 05:34   #77  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by juanitogan View Post
Yeah, I need a better transfer -- why I mentioned my experience with just8mm.com -- so others know what to expect from them (and their sister sites: 8mmtodvd.com and dvdmemories.com). This was our first trial with transferring our collection and we sent a few reels to just8mm.com. We'll be trying somewhere with better equipment as budget allows.
I suggest that you put aside an hour or two to do some researching for promotions and deals on film transfer. Until a few years ago, you are correct that there was usually a big difference between the really cheap local transfer houses; the big chains (like YesVideo, which does the Costco and Walmart transfers); and the true professional labs that use a DataCine scanner.

These high-end scanners cost well over $100,000 so the labs have to charge a lot to recover that cost. A few years ago Blackmagic acquired one of the main companies in the business, and I think they reduced the prices quite a bit. That move, coupled with the natural decline in high-end film transfer business has made many labs very eager to take business such as yours. I have very often seen deals for 8mm, Super 8, and 16mm transfers at prices not much higher than those charged by small labs using Roger Evan's MovieStuff equipment which, while good, can never equal the quality of these "ultimate" transfer units.

So, start locally in NM, and if you can't find something in-state, start looking in Los Angeles where I think you may be surprised at the pricing you can get.
johnmeyer is offline   Reply With Quote
Old 21st January 2017, 11:13   #78  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
You can also use a precleaned clip (detect) for the GamMac analysis, for example with removedirt().
juanitogan, dont overlook above quote from VideoFred, you could clean the DC detect clip quite aggressively and should still produce good results.
EDIT: Even a simple Blur(1.5) or similar would calm isolated noise pixels and reduce their affect on histograms and amount of correction.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 21st January 2017 at 11:22.
StainlessS is offline   Reply With Quote
Old 23rd January 2017, 07:54   #79  |  Link
juanitogan
Long-time Reader
 
juanitogan's Avatar
 
Join Date: Jan 2017
Location: New Mexico
Posts: 5
Thanks for the advice all. Don't worry, though, I may not have tested loTh high enough, but I don't feel like I've overlooked much:

Quote:
Originally Posted by juanitogan View Post
I've been using johnmeyer's excellent adaption of videoFred's amazing scripts. ... You will see that I was testing GamMac in two places... I must say, putting it up front is a particularly bad idea because it makes all the later filters less effective. It does, however, reveal the issue I'm getting at much more vividly up front. In other words, adding GamMac after all the other filtering tames down the flickering quite a bit...
So, yes, I'm already on top of the effects of dirt removal and such with GamMac. I also had already spent some time shopping around for a better service to get a better transfer ("twice as much" is far better than I expected to find for a wet transfer):

Quote:
Originally Posted by juanitogan View Post
I will be more than happy to pay twice as much for a place like CinePost next time
I just thought there was maybe more intel I could contribute.

Thanks again, I'll keep tweaking GamMac.
juanitogan is offline   Reply With Quote
Old 30th January 2017, 12:18   #80  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Terneuzen, Zeeland, the Netherlands, Europe, Earth, Milky Way,Universe
Posts: 689
I have just discovered that GamMac also can be used as a deflicker filter, with LockChan -1 . The results are truly amazing!

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Reply

Tags
color cast, correction, fade, gamma

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.