GamMac v1.10 - 15 June 2018

[StainlessS]

GamMac() v1.07, new version, see 1st post [Beta lifted].

    v1.07, Bug fixed in GuessGamma(), previously found gamma could have been slightly nearer than result.

EDIT:

Previously implemented something like this

    Function GuessGamma(clip c,float reqAveLuma,float "GamHi",Float "GamLo") {                  # Single Frame Clip only
        Assert(c.IsYV12,"GuessGamma: Requires YV12")
        gamHi = Default(GamHi,2.0)  gamLo = Default(GamLo,1.0/gamHi)        
        Result = -1.0
        ALDif=0.0001
        PrevAveL = -1.0
        while(GamLo < gamHi) {
            gamMid = (gamLo + gamHi) / 2.0
            AveL=c.Levels(0,gamMid,255,0,255,coring=false).RT_AverageLuma(0)            
            if(abs(AveL-PrevAveL)<=ALDif) {                                  
                Result = gamMid
                gamLo = gamHi + 1.0                                 # Force Exit, Not getting any nearer 
            } else if(AveL < reqAveLuma) {
                gamLo = gamMid
            } else if(AveL > reqAveLuma) {
                gamHi = gamMid
            } else {
                Result = gamMid
                gamLo = gamHi + 1.0                                 # Force Exit, exact match 
            }
            PrevAveL = AveL 
        }
        Return Result         
    }

Now implemented something like this

    Function GuessGamma(clip c,float reqAveLuma,float "GamHi",Float "GamLo") {                  # Single Frame Clip only
        Assert(c.IsYV12,"GuessGamma: Requires YV12")
        gamHi = Default(GamHi,2.0)  gamLo = Default(GamLo,1.0/gamHi)        
        Result = -1.0
        PrevAveL = -1.0
        Bestgmid=GamHi
        BestDif=256.0
        while(GamLo < gamHi) {
            gamMid = (gamLo + gamHi) / 2.0
            AveL=c.Levels(0,gamMid,255,0,255,coring=false).RT_AverageLuma(0)            
            dif=abs(AveL-reqAveLuma)
            if(dif<BestDif) {
                Bestgmid=gamMid;
                BestDif=dif;
            }
            if(abs(AveL-PrevAveL)==0.0)  {
                result = Bestgmid 
                gamLo = gamHi + 1.0                                 # Force Exit, Not getting any nearer 
            } else if(dif<0.00001 ) {
                result = Bestgmid;                                  # close enough 
                gamLo = gamHi + 1.0                                 # break 
            } else if(AveL>reqAveLuma) {
                gamHi = gamMid
            } else {
                gamLo = gamMid
            }
            PrevAve=AveL
        }
        Return Result         
    }

EDIT: Although above is for YV12 and GamMac is for RGB.

[StainlessS]

GamMatch() v1.03, new version

    v1.03, Bug fixed in GuessGamma(), previously found gamma could have been slightly nearer than result.

NOTE, prev posted GamMac update was posted in wrong folder, ie not available, corrected at MediaFire in Below sig (or also on SendSpace, again below).
See MediaFire link below in sig or SendSpace link.

[johnmeyer]

I'm late to this game, but I'm in it now.

I have to agree with everyone else: this is a pretty amazing filter.

I have a test clip I created which contains thirty-nine six-frame scenes from a big film transfer job I did a year ago. These clips are the raw capture. Normally I apply a histogram to gain the lower shadows before I feed the video to any script, but I thought it would be best to use a completely unaltered capture. Here is that clip.

Test Clip

I tried to pick different types of issues that represent what I get when transferring old film (most of this is from the 1940s).

This is a 14.98 fps video using the DV codec. This particular DV codec (MainConcept) permits arbitrary frame rates, but still insists on setting the interlaced flag. This video is progressive, however, and you can set any script accordingly.

On about thirty of the thirty-nine scenes, the script works brilliantly. On the rest, there are some issue.

The first issue is that sometimes I get way too much contrast. Here's an example:

Input:


GamMac'd


I have this problem even on clips without a big chunk of brightness.

The bigger issue is that on some frames I get color that is shifted completely wrong:

Input


GamMac'd


As you can see, the input clip has pretty decent color balance, but it would be nice to gain the gamma. However, GamMac turns the fireplace brick a rather unpleasant blue-green. I'm pretty sure the original was an off-white.

I guess I can play around with the various "mul" overrides, but if I have to do that, I might as well balance each scene by hand in my NLE, which is how I've been doing this for the past six years.

I've spent several hours playing around with all the variables, but haven't hit on anything that fixes either of these problems on these scenes. Even if the function can't be used in a "set it and forget it" mode, I was still hoping I could tune it for these problem scenes.

So, in summary:

1. I was hoping for something that would gain the lower (darker) part of the gamma curve, without disturbing the highlights and black point;

2. I was hoping for more consistent color correction.

I'm not complaining, and I will still be able to get lots of use out of this for certain clips.

Here is the test script I'm using. These GamMac settings are pretty close to the defaults, but I have played around with each and every variable in order to learn about the filter, and also try to get better results for the problem scenes:

O=Last
DC=Last
LockChan = -2           # Chan for lock to Ave, 0=R, 1=G, 2=B
                        #    -1 = Use LockVal below.
                        #    -2 = LockVal=(RedAve+GrnAve+BluAve)/3.0 for LockVal.
                        #    -3 = LockVal=Median(RedAve,GrnAve,BluAve)
Scale=2

RedMul    = 1.0         # Required Ave multiplier for Red Channel, applied when requesting GuessGamma(reqAve*RedMul), Even applied when LockChan=-1.
GrnMul    = 1.0         #    Same for Green channel. GrnMul even applies when LockChan is Green Channel, etc for chans.
BluMul    = 1.0         #    Same for Blue channel.  Allows tinkering/fine tuning.

Th   	  = 0.04
loTh      = 0.04
hiTh      = 0.04
LockVal   = 128.0       # Only valid if LockChan == -1

RngLim    = 11

GamMax    = 10.0
Show      = True        # Metrics
Verb      = 2           # Verbocity FULL
SHOWCOORDS= True        # Show Original with Coords

x =20                    # Coords (for dc Detection Clip)
y =20
w=-20
h=-20

omin=0                  # Output channels minimum (footroom for manual editing).
omax=255                # Output channels maximum (headroom for manual editing).


source=AVISource("E:\fs.avi").KillAudio()

output=GamMac(source,LockChan=LockChan,SCALE=SCALE,RedMul=RedMul,GrnMul=GrnMul,BluMul=BluMul,
    \ Th=Th,LockVal=LockVal,RngLim=RngLim,GamMax=GamMax,
    \ dc=DC,loTh=loTh,hiTh=hiTh,
    \ x=x,y=y,w=w,h=h,
    \ omin=omin,omax=omax,
    \ Show=Show,Verbosity=Verb)

#return output
#stackhorizontal(source,output)

#/*
return Interleave(
\    source
\  , output
\ )
#*/

[StainlessS]

Hi John, give it a bash with Scale=1.
Scale=2, scale primaries individually giving in those cases color shift.
There is probably a big min/max range difference between the primaries.

I'll get your test clip and some point but not just yet.

EDIT: I got your clip John, Nice test clip (and not too big), thanx.

EDIT: John, request:, can post any further SOURCE images in this thread as png, result can be jpegs, thanx.

EDIT: John, can you repost or PM both those source images as png please (and verify is GamMac v1.07).

[johnmeyer]

I have appointments this morning. Should be back at my editing computer about noon, my time (8:00 p.m. your time, I think). I'll re-post roughly the same pics (I was just copy/pasting from NLE timeline, so the only way to avoid JPEG is to go back to the source). You now have the original AVI (it was losslessly cut from the original material), so you can get to the original color and avoid the compression artifacts by using that. I don't think I included the exact frames that I used for the still shots, but you do have six frames from the same scene.

P.S. I just noticed that none of the images I posted are showing up. Photobucket must be having problems. They displayed fine last night when I previewed and then posted. I guess I'll try uploading to this site.

[johnmeyer]

OK, I changed the two "input" photos, in my post above, to PNG. The GamMac'd versions are still JPEG. Photobucket is working again this afternoon, after having been flaky the past 24 hours. Hopefully it will work for you.

[Bernardd]

Hello,

In my opinion, GamMac is a tool that allows to correct the colors according to the theory of the gray world. It seeks, in fact, the equality between the mean values of each channel.
For the images of the young children in front of the chimney, I think that the gray world theory is not adapted to this case. ColorYUV (autowhite = true) give also a green
dominant at the fireplace.

With this script, ( https://forum.doom9.org/showthread.p...99#post1797799 ) which exploits the RGBAdapt plugin, a autogain function allows you to change the colors independently of the gray world theory.
As the work is in RGB espace, this autogain reinforces light and darkness but also the colors. Its application may be a solution for scenes such as the chimney or on the beach.

Bernard

(google traduction)

[videoFred]

Originally Posted by Bernardd

With this script, ( https://forum.doom9.org/showthread.p...99#post1797799 ) which exploits the RGBAdapt plugin, a autogain function allows you to change the colors independently of the gray world theory.
As the work is in RGB espace, this autogain reinforces light and darkness but also the colors. Its application may be a solution for scenes such as the chimney or on the beach.

Hello Bernard,

Can you show us a result of your script applied on these specific frames?

Fred.

[videoFred]

Originally Posted by johnmeyer

The first issue is that sometimes I get way too much contrast.

Hi John, this can be solved with oMin and oMax (see examples)

The bigger issue is that on some frames I get color that is shifted completely wrong

Yes, I have the same problem on some frames and I think it happens when one color is dominating.

As you can see, the input clip has pretty decent color balance, but it would be nice to gain the gamma. However, GamMac turns the fireplace brick a rather unpleasant blue-green. I'm pretty sure the original was an off-white.

On this specific frame, the red channel is dominant. When set LockChan to 0 (red) the green cast is better, but still not perfect:



It needs more specific "mul" settings to be correct:



I wonder if a smart guy like StainlessS can see some consistency in all these values.

Anyhow, it might be an idea to add a "auto lockchan" option. If one color is realy dominating, lockchan should lock on that color.

It's very obvious on this frame for example:



But the color dominance is sometimes changing from frame to frame so I do not know if it can be done without flickering effects etc... Also, estimated gamma will be pretty high.

Fred.

[Bernardd]

Below three results of script

two first given with only gain_strength = 0.8

https://www.dropbox.com/s/ogj4emqu2j...minee.png?dl=0

https://www.dropbox.com/s/cosk1qfjar...Plage.png?dl=0

the last given with default args gain_strength = 1.0, rpow_strength = 1.0 and bias_strength = 1.0

https://www.dropbox.com/s/57qzp5o4ck...neige.png?dl=0

Bernard

[johnmeyer]

Originally Posted by Bernardd

Below three results of script

The first two results are very, very good. The last is not good.

Do you have a list of plugins required for your script?

Originally Posted by videoFred

Hi John, this can be solved with oMin and oMax (see examples)...

I thought I had tried that, but I'll go back and try again. Thanks for that help.

Originally Posted by videoFred

On this specific frame, the red channel is dominant. When set LockChan to 0 (red) the green cast is better, but still not perfect:

That result looks exceptionally good. Once again, I thought I had tried that.

I don't mind too much if I have to run the script several times with different settings. I often create 2-3 versions of a given film and then line them up on the NLE timeline. I quickly scrub through the version that gives me the best result for the majority of the film, and when that fails, I switch to one of the other versions. It obviously takes more computer time, but since I can do other things while the computer cranks, it is not too much of a problem.

I'll go back and "play around" with the settings some more, now that I have these hints.

[videoFred]

Originally Posted by johnmeyer

I often create 2-3 versions of a given film and then line them up on the NLE timeline. I quickly scrub through the version that gives me the best result for the majority of the film, and when that fails, I switch to one of the other versions.

Yes, this is the way to go until we have a perfect all automatic script (probably impossible).
Have you ever tried ClipClop John? Another StainlessS idea and it works like a charm.

Fred.

[videoFred]

Originally Posted by Bernardd

Below three results of script

I agree with John: cheminee.prg looks very, very good Bernard.

I will see if I can get your script running here.

Fred.

[Bernardd]

The script is based on StainlessS's RGBADapt plugin http://forum.doom9.org/showthread.php?t=170642
It need StainlessS's RT_Stats plugin http://forum.doom9.org/showthread.php?t=165479
It need Gavino's GRunT plugin http://forum.doom9.org/showthread.php?t=139337 and GScript plugin http://forum.doom9.org/showthread.php?t=147846
Bernard

[StainlessS]

I am chewing on it. (tis very chewy, my jaws are aching )

[Bernardd]

In my script, gain fonction uses channel MinMaxDiff values. In extraction process for these datas Rt_Stats has a Threshold arg
which is a percentage, stating how many percent of the pixels are allowed below minimum or above maximum.

For Rt_Stats default threshold value is 0.0, in my script default value is 0.10, but with 0.50 for snow scene the ouput is below.

https://www.dropbox.com/s/thuy67k2kq...eige2.png?dl=0

Bernard

[johnmeyer]

Originally Posted by videoFred

Have you ever tried ClipClop John? Another StainlessS idea and it works like a charm. Fred.

I am aware of it, and as he was developing it I skimmed the thread. For this particular situation, I'm not sure it would save any time compared to doing the work in my NLE (Vegas Pro) which has a built-in multi-camera capability. I can view many versions, playing all at once, and cut between them (I can even do this while playing) simply by pressing a single key on the keyboard that has been assigned to that camera. I can immediately undo anything I don't like, or can "slip" the edit point one way or the other. I can also cross-fade between cameras. This is useful in this particular situation if I want to make less obvious and less abrupt the transition between two different corrections.

[StainlessS]

Originally Posted by Bernardd

For Rt_Stats default threshold value is 0.0, in my script default value is 0.10, but with 0.50 for snow scene the ouput is below.

RT_Stats Threshold and GamMac Th, (or individual set loTh, and hiTh) are pretty much the same thing. JFYI.

[videoFred]

Originally Posted by Bernardd

The script is based on.......

Yes, I know. Are you sure you have posted a good working script Bernard? I'm getting GScript syntax errors on lines 82, 149, 342 and 410. Might be a copy/paste error.. Can you upload your script in one part somewhere?

Fred.

[Bernardd]

Fred,
in DropBox, script file
https://www.dropbox.com/s/yis4gwsdb0...ocess.avs?dl=0
Bernard