detect bright/dark

[fenomeno83]

Hi.
Do you know some algorithm/plugin to delect if a frame/image is dark or bright?
something that calculates a value of "dark" or "bright" perception.

thanks

[StainlessS]

Have you tried AverageLuma, http://avisynth.nl/index.php/Internal_functions#Average
(required for use in eg Scriptclip) See also the following few functions.

[fenomeno83]

Hi...I need to use in a visual studio c++ project

IScriptEnvironment* env = CreateScriptEnvironment();
AVSValue input = env->Invoke("ImageReader", filename);

input = env->Invoke("ConvertToYV12", input);

AVSValue luma = env->Invoke("AverageLuma", input);

but I've this error: Average Plan: this filter can only be used within runtime filters...

I've no problem with others filters,plugin.
how can use AverageLuma?

Thanks

[fenomeno83]

Hi..i've tried averageluma, but i don't like for my purpouse.
I need something more intellingent:
For example if i have a subject with over exposure, but in "dark" background(for example when you make a photo with flash; a close up face ), the average luma is low, but i need to understand that the image is bright..

Thanks

[wonkey_monkey]

You'll first need to decide how you want to define "bright" and "dark," then. Is it when 50% of pixels are brighter than 128? Or 25%? Or 10%?

Or maybe there isn't an answer, because it depends too much on subjective judgement.

[fenomeno83]

Something like this
https://stackoverflow.com/questions/...20098#22020098

[raffriff42]

My approach is:
* Find histogram maximum (HMax) using threshold for removing hot pixels.
* Calculate mean values of all pixel between HMax * 2/3 and HMax

Something like this?

ScriptClip(Last, """
  Grayscale
  ## histogram maximum (HMax) using threshold for removing hot pixels.
  hmax = YPlaneMax(threshold=1.0)
  
  ## mean value of all pixels between HMax * 2/3 and HMax
  max2  = 0.667*hmax
  sclip = Levels(Round(max2), 1.0, hmax, 0, 255, coring=false)
  smean = sclip.AverageLuma * (hmax-max2)/256.0 + max2
  
  #BlankClip(Last, color_yuv=(Round(smean)*$10000) + (128*$100) + 128)
  
  Subtitle("adj. mean =",
  \   size=Height/12, align=8, text_color=$33e0e0e0)
  Subtitle(
  \ String(smean, "%0.3f"), 
  \ size=Height/8, align=8, y=Height/8, text_color=$33e0e0e0)
""")

[fenomeno83]

YOU'RE GREAT!!! THANKS!
I'll try your script

[StainlessS]

Great stuff Raff.

Below an alternative of same script but using Eval instead of ScriptClip, but pretty much as Raff script (processes single frame 0 only, no matter how long clip is).

FN="a.jpg"
#FN="b.jpg"

ImageSource(FN,end=0).ConvertToYv12

Crop(0,96,0,0)  # Crop existing metrics off Raff's images

SSS="""
    ### CONFIG ###
    current_frame = 0            # Simulate within ScriptClip, (set dummy run time frame number)
    HOTTHRESH     = 1.0          # Ignore up to 1.0% of HOT PIXELS when getting max
    HMUL          = 0.667
    ##############

    Grayscale
    ## histogram maximum (HMax) using threshold for removing hot pixels.
    hmax = YPlaneMax(threshold=HOTTHRESH)

    ## mean value of all pixels between HMax * 2/3 and HMax
    max2  = HMUL*hmax
    sclip = Levels(Round(max2), 1.0, hmax, 0, 255, coring=false)
    smean = sclip.AverageLuma * (hmax-max2)/256.0 + max2

    #BlankClip(Last, color_yuv=(Round(smean)*$10000) + (128*$100) + 128)

    Subtitle("adj. mean =",
    \   size=Height/12, align=8, text_color=$33e0e0e0)
    Subtitle(
    \ String(smean, "%0.3f"),
    \ size=Height/8, align=8, y=Height/8, text_color=$33e0e0e0)
    return last
"""

Eval(SSS)

I guess that you could return a number, and collect within CPP as an AVSValue.AsInt. EDIT: AVSValue.AsFloat

fenomeno83,
When you first posted, I said as afterthought that your thread should not be in Devs forum, but in Usage or Newbie forum. On exit,
I found that thread had already been moved to Usage by Wilbert(Moderator), and so deleted that comment.
It took us till post #4 to find out what your requirement actually was, and that it really should have been in Devs forum.
Maybe you should be a little more explicit in first post what it is that you actually want.

[StainlessS]

Simulated CPP use,

# Req RT_Stats:- http://forum.doom9.org/showthread.php?t=165479&highlight=rt_Stats

A=ImageSource("a.jpg",end=0)
B=ImageSource("b.jpg",end=0)

A++B                            # Make 2 frame clip

ConvertToYv12.Crop(0,96,0,0)    # Crop existing metrics off Raff's images

# Template String
SSS="""
    ### CONFIG ###
    current_frame = %d           # Simulate within ScriptClip, (set dummy run time frame number)
    HOTTHRESH     = %f           # Ignore up to 1.0 Percent of HOT PIXELS when getting max
    HMUL          = %f
    ##############

    Grayscale
    ## histogram maximum (HMax) using threshold for removing hot pixels.
    hmax = YPlaneMax(threshold=HOTTHRESH)

    ## mean value of all pixels between HMax * 2/3 and HMax
    max2  = HMUL*hmax
    sclip = Levels(Round(max2), 1.0, hmax, 0, 255, coring=false)
    smean = sclip.AverageLuma * (hmax-max2)/256.0 + max2
    return smean
"""

# Simulate CPP usage (Requires GSCript:- http://forum.doom9.org/showthread.php?t=147846&highlight=GSCript)
GSCript("""
    for(i=0,Last.FrameCount-1) {
        S=RT_String(SSS, i, 1.0, 2.0/3.0)                   # set configs [EDIT: As for sprintf.]
        smean = Last.Eval(S)                                # Simulated CPP call to get smean for frame i.
        RT_DebugF("%d] SMean = %f",i,smean)                 # Requires DebugView:- https://docs.microsoft.com/en-us/sysinternals/downloads/debugview
    }
""")

Return MessageClip("ALL DONE, See DebugView")

EDIT:
Debugview Output (top of frames cropped so diff to orig Raff's image results)

00000007	382.33502197	[3644] RT_DebugF: 0] SMean = 59.007420
00000008	382.35046387	[3644] RT_DebugF: 1] SMean = 98.937561

EDIT: Not sure if below would work, (Gavino would defo know)

# Req RT_Stats:- http://forum.doom9.org/showthread.php?t=165479&highlight=rt_Stats

A=ImageSource("a.jpg",end=0)
B=ImageSource("b.jpg",end=0)

A++B                            # Make 2 frame clip

ConvertToYv12.Crop(0,96,0,0)    # Crop existing metrics off Raff's images

# Function, Not sure how to add this in CPP, Maybe an Eval would work, maybe not.
Func_S="""
    Function MyFunc(clip c, Int n, Float Th, Float HMul) {
        c
        n = (n < 0) ? 0 : (n >= FrameCount) ? FrameCount-1 : n     # Range limit n
        current_frame = n
        ## histogram maximum (HMax) using threshold for removing hot pixels.
        hmax   = YPlaneMax(threshold=Th)
        ## mean value of all pixels between HMax * 2/3 and HMax
        max2   = HMul*hmax
        sclip  = Levels(Round(max2), 1.0, hmax, 0, 255, coring=false)
        Return sclip.AverageLuma * (hmax-max2)/256.0 + max2
    }
"""

Eval(Func_S)    # make permanently available function

# Simulate CPP usage (Requires GSCript:- http://forum.doom9.org/showthread.php?t=147846&highlight=GSCript)
GSCript("""
    HOTTHRESH     = 1.0           # Ignore up to 1.0 Percent of HOT PIXELS when getting max
    HMUL          = 2.0/3.0
    for(i=0,Last.Framecount-1) {
        S=RT_String("MyFunc(%d,%f,%f)", i, HOTTHRESH, HMUL) # Func call string : RT_String == sprintf
        #S=String(i,"MyFunc(%.0f,")+String(HOTTHRESH,"%f,")+String(HMUL,"%f)")   # Alternative without RT_Stats, but still required for RT_DebugF()
        smean = Last.Eval(S)                                # Simulated CPP call to get smean for frame i.
        RT_DebugF("%d] SMean = %f",i,smean)                 # Requires DebugView:- https://docs.microsoft.com/en-us/sysinternals/downloads/debugview
    }
""")

Return MessageClip("ALL DONE, See DebugView")

[fenomeno83]

Thank you all guys!!

[Gavino]

Originally Posted by StainlessS

Not sure if below would work, (Gavino would defo know)

Do you mean the bit about Invoke'ing Eval() with the function text to get the function declared when using CPP?
Yes, that should work.

[StainlessS]

Yep, thats what i meant, thanx big G.

Mobile,

[fenomeno83]

mmm...seems that the algorithm doesn't reach my purpouse..
consider these 4 images:
1)

2)

3)

4)


Here original images:
https://www.dropbox.com/s/dev79m0ve0c623l/img.zip?dl=0

I get respectively 159,161,168 and 161.
Values seems close. Am I wrong something?

I'd like something that help me to understand automatically:
1° image is very dark (so I need to increase brightness/contrast), 2° image is good or a little over exposed, 3° a little dark, 4° is good

[lansing]

You suppose to be able to fix it with highlightlimiter + raising the gamma

highlightlimiter()
Levels(0, 1.6, 255, 0, 255)

But highlightlimiter is forever loading with image inputs, don't know why.

[StainlessS]

I guess that assigning exposure quality based on pixel count/histogram is prone to error.
Above, only #2 shows distinctly overbright (brightness added to ALL luma levels). EDIT: Quote "2° image is good or a little over exposed".
Perhaps only something like OpenCV library (or similar) that compare with huge database of images (looking for similar image),
could (perhaps) reduce number of erroneous guesses. [ EDIT: Assuming OpenCV lib does have that capability. ]

EDIT: Based only on above (luma) histograms, how would you create a rule to get it always correct ?
EDIT: Also consider two perfectly exposed images,
#1) Star on black sky,
#2) House fly on hospital wall.

EDIT: I'm no photographer and have no idea at all how one would perfectly expose #3 (on camera shoot).
Perhaps #3 is the one lansing was talking about with highlightlimiter (or maybe #1).

EDIT: Lansing script, does indeed fix #1 and #3 a bit.

function HighlightLimiter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", bool "softlimit", int "method")
{
gradient = default (gradient,true) #True uses the gaussian blur to such an extent so as to create an effect similar to a gradient mask being applied to every area that exceeds our threshold.
gblur = (gradient==true) ? default (gblur,100) : default (gblur,5.0) #The strength of the gaussian blur to apply.
threshold = default (threshold,150) #The lower the value, the more sensitive the filter will be.
twopass = default (twopass,false) #Two passes means the area in question gets darkened twice.
amount = default (amount,10) #The amount of brightness to be reduced, only applied to method=2
softlimit = default (softlimit,false) #If softlimit is true, then the values around the edges where the pixel value differences occur, will be averaged.
method = default (method, 1) #Method 1 is multiply, the classic HDR-way. Any other method set triggers a brightness/gamma approach.

amount = (amount>0) ? -amount : amount

darken=v.Tweak(sat=0).mt_lut("x "+string(threshold)+" < 0 x ?")
blurred= (gradient==true) ? darken.gaussianblur(gblur).gaussianblur(gblur+100).gaussianblur(gblur+200) : darken.gaussianblur(gblur)
fuzziness_mask=blurred.mt_edge(mode="prewitt", Y=3, U=2, V=2).mt_expand(mode="both", Y=3, U=2, V=2)
multiply = (method==1) ? mt_lut(v,"x x * 255 /") : v.Tweak(bright=amount)
multiply = (method==1) ? eval("""
(twopass==true) ? mt_lutxy(multiply,v,"x y * 255 /") : multiply""") : eval("""
(twopass==true) ? multiply.SmoothLevels(gamma=0.9,smode=2) : multiply""")

merged=mt_merge(v,multiply,blurred)
fuzzy= (softlimit==true) ? mt_merge(merged,mt_lutxy(v,merged,"x y + 2 /"),fuzziness_mask) : merged
return fuzzy
}

A=ImageSource("1.jpg",end=0)    B=ImageSource("2.jpg",end=0)    C=ImageSource("3.jpg",end=0)    D=ImageSource("4.jpg",end=0)    A++B++C++D

ConvertToYv12

O=Last

highlightlimiter()
Levels(0, 1.6, 255, 0, 255)

StackVertical(O,Last)

#3

[raffriff42]

Originally Posted by fenomeno83

I'd like something that help me to understand automatically:
1° image is very dark (so I need to increase brightness/contrast), 2° image is good or a little over exposed, 3° a little dark, 4° is good

Try different numeric functions. I used yours, since you seemed to be very specific. A simple AverageLuma would probably work better for your purposes.

And you did not ask about correction, only detection. Correction of a badly backlit subject like you have here will be -- at best -- a tradeoff between showing the subject and blowing out the background.

Here's a manual correction with brighter foreground, but blown-out windows and a washed-out appearance. It took some fiddling to get even that.

[StainlessS]

Lansing/Javlak correction for #1.

[lansing]

I also did some gamma tests between the internal levels(), smoothlevels() and the levels in photoshop. They all produced different results with the same gamma.

The internal levels has no limiting, when you have a big gamma increase like >2.0, the dark would also became brighter, so that's not good.

Smoothlevels has limiting and smoothing, so it's able to keep the darks in place even with big gamma increases, and the smoothing help prevent artifacts. But the downside is that it desaturate the color.

Photoshop's levels has dark limiting, smoothing and preserved saturation.

Photoshop's levels with gamma 2.08


It has better color compares to the other two.

So to the OP, if you're just doing it with photos, sticks with photoshop or lightroom, they'll give you better results. And I think lightroom has feature to deal with overexposed highlight too.

[poisondeathray]

Is he just looking for "perception" detection , or actually doing something about it ?