How does one get the average color of the frame?

[D9JK]

Hello,

I thought this would be a simple task, but it's giving me a bit of trouble. I need a filter that averages the entire (current) frame into a single color, can be grayscale as well. If you've used Photoshop, I'm after an effect that is similar to "Filter/Blur/Average".

Some background (not necessary to read):

First I tried simply general blur filters with obscenely high values, but where doing this seemed to produce a flat color, it was simply gray across all frames, which was wrong because the average brightness should clearly have varied across the video frames. Additionally, applying excessive blurring seems to pick up some blackness from outside the image frame, but this may depend on which blur filter is being used.

Next I tried a slightly hacky way; resizing a frame to 1x1 (in image editing, this works as average blur filter), and then bringing it back to the original size. But the lowest resolution the resize filters allow is 4x4, so that didn't work.

Then I found a mention of a filter called "AverageLuma". As this sounded promising, I gave it a try. Apparently simply entering AverageLuma() causes an error message about invalid parameters for the filter... but I cannot find any helpful documentation about it. The filter is not listed neither in the internal nor external filters of Avisynth.

Perhaps I'm missing something.

[StainlessS]

Suggest take a look at Showchannels.#
http://forum.doom9.org/showthread.php?t=163829

[D9JK]

Your filter is a useful analyzing tool, but currently I'm looking for a filter that averages the entire frame into a single, flat color. That is, the filter should actually modify the frame contents.

Thank you anyway.

[poisondeathray]

Originally Posted by D9JK

Next I tried a slightly hacky way; resizing a frame to 1x1 (in image editing, this works as average blur filter), and then bringing it back to the original size. But the lowest resolution the resize filters allow is 4x4, so that didn't work.

If you work in RGB, you can use pointresize(1,1)

[Gavino]

Originally Posted by poisondeathray

If you work in RGB, you can use pointresize(1,1)

But that won't average the pixels, it will just replicate the centre one.
EDIT: Not the centre pixel, but the corner one (bottom left for RGB).

Originally Posted by D9JK

Then I found a mention of a filter called "AverageLuma". As this sounded promising, I gave it a try. Apparently simply entering AverageLuma() causes an error message about invalid parameters for the filter... but I cannot find any helpful documentation about it. The filter is not listed neither in the internal nor external filters of Avisynth.

AverageLuma() is not a filter - it is a run-time function that gives a numeric result, and it can only be used inside a run-time filter such as ScriptClip(). This should give you what you want (assuming a YUV source):

ScriptClip("""
  y = round(AverageLuma())
  u = round(AverageChromaU())
  v = round(AverageChromaV())
  BlankClip(last, color_yuv=65536*y + 256*u + v)
""")

[D9JK]

Thanks Gavino, and sorry for my ignorance, but how do I use it exactly? I've written a few custom functions, but ScriptClip looks a bit different.

[Gavino]

Just add that code after your source filter, eg

AviSource("mysource.avi") # or whatever you are using
ScriptClip("""
... as shown earlier ...
""")

[D9JK]

Ah, I see. I'm usually using filters this way:

Changed = Original.Grayscale().Invert()

The script is a bit long when I have other filters in there already, but I suppose I can put it in a new custom function, then give it a name short enough.

Thanks a bunch!

[pandy]

Perhaps resize will work for you? (1 pixel probably not).
Bilinear work to 2x2 then 4 - 5 time in series Blur (1.5, 1.5) should solve your problem (at least from my perspective looks like all 4 pixels have same value - obviously this is RGB space, for YCbCr can be else, probably limit is 4x4 and multiple pass Blur (16 times or more) is required).

[StainlessS]

This will also work with RGB & YUY2, Need Gavino's Grunt() plugin for the "After_frame=True" arg.
EDIT: No you dont, see next post.

ShowChannels(SetVar=True,Show=False)
ScriptClip("""
  SC_Ave_1 = (SC_Ave_1<0.0) ? 0.0 : SC_Ave_1      # If Avisynth v2.6, convert Y8 channels 1 and 2 to 0 (will be -1.0)
  SC_Ave_2 = (SC_Ave_2<0.0) ? 0.0 : SC_Ave_2
  z = (256*Round(SC_Ave_0)+Round(SC_Ave_1))*256+Round(SC_Ave_2)
  IsRGB ? BlankClip(last, color=z) : BlankClip(last, color_yuv=z)
""",after_frame=True)

EDITED: Added Y8 guard.

You can even select an area of the frame to examine via ShowChannels x,y,w,h args.

Changed = Original.Grayscale().Invert()

will work equally well as here (you dont need to have everything on one line)

Original.Grayscale()
Invert()
ShowChannels(SetVar=True,Show=False)
ScriptClip("""
  SC_Ave_1 = (SC_Ave_1<0.0) ? 0.0 : SC_Ave_1      # If Avisynth v2.6, convert Y8 channels 1 and 2 to 0 (will be -1.0)
  SC_Ave_2 = (SC_Ave_2<0.0) ? 0.0 : SC_Ave_2
  z = (256*Round(SC_Ave_0)+Round(SC_Ave_1))*256+Round(SC_Ave_2)
  IsRGB ? BlankClip(last, color=z) : BlankClip(last, color_yuv=z)
""",after_frame=True)
Changed=Last

[Gavino]

Originally Posted by StainlessS

Need Gavino's Grunt() plugin for the "After_frame=True" arg.

The built-in ScriptClip() also supports the after_frame arg.

[D9JK]

Although my original intention was something else, I also wrote a crude automatic white balance function as a test:

function AutoWB(clip X, float O)
 {
  X
  ConvertToYV12()
  ScriptClip(""" u=round(AverageChromaU()) v=round(AverageChromaV()) BlankClip(last, color_yuv=65536*128 + 256*u + v) """)
  Invert()
  Overlay(X, mode="softlight")
  Overlay(X, mode="chroma", opacity=1-O)
 }

I'm probably not going to use it very much, as automatic white balance functions would have to be much more sophisticated to be practical. But out of curiosity: is ScriptClip limited to working on one frame at a time, or would it be possible to somehow take nearby frames into consideration? I'm asking this because the AutoWB function above can of course produce very abrupt adjustments on the WB, and averaging the color of eg. 10-30 neighboring frames would help in making the effect less apparent.

[Gavino]

Originally Posted by D9JK

is ScriptClip limited to working on one frame at a time, or would it be possible to somehow take nearby frames into consideration?

Each output frame can be derived from several input frames, although with the standard ScriptClip(), to get access to previous or future frames you have to construct additional clips via filters such as Trim() or SelectEvery().
It's easier with the GRunT version of ScriptClip() - for example, AverageChromaU(-2) gives the average U value of the frame two before the current one.

[D9JK]

Thanks Gavino, I'll take a look. That's a funny abbreviation, too. :-)

[Dogway]

Gavino, do you know how to access every other frame? I am trying to compare YDifferenceToNext() between current and current+2, but I'm out of ideas, this is what I'm trying:

scriptclip("""Subtitle(c,string(YDifferenceToNext(Fmod(current_frame, 2.0)==0.0?even:odd)),align=7,size=30)""",
            \ args="c,even=selectevery(c,2,0,2),odd=selectevery(c,2,1,3)")
 }

What I get is a value of 0 every other frame as if they were repeated.

[StainlessS]

 RT_YDifference(clip,int "n"=current_frame,int "delta"=1,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
      bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2))
  Returns FLOAT value luma difference (0.0 -> 255.0) between frame n area x,y,w,h, and frame (n+delta) area x2,y2,w,h.
  Note, by default it will be equivalent to YDifferenceToNext as delta defaults to 1 and x,y,w,h defaults full frame.
  Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

  Eg, RT_YDifference(clip,delta=0,y2=1,interlaced=true)
    Would difference the even and odd lines of the same frame,

RT_YDifference(delta=2)

[Dogway]

If that means there's no way with built-in functions then thanks for the help.

[StainlessS]

Avisource("E:\v\avatar.avi")
target = Trim(2,0)
# target = SelectEvery(2,2,3) # EDIT: Added


SS="""
    x = LumaDifference(Target)
    y = RT_YDifference(delta=2)
    Subtitle(string(x) + " " + String(y),align=7,size=30)
"""

Last.scriptclip(SS, args="Target")

[Dogway]

Yes, thank you, RT_Stats plugin. If possible I will try to keep using built-in function, if not possible I guess RT_YDifference() is fine.

[StainlessS]

Already given built-in ie LumaDifference, RT_ was just a check for valid.
can also use SelectEvery(2,2,3) instead of trim.

Or more like your original post

return Avisource("E:\v\avatar.avi").scriptclip("""Subtitle(string(LumaDifference(SelectEvery(2,2,3))),align=7,size=30)""")

EDIT: Or probably SelectEvery(1,2) better

return Avisource("E:\v\avatar.avi").scriptclip("""Subtitle(string(LumaDifference(SelectEvery(1,2))),align=7,size=30)""")