Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th January 2022, 16:31   #1  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 105
New grayworld video filter in FFmpeg 5.0

The new version of FFmpeg (News) has a grayworld video filter for changing the white balance using an algorithm based on the gray world hypothesis. Read more about the algorithm here.
Help from FFmpeg is here.

Code:
11.102 grayworld
A color constancy filter that applies color correction based on the grayworld assumption
The algorithm uses linear light, so input data should be linearized beforehand (and possibly correctly tagged).

ffmpeg -i INPUT -vf zscale=transfer=linear,grayworld,zscale=transfer=bt709,format=yuv420p OUTPUT

Is it possible to make a filter or script function for AviSynth?

This is a port of the FFmpeg filter grayworld: grayworld

Picture from pdf file:



The vf_grayworld.c file from the source code in the attachment. (change .txt to .c)
Attached Files
File Type: txt vf_grayworld.txt (9.7 KB, 19 views)

Last edited by Arx1meD; 25th April 2022 at 18:42.
Arx1meD is offline   Reply With Quote
Old 24th January 2022, 18:21   #2  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,004
This is a cheap approach in YUV.
Code:
function GreyWorld(clip a) {

rgb  = isRGB(a)
isy  = isy(a)
px   = PixelType(a)
bi   = BitsPerComponent(a)
fs   = propNumElements (a,"_ColorRange")  > 0 ? \
       propGetInt      (a,"_ColorRange") == 0 : rgb
       
ScriptClip(a,function[bi,fs,px] () {

    U  = PlaneMinMaxStats(0, 0, 1, false)
    V  = PlaneMinMaxStats(0, 0, 2, false)
    H  = ex_bs(128,  8, bi, true, float=true)
    U  = H - U[4]
    V  = H - V[4]

    Expr(last,SwapUV(),"",Format("x {U} +"),Format("x {V} +"), Format=px)
} ) }
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 24th January 2022, 18:54   #3  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,450
Quote:
Originally Posted by Arx1meD View Post
The vf_grayworld.c file from the source code in the attachment. (change .txt to .c)
For better accessibility, source is here: https://github.com/FFmpeg/FFmpeg/blo...vf_grayworld.c

FFmpeg over the past few years has added a lot of interesting audio/video filters. Most of the unique filters for avs/vs have been ported over to FFmpeg. I'd be nice to access to the collection of filters in AviSynth.

Quote:
Originally Posted by Dogway View Post
This is a cheap approach in YUV.
GreyWorld()

Last edited by Reel.Deel; 24th January 2022 at 19:11.
Reel.Deel is offline   Reply With Quote
Old 24th January 2022, 20:38   #4  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,004
Yes, that example wasn't so good, the other one was better but I wanted to avoid going through RGB or LAB since that's very slow. Anyway I had the project in my todo list. There was an attempt before.

__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 24th January 2022, 21:04   #5  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 105
Dogway,
I know about GrayWorldSimple and ShadesOfGraySimple. They change the white balance quite normally, but they have a big drawback. When the scene changes or when an object with a different color suddenly appears in the frame, the colors flash.

Last edited by Arx1meD; 24th January 2022 at 21:08.
Arx1meD is offline   Reply With Quote
Old 24th January 2022, 21:29   #6  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 135
Quote:
Originally Posted by Arx1meD View Post
Dogway,
I know about GrayWorldSimple and ShadesOfGraySimple. They change the white balance quite normally, but they have a big drawback. When the scene changes or when an object with a different color suddenly appears in the frame, the colors flash.
Probably due to no scene detection or too much weight for one frame.

I'd recommend whoever will make an avisynth plugin out of this to take some things from this http://avisynth.nl/index.php/AutoAdjust
takla is offline   Reply With Quote
Old 24th January 2022, 21:47   #7  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,004
That will happen with mine or the paper algorithm as well because metrics is a different topic. As I said in another thread we need a plugin or something internal that accumulates/averages metrics per scenes. It requires forward look ahead.

EDIT: Improved

Code:
function GreyWorld(clip a) {

rgb  = isRGB(a)
isy  = isy(a)
px   = PixelType(a)
bi   = BitsPerComponent(a)

ScriptClip(a,function[bi,px] () {

    U  = PlaneMinMaxStats(1, 0, 1, false)
    V  = PlaneMinMaxStats(1, 0, 2, false)
    H  = ex_bs(128,  8, bi, true, float=true)
    Ux  = H - U[4]
    Vy  = H - V[4]
    Us = abs(Ux) > abs(Vy) ? "range_half - "+string(U[4] / V[4])+" * range_half +" : ""
    Vs = abs(Ux) < abs(Vy) ? "range_half - "+string(V[4] / U[4])+" * range_half +" : ""

    Expr(last,SwapUV(),"",Format("x {Ux} + "+Us),Format("x {Vy} + "+Vs), Format=px)
} ) }
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 24th January 2022 at 22:40.
Dogway is offline   Reply With Quote
Old 31st March 2022, 08:18   #8  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 105
This is a port of the FFmpeg filter grayworld: grayworld. Thank you very much Asd-g.
The description says the input clip must be in RGB(A) 32-bit planar format and in linear light.
I can't understand how to convert the clip to RGB planar format. ConvertToRGB32() and ConvertToPlanarRGBA() filters are not suitable.
Arx1meD is offline   Reply With Quote
Old 31st March 2022, 08:28   #9  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,450
Quote:
Originally Posted by Arx1meD View Post
The description says the input clip must be in RGB(A) 32-bit planar format and in linear light.
I can't understand how to convert the clip to RGB planar format. ConvertToRGB32() and ConvertToPlanarRGBA() filters are not suitable.
Use avsresize:

Code:
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="rgb:srgb:709:full=>rgb:linear:709:full")
grayworld()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="rgb:linear:709:full=>rgb:srgb:709:full")
I hope that is right, I added this to the examples section of avsresize but did not test it.

Edit: tested it with the example image in the first post:
Code:
# load jpeg in native YUV colorspace, convert it to linear RGB, process it with
# grayworld() and finally convert the result back to 8-bit sRGB.

FFImageSource("KdoTPHuv_o.jpg")
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="601:601:470bg:full=>rgb:linear:470bg:full", chromaloc_op="center=>center")
grayworld()
z_ConvertFormat(pixel_type="RGBP8", colorspace_op="rgb:linear:709:full=>rgb:srgb:709:full")
Result:


Hmm, result looks a little different than in the example. I found the original image here: https://i2.wp.com/worldadventuredive...8/img_4477.jpg

Quote:
Originally Posted by Arx1meD View Post
I can't understand how to convert the clip to RGB planar format. ConvertToRGB32() and ConvertToPlanarRGBA() filters are not suitable.
And to answer your question, only a few of the convert filters have a bitdepth designation, e.g ConvertToRGB64 will convert any clip to 16 bit RGBA (packed). However for the Convert functions that do not have a bitdepth designation, you have to convert the bipdepth yourself (before or after, depending on what your're doing). So lets say that you have an RGB clip and you wanted to convert it to float, first thing is to convert it to planar, if the clip is not already, then comes ConvertBits(32) which converts the clip to float, resulting colorspace is now RGBPS. The wiki does not mention any of this and I find it a little confusing also, and that table there confused me even more when I first was experimenting lol.

Last edited by Reel.Deel; 31st March 2022 at 11:54. Reason: typo
Reel.Deel is offline   Reply With Quote
Old 31st March 2022, 11:27   #10  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 105
Thank you Reel.Deel
I made several tests comparing the result of different filters.

Source clips: source_bubbles, source_carnival, source_cat

My comparison result of grayworld, GamMac and AWB filters:

Video comparison: test_bubbles, test_carnival, test_cat

It is difficult to compare which filter is better. I think, when the colors are heavily distorted, then grayworld gives better results, and when the colors are almost normal, then the result is worse.

Last edited by Arx1meD; 31st March 2022 at 11:29.
Arx1meD is offline   Reply With Quote
Old 31st March 2022, 14:20   #11  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,450
I can't get the same results as the example picture. In the example result it looks better, the highlights are not blown out. I tried FFmpeg also and get similar results to AviSynth.

Not very experienced with FFmpeg but I used this:
Code:
ffmpeg -i "img_4477.jpg" -vf zscale=transfer=linear,format=gbrpf32be,grayworld,zscale=transfer=bt709,format=rgba "img_4477.png"
Reel.Deel is offline   Reply With Quote
Old 31st March 2022, 16:55   #12  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 105
Reel.Deel, I took picture examples from Bianco_etal_ISPRS_CIPA.pdf

It seems to me that vf_grayworld.c has a different algorithm than Bianco_etal_ISPRS_CIPA.pdf (see APPENDIX on the last page).

From vf_grayworld.c:
Code:
static const float lms2lab[3][3] = {
    {0.5774, 0.5774, 0.5774},
    {0.40825, 0.40825, -0.816458},
    {0.707, -0.707, 0}
};

static const float lab2lms[3][3] = {
    {0.57735, 0.40825, 0.707},
    {0.57735, 0.40825, -0.707},
    {0.57735, -0.8165, 0}
};

static const float rgb2lms[3][3] = {
    {0.3811, 0.5783, 0.0402},
    {0.1967, 0.7244, 0.0782},
    {0.0241, 0.1288, 0.8444}
};

static const float lms2rgb[3][3] = {
    {4.4679, -3.5873, 0.1193},
    {-1.2186, 2.3809, -0.1624},
    {0.0497, -0.2439, 1.2045}
};
From Bianco_etal_ISPRS_CIPA.pdf:

Last edited by Arx1meD; 31st March 2022 at 17:13.
Arx1meD is offline   Reply With Quote
Old 31st March 2022, 20:43   #13  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,450
Interesting find Arx1meD. Looking around I found another implementation in Python: The color transformations there are consistent with what is used in the pdf.

Edit:

I found this page: Paralenz color correction algorithm and it produces a decent looking result for the example image.



The GitHub page describes the process:

Quote:
The algorithm

Calculate the average color for image.
Hueshift the colors into the red channel until a minimum red average value of 60.
Create RGB histogram with new red color.
Find low threshold level and high threshold level.
Normalize array so threshold level is equal to 0 and threshold high is equal to 255.
Create color filter matrix based on the new values.
Maybe something that can already be done in AviSynth?

Last edited by Reel.Deel; 1st April 2022 at 09:04.
Reel.Deel is offline   Reply With Quote
Old 1st April 2022, 14:27   #14  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 227
IMO it is worse, gives blue tint.
richardpl is offline   Reply With Quote
Old 1st April 2022, 15:11   #15  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,983
Quote:
Originally Posted by Reel.Deel View Post
Interesting find Arx1meD. Looking around I found another implementation in Python: The color transformations there are consistent with what is used in the pdf.
Were you able to reproduce the paper results or those in the 1st post ?

I ran the python script and was not able to reproduce the paper results using "KdoTPHuv_o.jpg" as input

"--lab_constrast_correction equalisation" mode came the closest but there was still channel clipping, and playing with --constrast_cut_off values did not help

Code:
parser.add_argument('--colour_correction', default="LAB", help="perform colour correction in [LAB, CIELAB, LUV, YCBC] spaces")
parser.add_argument('--lab_constrast_correction', default="none", help="perform contrast correction with [stretching, CLAHE, equalisation, none] methods")
parser.add_argument('--constrast_cut_off', default=1e-3, help="cut-off level for lab contrast correction")


Quote:
Originally Posted by richardpl View Post
IMO it is worse, gives blue tint.
paralenz -
Yes, and red channel looks damaged or missing (pinkish splotch below fish becomes blue) .

Maybe some of the settings can be altered to reduce the channel loss ?

Last edited by poisondeathray; 1st April 2022 at 15:15.
poisondeathray is offline   Reply With Quote
Old 1st April 2022, 15:31   #16  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,450
Quote:
Originally Posted by poisondeathray View Post
Were you able to reproduce the paper results or those in the 1st post ?

I ran the python script and was not able to reproduce the paper results using "KdoTPHuv_o.jpg" as input

"--lab_constrast_correction equalisation" mode came the closest but there was still channel clipping, and playing with --constrast_cut_off values did not help
I did not, I was just point out another implementation that used the same color transformation as the paper. Interesting that you couldn't get the same result as what the paper shows

Quote:
Originally Posted by poisondeathray View Post
paralenz -
Yes, and red channel looks damaged or missing (pinkish splotch below fish becomes blue) .

Maybe some of the settings can be altered to reduce the channel loss ?
I did not play around with the settings, I just set the gain to the highest level and saved the image. The author mentioned there are some images that are problematic.

https://github.com/nikolajbech/under...ction/issues/1
Reel.Deel is offline   Reply With Quote
Old 1st April 2022, 15:38   #17  |  Link
anton_foy
Registered User
 
Join Date: Dec 2005
Location: Sweden
Posts: 456
Quote:
Originally Posted by Reel.Deel View Post
I did not play around with the settings, I just set the gain to the highest level and saved the image. The author mentioned there are some images that are problematic.

https://github.com/nikolajbech/under...ction/issues/1
Actually only one image, the "Lena" image
But seriously I am curious to see how this handles skintones, that is what color analog film was all about.
anton_foy is offline   Reply With Quote
Old 1st April 2022, 15:43   #18  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,706
https://github.com/Asd-g/AviSynthPlus-grayworld
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 1st April 2022, 16:09   #19  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,983
Quote:
Originally Posted by ChaosKing View Post
This was mentioned in post #8

There are some differences between the ffmpeg version, avs port vs. the paper results
poisondeathray is offline   Reply With Quote
Old 11th April 2022, 00:05   #20  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 1,252
Latest ver
https://github.com/Asd-g/AviSynthPlu...world/releases
kedautinh12 is offline   Reply With Quote
Reply

Tags
ffmpeg, filter, grayworld

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.