Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th January 2019, 08:27   #181  |  Link
Wolfberry
Helenium(Easter)
 
Wolfberry's Avatar
 
Join Date: Aug 2017
Location: Hsinchu, Taiwan
Posts: 65
From vapoursynth.com/doc/pythonreference.html#get_core
Quote:
core
Gets the singleton Core object. If it is the first time the function is called, the Core will be instantiated with the default options. This is the preferred way to reference the core.

get_core([threads = 0, add_cache = True])
Deprecated, use the core attribute instead.

Get the singleton Core object. If it is the first time the function is called, the Core will be instantiated with the given options. If the Core has already been instantiated, all options are ignored. Setting threads to a value greater than zero overrides the autodetection.
It's not recommended to use get_core as it is deprecated.
Just use vs.core instead of vs.get_core() (also simpler).
Alternatively, you can use "from vapoursynth import core".
__________________
RAC#1-6

Last edited by Wolfberry; 4th January 2019 at 08:30.
Wolfberry is offline   Reply With Quote
Old 4th January 2019, 10:21   #182  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 541
Thx, I updated it in my post. One line less now :-)
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 4th January 2019, 10:27   #183  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,497
Quote:
Originally Posted by zorr View Post
I haven't looked at Vapoursynth yet. I know almost nothing about it so it's hard to say how much effort it would take. Can you recommend a tutorial / site to get one started with VapourSynth?
As others have posted, it's quite easy to grasp once you get the basics of Python code structure.

This one is something that could provide a nice enhancement for VS-based optimization compared to SSIM:
http://forum.doom9.org/showthread.php?t=175862.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 4th January 2019, 16:14   #184  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 986
Based on some tests I guess prefiltering is a gimmick lol, aside so many filtering conventions sticked to our brains for so long (like blocksize 16 for 1080p). Thanks for the tool it takes some of the voodoo-guessing out of the equation. I think implementing VMAF is worth here, after reading the Netflix VMAF article a few weeks ago it seemed that SSIM did some bad assumptions, in my case contrasharpening was defaulting to 0 or false which in my opinion wasn't making justice to what I observed. That or if your port it I might start using VapourSynth occasionally for VMAF and BM3D, I just don't like how it requires you to write more, and script readability. Also something for x264 like SSIM Plus would be cool if someone knows about a free open source alternative.

Last edited by Dogway; 4th January 2019 at 16:42.
Dogway is offline   Reply With Quote
Old 5th January 2019, 01:09   #185  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Thanks @ChaosKing and @StainlessS, I have now read the Python tutorial, the official VapourSynth site and ChaosKing's FATPACK thread. I found a SSIM plugin for VapourSynth as well.

Is there a plugin for measuring runtime (preferably per frame)?

I suppose Python's standard I/O routines can be used to write stuff to a file during the script evaluation?
zorr is offline   Reply With Quote
Old 5th January 2019, 03:48   #186  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,304
Maybe Groucho has better code for timing stuff.

But some stuff from RT_Stats below. [EDIT: Use as you will]

Code:
AVSValue __cdecl RT_LocalTimeString(AVSValue args, void* user_data, IScriptEnvironment* env) {
    const bool file = args[0].AsBool(true);
    SYSTEMTIME  st = { 0 };
    if(file) {
        DWORD tick=GetTickCount();
        while(GetTickCount()==tick) // Wait until system clock goes TICK (prevent two separate calls returning same time)
            Sleep(0);
    }
    GetLocalTime(&st);
    char bf[64];
    if(file) {
        sprintf(bf,"%4d%02d%02d_%02d%02d%02d_%03d",st.wYear,st.wMonth,st.wDay,st.wHour,st.wMinute,st.wSecond,st.wMilliseconds);
    } else {
        sprintf(bf,"%4d-%02d-%02d %02d:%02d:%02d.%03d",st.wYear,st.wMonth,st.wDay,st.wHour,st.wMinute,st.wSecond,st.wMilliseconds);
    }
    return env->SaveString(bf);
}
Quote:
RT_LocalTimeString(Bool "file"=True)
Returns current local time as a string.
Where digits Y=Year, M=Month, D=Day, H=Hour, M=Minute, S=Second, m=millisecond.
When bool file==False, then string in format "YYYY-MM-DD HH:MM:SS.mmm"
When bool file==True (Default) string is in format "YYYYMMDD_HHMMSS_mmm"
Also when file==True, function first waits until the system tick count is incremented (about every 10ms)
before inquiring system time. This is to prevent 2 consecutive calls returning the same time string.
Perhaps useful for temporary filename generation.
Code:
AVSValue __cdecl RT_Timer(AVSValue args, void* user_data, IScriptEnvironment* env) {
    return  double(clock()) / double(CLOCKS_PER_SEC);
}


double __cdecl RT_TimerHP_Lo(void) {
    LARGE_INTEGER liPerfCounter = {0,0};
    LARGE_INTEGER liPerfFreq    = {0,0};
    bool bStat              = true;
    DWORD_PTR dwpOldMask    = SetThreadAffinityMask(GetCurrentThread(), 0x01);
    Sleep(0);
    if ((QueryPerformanceFrequency(&liPerfFreq) == 0) || (QueryPerformanceCounter(&liPerfCounter) == 0))
        bStat = false;
    double tim;
    if(!bStat) { // High precision NOT available.
        static clock_t first_time32=0;
        static bool done32=false;
        const clock_t t = clock();
        if(!done32) {
            first_time32 = t;
            done32=true;
        }
        tim = double(t-first_time32) / double(CLOCKS_PER_SEC);          // Fallback, low rez timer
    } else { // High precision IS available.
        static __int64 first_time64=0;
        static bool done64 = false;
        if(!done64) {
            first_time64 = liPerfCounter.QuadPart;
            done64=true;
        }
        tim = (double)(liPerfCounter.QuadPart-first_time64) / (double)liPerfFreq.QuadPart;
    }
    SetThreadAffinityMask(GetCurrentThread(), dwpOldMask);
    Sleep(0);
    return tim;                                                 // return double for C client
}


AVSValue __cdecl RT_TimerHP(AVSValue args, void* user_data, IScriptEnvironment* env) {
    return RT_TimerHP_Lo();                                                  // Implicit type conversion to AVSValue float
}
Quote:
RT_Timer()
Returns time in seconds since start of process.
Usage:
s=RT_Timer()
Test()
e=RT_Timer()
Str=RT_String("Test Start=%.2f End=%.2f Time=%.2f secs",s,e,e-s)
SubTitle(str)
RT_Debug(str)

***
***
***

RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elapsed in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only thing that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str=RT_String("Test Start=%.3f End=%.3f Time=%.6f secs",s,e,e-s)
SubTitle(str)
High precision timer quite often called HPET in bios setup, usually defaults to OFF. [EDIT: Presumably, High Precision Event Timer]

EDIT:
Code:
env->AddFunction("RT_LocalTimeString",         "[file]b",RT_LocalTimeString, 0);
env->AddFunction("RT_Timer",                   "",RT_Timer, 0);
env->AddFunction("RT_TimerHP",                 "",RT_TimerHP, 0);
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th January 2019 at 04:01.
StainlessS is offline   Reply With Quote
Old 5th January 2019, 05:06   #187  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 4,033
Quote:
Originally Posted by StainlessS View Post
Maybe Groucho has better code for timing stuff.
Your 'RT_TimerHP_Lo' function looks pretty good in terms of precision and resolution (and somewhat familiar ).

Edit: Déjà vu...

Last edited by Groucho2004; 5th January 2019 at 05:11.
Groucho2004 is offline   Reply With Quote
Old 5th January 2019, 06:00   #188  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,304
Quote:
Originally Posted by Groucho2004 View Post
Your 'RT_TimerHP_Lo' function looks pretty good in terms of precision and resolution (and somewhat familiar ).

Edit: Déjà vu...
Yep, If's I recalls correctly, me robbed some of that from your PM (I was wantin' to keep that a secret )
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th January 2019 at 06:56.
StainlessS is offline   Reply With Quote
Old 5th January 2019, 12:28   #189  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 4,033
Quote:
Originally Posted by StainlessS View Post
Yep, If's I recalls correctly, me robbed some of that from your PM (I was wantin' to keep that a secret )
Ah, yes. Alzheimer slowly kicking in.
Groucho2004 is offline   Reply With Quote
Old 6th January 2019, 01:05   #190  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Quote:
Originally Posted by StainlessS View Post
But some stuff from RT_Stats below. [EDIT: Use as you will]
Thanks, I forgot to mention I need it in a VapourSynth script. I guess this VapourSynth discussion is a bit off topic here so I made a new thread in the VapourSynth section.
zorr is offline   Reply With Quote
Old 16th January 2019, 14:15   #191  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 986
I wasn't convinced with leaving sharpness at 0, I run some comparisons with VMAF and other algorithms in MSU VQMT and indeed SSIM (and PSNR) are the only ones that favor blurry filters.
Attached Images
 
Dogway is offline   Reply With Quote
Old 17th January 2019, 17:43   #192  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 87
So my assumptions during MFlow testing were correct, the metricts favor blur, hence the sharp=0 (and possibly pel=4) bias. Maybe you could try downscaling before calculation as well.


zorr, is there a length limit for the combination of filters and limits for a single parameter?
Whenever I add more params to dependencies than I have here

Quote:
Clevel = 8/10.0 # optimize Clevel = _n_/10.0 | 0..10 ; min:super_pel2 1 > 2 0 ? super_sharp2 0 > 2 0 ? super_rfilter2 0 > 2 0 ? blockSize2 4 > 2 0 ? searchAlgo2 0 > 2 0 ? searchRange2 1 > 2 0 ? searchRangeFinest2 1 > 2 0 ? divide2 0 > 2 0 ? overlap2 0 > 2 0 ? CdoRecalc true == 2 0 ? truemotion2 true == 2 0 ? max | Clevel
I get "Unknown element badSA2" (I was adding badSAD2 after truemotion2, strange how it's skipping a letter while keeping the digit instead of just cutting the name).
Same here, add CRscaleCSAD2 to
Quote:
CdoRecalc=true # optimize CdoRecalc=_n_ | false,true ; filter:Clevel 1 > Csmooth 0 > CRsearchAlgo 0 > CRsearchRange 1 > CRdivide 0 > CRoverlap 0 > Cthsad 0 > CRtruemotion true == CRmeander true == x false == or | CdoRecalc
and get "Unknown element CRscaleCSA"
Seedmanc is offline   Reply With Quote
Old 17th January 2019, 21:56   #193  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Quote:
Originally Posted by Dogway View Post
I wasn't convinced with leaving sharpness at 0, I run some comparisons with VMAF and other algorithms in MSU VQMT and indeed SSIM (and PSNR) are the only ones that favor blurry filters.
That's a great find and good to know. This screenshot from the MSU page shows per pixel SSIM values and the resulting image is clearly blurry.



Also the SSIM index for the pixel is calculated using a 8x8 neighbourhood (fast method) or using a gaussian weighting function (high quality method). That explains why it's not able to "see" fine details.

I'm running some Vapoursynth tests now and there we also have GMSD which gave much better results than SSIM in my denoising test (best SSIM result had ringing artifacts). I haven't tried VMAF yet.

It would be great to have these quality metrics as Avisynth plugins too.
zorr is offline   Reply With Quote
Old 17th January 2019, 22:04   #194  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Quote:
Originally Posted by Seedmanc View Post
So my assumptions during MFlow testing were correct, the metricts favor blur, hence the sharp=0 (and possibly pel=4) bias. Maybe you could try downscaling before calculation as well.
Yes you were correct. But I think that to counter a blurry quality measurement you should upscale and not downscale.

Quote:
Originally Posted by Seedmanc View Post
zorr, is there a length limit for the combination of filters and limits for a single parameter?
There shouldn't be, I will take a look. Just curious though, what is that filter doing?

[EDIT] I tried the Clevel = ... line and it worked. But there are some small fixes in the parser code which are not released yet, so maybe only my development version accepts it. You should check that there are no line breaks anywhere, the optimizer doesn't support continuing lines with \ like Avisynth does.

Oh and if I'm interpreting it correctly you're trying to take the maximum of many numbers. That's not going to work with only one max function since it only takes the top two numbers from the stack.

Last edited by zorr; 17th January 2019 at 22:30.
zorr is offline   Reply With Quote
Old 18th January 2019, 10:49   #195  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 87
It's just a part of my ever-increasing MFlow script that now does the initial upsample, follows by the MCompensate trick and then tries to add recalculation to it (over 40 params now). However it always chooses not to do the latter part.
Since I have all the params of MCompensate tied to the weight at which I overlay the MCompensated version over the original one (so as not to try combination when the weight is under 0.2) I tried to follow the no-bias rule and tie the weight itself back to them. Same with the doRecalc switch later.

But the fact that max doesn't work with more than 2 arguments is a disappointing discovery, that means I won't be able to do the many-to-one parameter tying. Is the situation the same for the OR operator which I'm using to combine multiple boolean values?
Seedmanc is offline   Reply With Quote
Old 18th January 2019, 15:11   #196  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 55
Quote:
Originally Posted by Dogway View Post
I wasn't convinced with leaving sharpness at 0, I run some comparisons with VMAF and other algorithms in MSU VQMT and indeed SSIM (and PSNR) are the only ones that favor blurry filters.
Try MSU Blurring. It will show not only blur but also sharp.

Using MSU_Blurring can improve SSIM:
(SSIM+SSIM*BLUR)/(SSIM+1)

BLUR - if encode_blur < source_blur then (encode_blur / source_blur)
BLUR - if encode_blur > source_blur then (source_blur / encode_blur)

UPD: Fix calculation of the value, relative to SSIM. The lower the SSIM, the lower the effect of the blur on the final value. Example:
SSIM=0.9 BLUR=0.5 - Result=0.7105
SSIM=0.5 BLUR=0.9 - Result=0.6333
Previously, in both situations it was 0.7

p.s. There is also a B-SSIM metric that fixes SSIM problems for blurry images.

Fig.b SSIM = 0.8241, B-SSIM = 0.6434
Fig.c SSIM = 0.7276, B-SSIM = 0.4526

Last edited by zub35; 20th January 2019 at 12:33.
zub35 is offline   Reply With Quote
Old 18th January 2019, 21:43   #197  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Quote:
Originally Posted by Seedmanc View Post
It's just a part of my ever-increasing MFlow script that now does the initial upsample, follows by the MCompensate trick and then tries to add recalculation to it (over 40 params now). However it always chooses not to do the latter part.
It's possible that you've hit the curse of dimensionality. Every new parameter multiplies the number of possible combinations. At some point the search space gets so ridiculously large that no algorithm can find good solutions. Or it could be something else.

I tried doing MFlow + MRecalculate at the same time and there MRecalculate brought some extra quality. I haven't tried all three at the same time (and my method of combining MCompensate is different than yours).

I also tried MFlow + MCompensate in phases, I first optimized MFlow only and once I had optimal values I "freezed" them. I then added the MCompensate and only optimized its parameters. That way the number parameters stays lower and the search is easier. It might however result in little less optimal result than what could be possible doing them both at the same time.

Quote:
Originally Posted by Seedmanc View Post
I tried to follow the no-bias rule and tie the weight itself back to them.
I'm not actually sure if it's less biased to set some default values to MCompensate when it's not used because then those default values appear more often than others. You could try running with and without "no bias" and see which one gives better results.

Quote:
Originally Posted by Seedmanc View Post
But the fact that max doesn't work with more than 2 arguments is a disappointing discovery, that means I won't be able to do the many-to-one parameter tying. Is the situation the same for the OR operator which I'm using to combine multiple boolean values?
Yes it's the same with OR and all the other functions except "?" which takes three arguments. That's the way this reverse polish notation works. But it's not a limitation of reverse polish notation, it can do anything the infix form can and do it without parenthesis (which is the reason it's used, it's so easy to parse).

So if you need the max of say, three values, you can do it this way:
max(a,b,c) == max(a, max(b,c))

In rpn it's even simpler: a b c max max. So you just add (n-1) max functions to max n numbers. The same works with OR as well.
zorr is offline   Reply With Quote
Old 18th January 2019, 23:36   #198  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
Quote:
Originally Posted by zub35 View Post
Try MSU Blurring. It will show not only blur but also sharp.

Using MSU_Blurring can improve SSIM:
(SSIM + MSU_Blurring*) / 2
* if encode_blur < source_blur then [encode_blur / source_blur]
* if encode_blur > source_blur then [source_blur / encode_blur]
That's an interesting approach, do you have link for more detailed explanation?
You could also calculate the blurriness and optimize for both SSIM and blurriness. The pareto front will then have results with increasing SSIM and blurriness and you can choose which one looks best.

Of course we'd need a blurriness detection in Avisynth to do that. I found a promising method which is simple to implement and should give good results: variance of the laplacian. The laplacian filter is easy to calculate with MaskTools and StainlessS's RT_YPlaneStdev can be used to calculate the variance.

Quote:
Originally Posted by zub35 View Post
p.s. There is also a B-SSIM metric that fixes SSIM problems for blurry images.
I took a look at B-SSIM and looks like it's using almost the same method as the "variance of the laplacian", it just replaces laplacian with sobel and takes the standard deviation instead of variance. The SSIM value is then multiplied by (2*SIf*SIh)/(SIf^2 + SIh^2) where SIf is the stddev(sobel) of original frame and SIh is the same for the modified frame. Looks like that can also be implemented in a simple script.

[EDIT] Just remembered that SSIM value can be negative, the valid range is -1 .. 1. So by multiplying a really bad SSIM with < 1 you can actually make the metric better.

Last edited by zorr; 18th January 2019 at 23:44.
zorr is offline   Reply With Quote
Old 19th January 2019, 22:58   #199  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 175
B-SSIM seemed too intriguing to pass so I went ahead and implemented it. Or at least a variation of it. The original uses the Sobel filter for edge detection, I'm using the Scharr operator instead because it's more accurate. And by the way the "sobel" in MaskTools is not actually sobel, it detects edges but the mask is incorrect and gives sub-par results.

Other difference is that I'm not taking the maximum of standard deviations from the whole clip sample, I just take one from each frame. Taking the maximum would need a two pass algorithm and I wanted to avoid that. The difference to the original shouldn't be that large.

In my tests this seems to work as it should giving lower scores to blurrier frames.

Anyway, here's a sample code to compare a blurred clip to the original, showing both SSIM and B-SSIM(mod).

Code:
# source clip
AVISource("d:\process2\1 deinterlaced.avi")

ConvertToYV16() # for MaskTools
orig = last

blurred = orig.GaussianBlur(1.5)

scharr_orig = scharr(orig)
scharr_blurred = scharr(blurred)

final = ScriptClip(last, """
	global ssim = SSIM_FRAME(orig, blurred)
	bssim = BSSIM_MOD(scharr_orig, scharr_blurred, ssim)	
	return orig.SubTitle("SSIM "+String(ssim)+" BSSIM "+String(bssim))
""")	

return final

function scharr(clip c) {
	scharr_x = c.mt_edge("3 0 -3 10 0 -10 3 0 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
	scharr_y = c.mt_edge("3 10 3 0 0 0 -3 -10 -3", thY1 = 0, thY2 = 255, y=3, u=1, v=1)
	scharr = mt_lutxy(scharr_x, scharr_y, yexpr=mt_polish("((x*x)+(y*y))^0.5"), u=1, v=1)
	return scharr
}

function BSSIM_MOD(clip scharr_orig, clip scharr_alt, float ssim) {
	stddev_orig = RT_YPlaneStdev(scharr_orig)
	stddev_alt = RT_YPlaneStdev(scharr_alt)
	
	mul = (2*stddev_orig*stddev_alt) / (stddev_orig*stddev_orig + stddev_alt*stddev_alt)
	return mul*ssim
}
The implementation needs StainlessS's RT_Stats plugin for the standard deviation.

Seedmanc and Dogway, can you try the B-SSIM variation and see if it fixes the blurring issue?

Last edited by zorr; 19th January 2019 at 23:03.
zorr is offline   Reply With Quote
Old 29th January 2019, 01:39   #200  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 541
An additional "toy" for comparing video qualitiy: https://github.com/fdar0536/vapoursynth-butteraugli
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.