Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th June 2011, 01:57   #1  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Runtime filtering

Im trying to execute the next code, aimed to feed denoising values upon motion, but Im running in some syntax issues about input types, quotes, etc

Ideally I would write this in a single line, but spliced it to get things working first and then optimizing.

The error I get is "h had the wrong type":

Code:
v=TemporalSoften(3,255,255,0,2)
h=scriptclip("""string(YDifferenceFromPrevious(v)/10+0.5,"%1.2f")""")
s=string(tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h=h,a=2.0,sse=false))
filtered1=MT("Dither1Pre(flt="+s+",stacked=true)",2,2,true)

Last edited by Dogway; 29th June 2011 at 07:59.
Dogway is offline   Reply With Quote
Old 24th June 2011, 07:48   #2  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 617
You have to put everything depending on h in the ScriptClip part. Also, from the doc:
Quote:
"Restrictions" - the output of the script MUST be exactly like the clip delivered to ScriptClip (same colorspace, width and height).
Dither1Pre() doubles the clip height so it won't work. This one should be OK:
Code:
v = TemporalSoften(3,255,255,0,2)
filtered1 = Dither_convert_8_to_16 ().ScriptClip ("""
	Dither_get_msb ()
	h = string(YDifferenceFromPrevious(v)/10+0.5,"%1.2f")
	s = "tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h="+h+",a=2.0,sse=false)"
	MT("Dither1Pre(flt=s,stacked=true)",2,2,true)
""")
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r22 for Vapoursynth | trimx264opt segmented encoding

Last edited by cretindesalpes; 24th June 2011 at 07:51.
cretindesalpes is offline   Reply With Quote
Old 24th June 2011, 09:59   #3  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Using MT() inside ScriptClip might not give any speed advantage since its startup overheads (thread creation, etc) will be incurred on every frame. Try it with and without MT() and see what difference you get.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 24th June 2011, 14:41   #4  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
neuron2: Its hard to understand the behaviour, I know that the output of scriptclip is not a float, so you are just rephrasing my question. Any way to output float using YDifferenceFromPrevious? Do you know?

Thanks a lot cretindesalpes, my logic didn't think in that way, I thought that getting values was a separate thing and stored in a variable in float changing at every frame. Not able to use MT() is going to slow down things a lot! Regarding input=output maybe I can switch Dither_convert_8_to_16 to stackvertical(last,last), looking at the functions it seems to do the same. Same for Dither_get_msb. Well, its only to dont mix concepts...
Dogway is offline   Reply With Quote
Old 24th June 2011, 16:02   #5  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Quote:
Originally Posted by Dogway View Post
Any way to output float using YDifferenceFromPrevious?
...
I thought that getting values was a separate thing and stored in a variable in float changing at every frame.
The only thing that (potentially) changes on every frame are variables in the code executed by ScriptClip (or whatever run-time filter). Variables in the main script are set just once, at compile-time.

YDifferenceFromPrevious does return a float, but its value can only be used inside the run-time environment. The only way of getting it 'out' is to write it to a file with WriteFile or encode it into the output clip in some way.

You should read the following pages to understand more:
http://avisynth.org/mediawiki/Runtime_environment
http://avisynth.org/mediawiki/The_sc...xecution_model
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 24th June 2011, 20:18   #6  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Thanks for the links, overall I have the theory, but glue it all together and make it work is what I fail at. Getting it out with WriteFile isn't going to help me in anything (related to MT) if in the end I need to feed it inside a runtime function.
Dogway is offline   Reply With Quote
Old 24th June 2011, 21:45   #7  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
You have to read all the relevant documentation several times before you get the full picture. Slowly it will become clearer.

I wasn't actually suggesting Writefile as a solution to your problem - just pointing out it's one of the few ways to get data out of the run-time environment.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 27th June 2011, 02:11   #8  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Quote:
Originally Posted by Gavino View Post
I wasn't actually suggesting Writefile as a solution...
It only was a redundant question to neuron2.

I have read those pages around 10 or 15 times each. I just haven't done a thesis but learned very good tips and tricks. Im not going to take your smiley as a hint for a possible solution because I dont think there is one : D

Still one of my main questions is unresolved: can cretindesalpes code "h" and "s" variables be nested inside the Dither1pre function? or isn't there a way in avisynth to do this that doesn't affect performance?
Dogway is offline   Reply With Quote
Old 27th June 2011, 10:03   #9  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Quote:
Originally Posted by Dogway View Post
can cretindesalpes code "h" and "s" variables be nested inside the Dither1pre function? or isn't there a way in avisynth to do this that doesn't affect performance?
Since "h" is derived from YDifferenceFromPrevious(), it varies from one frame to another, and so can only be evaluated inside ScriptClip (or one of its friends). Therefore (since, as I said, you can't get values out of the run-time environment) anything that uses "h" must also be inside ScriptClip.

However, this might work:
v=TemporalSoften(3,255,255,0,2)
s="""scriptclip("tnlmeans(ax=3,ay=3,az=1,sx=2,sy=2,bx=0,by=0,h=YDifferenceFromPrevious(v)/10+0.5,a=2.0,sse=false)")"""
filtered1=MT("Dither1Pre(flt=s,stacked=true)",2,2,true)

I'm not sure if ScriptClip works within MT (I suspect not), so you might have to remove the MT bit. But at least Dither1Pre would then be outside ScriptClip.
__________________
GScript and GRunT - complex Avisynth scripting made easier

Last edited by Gavino; 27th June 2011 at 10:19.
Gavino is offline   Reply With Quote
Old 29th June 2011, 04:42   #10  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Indeed MT didn't work, but I see that the only workaround for (nested) quotes is to take them out as variables. Since it was going to be so damn slow I chose dfttest with mdegrain temporal, both biased upon motion, very neat. I hope one day tritical makes a more optimized tnlmeans, because still I don't consider NLmeans reliable enough...
Also you can see how I "prepared" the source for motion evaluation, I dont know if there is a more proper way probably with scene detection, only asking just in case...

Code:
...
v=TemporalSoften(3,255,255,0,2)
d=stackvertical(last,last).scriptclip("""
h1 = round(300/(YDifferenceFromPrevious(v)/0.5+0.5)+250)
h2 = YDifferenceFromPrevious(v)/1+1
d=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=h1,plane=0,lsb=true)
d.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")
...
edit: actually this fails miserably in an encoding test. Memory issues I think

Last edited by Dogway; 29th June 2011 at 06:18.
Dogway is offline   Reply With Quote
Old 29th June 2011, 09:40   #11  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Quote:
Originally Posted by Dogway View Post
Code:
...
d=stackvertical(last,last).scriptclip("""
...
d=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=h1,plane=0,lsb=true)
d.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")
...
That script won't work as ScriptClip must return a clip with the same dimensions as its input (see post #2), and the crop is changing the dimensions.

Also, the use of 'd' both inside and outside the ScriptClip is confusing. Is that intentional?
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 29th June 2011, 09:53   #12  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
No it wasn't. I can fix it.
Im aware that Im cropping input by half, but mdegrain3 is doubling the output height again. Let me test

Tested but it crashes again, maybe its too much for scriptclip?
Dogway is offline   Reply With Quote
Old 29th June 2011, 10:23   #13  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Quote:
Originally Posted by Dogway View Post
mdegrain3 is doubling the output height again.
Are you sure? I wasn't aware that mdegrain3 could change the height.
What does asuper refer to? (not shown in your post)
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 29th June 2011, 10:29   #14  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Quote:
Originally Posted by Gavino View Post
I wasn't aware that mdegrain3 could change the height.
it's the mod version of mvtools2 from the Dither pack.

I post the relevant code, I also tested without mrecalculate which is not sctrictly necessary, but note this code works when previewing single frames in avspmod.

Code:
blksize=8
overlap=4
asuper=MSuper(pel=2,sharp=2,hpad=blksize, vpad=blksize,chroma=false)
rsuper=MSuper(pel=2,sharp=2,hpad=blksize, vpad=blksize,levels=1,chroma=false)
b3v = MAnalyse(asuper, isb=true, delta=3, blksize=blksize, overlap=overlap,chroma=false)
b2v = MAnalyse(asuper, isb=true, delta=2, blksize=blksize, overlap=overlap,chroma=false)
b1v = MAnalyse(asuper, isb=true, delta=1, blksize=blksize, overlap=overlap,chroma=false)
f1v = MAnalyse(asuper, delta=1, blksize=blksize, overlap=overlap,chroma=false)
f2v = MAnalyse(asuper, delta=2, blksize=blksize, overlap=overlap,chroma=false)
f3v = MAnalyse(asuper, delta=3, blksize=blksize, overlap=overlap,chroma=false)

v=TemporalSoften(3,255,255,0,2)
d=stackvertical(last,last).scriptclip("""
h=YDifferenceFromPrevious(v)
h1 = 300/(h/0.5+0.5)+250
h2 = h+1
b3v = MRecalculate(rsuper, b3v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
b2v = MRecalculate(rsuper, b2v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
b1v = MRecalculate(rsuper, b1v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f1v = MRecalculate(rsuper, f1v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f2v = MRecalculate(rsuper, f2v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
f3v = MRecalculate(rsuper, f3v,overlap=overlap/2,blksize=blksize/2, thSAD=round(h1/2),chroma=false)
t=crop(0,0,0,-478).MDeGrain3(asuper, b1v, f1v, b2v, f2v, b3v, f3v, thSAD=round(h1),plane=0,lsb=true)
t.dfttest(sigma=h2,tbsize=1,lsb_in=true,lsb=true)""")

Last edited by Dogway; 29th June 2011 at 10:38.
Dogway is offline   Reply With Quote
Old 29th June 2011, 10:31   #15  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 617
You could also do almost all the processing out of ScriptClip() with a limited set of values (say 3 per filter: low, medium, high) and use the runtime functions only to select/interpolate the results (or maybe with just ConditionalFilter()).
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r22 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 29th June 2011, 10:37   #16  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Yes, I saw the examples in the help... if prefer this gradual way, but if it can't be I will try that out, using a thresholding for low/medium/high (not very enthusiastic but...). Do you think there is a reason using Dither's mdegrain3+dfttest is too much for scriptclip?

edit: Also this is a dumb question but as you see I switched the "dither_*" lines with stackvertical and crop, Im not sure at all if this has any impact on the performance because maybe it's only processed at the parsing phase(?)

Last edited by Dogway; 29th June 2011 at 11:27.
Dogway is offline   Reply With Quote
Old 29th June 2011, 14:24   #17  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,405
Quote:
Originally Posted by Dogway View Post
Do you think there is a reason using Dither's mdegrain3+dfttest is too much for scriptclip?
Do you get problems immediately or only after rendering a lot of frames?
cretindesalpes may have more inside knowledge.

Quote:
Im not sure at all if this has any impact on the performance because maybe it's only processed at the parsing phase(?)
At the parsing stage, everything inside ScriptClip is just a string, so basically no processing takes place.
At run-time, this string is parsed afresh for every frame served, instantiating filters to create an inner filter chain from which a frame is fetched.
__________________
GScript and GRunT - complex Avisynth scripting made easier

Last edited by Gavino; 29th June 2011 at 14:27.
Gavino is offline   Reply With Quote
Old 29th June 2011, 21:16   #18  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
maybe after rendering 20 frames. Ram usage starts to go up and up, and crashes around 2.2Gb. If I remove Mrecalculate it crashes faster with ram steady at around 1.3Gb
Vdub log:

Code:
Crash reason: Unhandled Microsoft C++ Exception

Thread call stack:
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c921028: ntdll!wcsncpy [7c910000+1057f+aa9]
7c921086: ntdll!wcsncpy [7c910000+1057f+b07]
7c9201db: ntdll!RtlAllocateHeap [7c910000+100c4+117]
7c812afb: kernel32!RaiseException [7c800000+12aa9+52]
7c359aed: MSVCR71!_CxxThrowException [7c340000+19ab9+34]
10001972: AviSynth!00001972
100038e2: AviSynth!000038e2
10009b33: AviSynth!avs_release_value [10000000+75d0+2563]
014a1f45: MT!00001f45
7c80bb64: kernel32!lstrcmpi [7c800000+bb41+23]
01605541: TNLMeans!00005541
7c920041: ntdll!RtlFreeHeap [7c910000+ff2d+114]
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
10005e4f: AviSynth!00005e4f
7c9200b8: ntdll!RtlFreeHeap [7c910000+ff2d+18b]
7c920041: ntdll!RtlFreeHeap [7c910000+ff2d+114]
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
01608cfb: TNLMeans!_AvisynthPluginInit2@4 [01600000+8bb0+14b]
10009037: AviSynth!avs_release_value [10000000+75d0+1a67]
10009443: AviSynth!avs_release_value [10000000+75d0+1e73]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
014a1e53: MT!00001e53
7c91da1a: ntdll!NtRegisterThreadTerminatePort [7c910000+da0e+c]
7c80b729: kernel32!GetModuleFileNameA [7c800000+b56f+1ba]
Tested without the setmtmode(2,2) line, just left setmtmode(5,2) at the very first and it worked...

Last edited by Dogway; 29th June 2011 at 21:22.
Dogway is offline   Reply With Quote
Old 29th June 2011, 23:01   #19  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 617
I just did a quick test and observed that MRecalculate inside ScriptClip makes the memory usage increase linearly until it crashes (-> 2 GB memory barrier); the destructor must have a memory leak or something. Dfttest too, but much slower. MDegrain3 looks OK.

Maybe other functions have the same issue.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r22 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 30th June 2011, 02:20   #20  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,020
Yes, another test without setmtmode(2,2) but with mrecalculate also crashed, just later than using multithreading. Maybe my test with no multithreading no mrecalculate worked because I only tested a small section of 400 frames, not long enough to let scriptclip crash.
Dogway is offline   Reply With Quote
Reply

Tags
motion, quotes, syntax, types

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.