Avisynth+

[MysteryX]

I was thinking about the issue with "last" and global vars.

"last" isn't used within script functions so script libraries won't be affected by any issue here. If there are problems, it's going to happen at the main script level. If it works within script functions (without implicit "last"), then it's a huge improvement already. Once that is working, we can give it some more thoughts about how to handle variables like "last", as well as produce bogus scripts for testing. I think we're better to leave that one for later. If we're going to change it, we'll have to do it right, and for now we have no right solution.

Still, that doesn't prevent conditional functions from being supported.

[raffriff42]

Originally Posted by MysteryX

"last" isn't used within script functions so script libraries won't be affected by any issue here.

What what what!!!

[MysteryX]

Originally Posted by raffriff42

What what what!!!

Fct1() # uses implicit last -- possible issue with dynamic evaluation of expressions like Eval which reset "last"
Fct2()

function Fct1(clip c) {
   c = c.Fct2() # here we need to define the clip explicitely
   return Fct3(c)
}

[raffriff42]

I once told a guy -- can't find the link right now -- about the easy way to create a function from a bunch of script lines. Something like:

function foo(clip C)
{
    C ## Last==C
    Filter1
    Filter2
    Filter3
    return Last
}

...and some of my user functions look just like that example. All I'm saying is, don't even think about breaking my preciousss Last.

EDIT - I always use explicit Last with BlankClip and ScriptClip; maybe Eval should get special treatment (from the user) also.

[LigH]

Or so:

function Fct1(clip c) {
   c # means explicitly: last = c
   Fct2()
   return Fct3()
}

Of course, the scope in the function does not use the last clip from the main level inside. It creates an own last variable.

Fortunately, last is not "super-global". At least I hope so.

[MysteryX]

Right -- sorry my mistake.

We're not changing anything to "last". We're just worried it might have some of the same issues as "current_frame" with MT -- although we're not clear on how such problem manifests. We haven't seen any issue so far.

[StainlessS]

function Fct1(clip c) {
   c # means explicitly: last = c
   Fct2()
   x=sin(.5) 
   sin(.5) # EDIT: added
   return Last # return result of Fct2() # Assuming it was a clip
}

Last (so far as I understand it) stores the previous (clip only) return variable that was not assigned to anything else.

[LigH]

That's an interesting question. I really wonder if "return" returns a clip or a number. In my current mental state, I would bet on ... number. I should try it.

[pinterf]

That deadlock thing is hard to catch. I was working on the problem during December, now I have returned to it.

In my test code the variables last and current_frame were passed to the invoke with guards of mutex. That specific TLS (Thread Local Storage) Invoke - I had linked earlier which was doing context switch and made "current_frame" and "last" out of scope - was modified a bit to re-set those variables in the scope of the core. But this was only the first small step which only allowed the script not to fail in YDifferenceFromPrevious's constructor (which so far was complaining "current_frame" does not exists)

ScriptClip MT test script, which I'm using to ensure that last is always right.

Frames should follow each other displaying 16-32-48-96-16-32-48-96 sequence if frame order is OK.

There is a second ScriptClip line, because it makes our life shorter and increases the stress level a bit. We have another parallel work in runtime evaluation.

Case #1 happyness
When the second ScriptClip line is independent from the first ScriptClip(commented out c1=, c2= lines in code below), then there is no problem, I'm doing Prefetch(300) and nothing fails.

Case#2 deadlock
But when the second ScriptClip is using the output of the first ScriptClip, sometimes deadlock occurs.

Depending on the clip size, additional filters at the end, they all modify internal timings, order of cache hits, this "sometimes" can be never or 10 or 100 frames. I have experimented with a few hundred variants. Deadlock can be eliminated when ScriptClip is defined as "DONT_CACHE_ME", but this is no-go of course.

Sometimes Prefetch(80) is OK, but Prefetch(90) deadlocks. In other timing conditions the limit is Prefetch(8), etc.

len=1
w=64
h=48
c1=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$008080)
c2=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$108080) # diff 16
c3=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$308080) # diff 32
c4=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$608080) # diff 48 -> diff (-)96
c=c1+c2+c3+c4
c.Loop(100)
# this will deadlock sooner or later or never, more Prefetch makes deadlock to appear, even more -> deadlock sooner 
ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()))")
ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()),X=0,Y=30)")
/* This works for any Prefetch, no dependancy between them
c1=ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()))")
c2=ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()),X=0,Y=30)")
Stackvertical(c1,c2)
*/
ConvertToRGB32() #additional filters modify timing thus, frame evaluation order/cache hit order in MT
ConvertToRGB24()
ConvertToRGB32()
ConvertToRGB24()
Prefetch(60)

[MysteryX]

I tried doing a bit of multi-threading in C++ until I realized how much of a pain it was. I was reading many articles saying how MT was only for experts, and I then understood why.

My recommendation: write the thread management in .NET (no "undefined" state!!) but that's not really an option -- if rewriting was an option, it's VapourSynth all the way

so... Good luck

[StainlessS]

Originally Posted by LigH

That's an interesting question. I really wonder if "return" returns a clip or a number. In my current mental state, I would bet on ... number. I should try it.

function Fct2(clip c) {return c.info}

function Fct1(clip c) {
   c # means explicitly: last = c
   Fct2()
   z=c.Invert
   x=sin(.5) 
   sin(.5)
   return Last # return result of Fct2(), Colorbars with Info
}

Colorbars.killaudio

Fct1()

EDIT: Also, I would disagree with first comment, # means implicitly: last = c

[TheFluff]

Originally Posted by LigH

That's an interesting question. I really wonder if "return" returns a clip or a number. In my current mental state, I would bet on ... number. I should try it.

Last only works on clips (more specifically, every expression that isn't an explicit assignment and happens to evaluate to a clip assigns to last). It's a pretty unintuitive mechanism and I'd consider it a pretty awful hack in most languages but it is kinda excusable in a toy DSL like Avisynth script. Perl has something similar with $_, but at least it's more consistent about it and happily admits it's not really intended to write readable programs.

See here and the surrounding posts for an extensive earlier discussion on attempting to make it less weird. The tl;dr is that Avisynth script kinda needs an assignment operator that is actually a real operator rather than a special type of expression, but good luck with that.

[LigH]

Originally Posted by TheFluff

3) implicit assignments to last (all expressions that aren't one of the two first types and evaluate to a clip)

I see. So the statement "return last" is not ambiguous here. Rather the single "sin(.5)" without assignment is misleading.

Originally Posted by TheFluff

Perl has something similar with $_

Pascal has the reserved variable "Result" to explicitly assign a return value in the scope of a function.

[StainlessS]

Whoa, that surprised me when removing the 'Return Last' from Fct1(),

The script's return value was not a video clip, (Is a float, 0.479426).

EDIT: Nice post Fluffy.

[TheFluff]

Originally Posted by LigH

Pascal has the reserved variable "Result" to explicitly assign a return value in the scope of a function.

That's not the same thing at all. Pascal doesn't have a C-like "return <expression>" mechanism, so instead you assign to a special variable (either "result" or a variable with the same name as the function) to set the return value from a function.

$_ in Perl and last in Avisynth are special in that they act as a default parameter to many functions if no other suitable value is given. For example, in Perl saying "print;" is the same as saying "print($_);" much like how in Avisynth "blur()" is the same as "blur(last)". Since Perl is an imperative language, it doesn't need the implicit assignment part of the equation, though - function calls can just have side effects and mutate $_ with no problems (you still get implicit assignment to $_ when using it as an implicit iterator in loops though, but that's beside the point).

Avisynth script, however, is a quite poor excuse for a functional language that wants to pretend it is imperative, or perhaps more accurately, it's a schizophrenian language that has one data type that is purely functional and lazily evaluated, paired with a bunch of imperative scripting functionality that very much isn't. What you're writing in Avisynth script is a composition of functions (in other words, the filter chain), so from the script's point of view, clip variables are effectively immutable. Hence you get the hack that is the implicit assignment to last based on what the expression evaluates to - the clip data type is effectively living in a different programming language than everything else in Avisynth script and has its own semantics.

Arguably, the entire "assignment" part of clip variables is confusing, since you're not really assigning a value to a variable at all.

[MysteryX]

Originally Posted by pinterf

That deadlock thing is hard to catch. I was working on the problem during December, now I have returned to it.

I applied the latest ConditionalFilter code changes to the MT-pfmod branch, and this works.

Here's the x86 library of MT-pfmod with conditional filters, if anyone wants to play with it (no 16-bit stuff in this version).

len=1
w=64
h=48
c1=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$008080)
c2=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$108080) # diff 16
c3=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$308080) # diff 32
c4=BlankClip(width=w,height=h,length=len,pixel_type="YV24",color_yuv=$608080) # diff 48 -> diff (-)96
c=c1+c2+c3+c4
c.Loop(10000)
# this will deadlock sooner or later or never, more Prefetch makes deadlock to appear, even more -> deadlock sooner 
ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()))")
ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()),X=0,Y=30)")
/* This works for any Prefetch, no dependancy between them
c1=ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()))")
c2=ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious()),X=0,Y=30)")
Stackvertical(c1,c2)
*/
ConvertToRGB32() #additional filters modify timing thus, frame evaluation order/cache hit order in MT
ConvertToRGB24()
ConvertToRGB32()
ConvertToRGB24()
Prefetch(100)

No deadlock or anything, but CPU isn't being well utilized either. No problem found with frame order.

FPS (min | max | average):      87.34 | 14329 | 1564
Memory usage (phys | virt):     32 | 28 MiB
Thread count:                   112
CPU usage (average):            17%

Strangely, if I run FrameRateConverter(debug=true) with this version, the texts are displayed correctly for the first frame then only gibberish is displayed for additional frames.

In contrast, the latest release of Avisynth causes occasional crashes even without MT.

[gaak]

Hi,

Cannot get this script to run to completion w/ i386 r2455:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("DGDecodeNV",      3)
Import("\MKVideo Encoder NV\AVS Scripts MT\LoadPlugins.avs")
LoadPlugin("\MKVideo Encoder NV\DGDec\DGDecodeNV.dll")
DGSource("\MKVideo Files\Input_Video_File.dgi")
Spline64Resize(1280, 720)
RoboCrop(Laced=False, Align=True)
LSFmod(defaults="slow", preblur="DeGrainMedian(limitY=4, limitUV=6, mode=1, interlaced=false)", strength=200, smode=5, secure=true)
Prefetch(6)

Have tried Prefetch values from 3 to 7. As the value is increased the script completion percentage goes up but never past 85% when it crashes. Any other settings I can try?

Thanks.

[Gavino]

Originally Posted by StainlessS

Whoa, that surprised me when removing the 'Return Last' from Fct1(),
The script's return value was not a video clip, (Is a float, 0.479426).

No surprise, really:

http://avisynth.nl/index.php/Grammar

if return is not present in the final executable statement of a script (or script block), it is implied – the statement is treated as if return was present.

So if the last statement of a function is "sin(.5)", that's what will be returned.

[StainlessS]

Clever Cloggs

[Gavino]

Also note:

The full AviSynth grammar - Closing_Remarks

If there is no (explicit or implicit) return, a void value (ie a value of the 'undefined' type) is returned. For example, this will happen if the last statement is an assignment.