GRunT - Easier run-time scripting

[Gavino]

Avisynth's run-time environment is very powerful, supporting complex video processing that would be difficult or impossible to perform in a normal script. But it's not easy to use - the behaviour of run-time scripts (especially in combination) can be hard to understand and there are usability problems concerning scope and lifetime of variables.

"Cond. filters are quite complex - and sometimes I am even surprised of the outcome" - sh0dan

GRunT (Gavino's Run-Time ) is a plugin which addresses these and other problems, making the run-time system much easier to use.

Features:
- Simple, natural and robust way to pass variables into a run-time script from 'outside'
- A run-time script can be evaluated in its own independent scope
- Run-time functions can be called from a user function
- Run-time functions can be applied to any frame (relative to the current one)
- Additional variant of ConditionalFilter with single boolean expression
- Fixes a fairly serious bug in the run-time system
- Lightweight plugin extending the standard run-time environment, minimal time and memory overhead
- 100% backwards compatible with existing scripts

GRunT should be useful to anyone who uses the run-time filters, from those who make occasional use of ScriptClip to those who write complex functions based on run-time features (such as Restore24 or MRestore).

Details

The plugin provides extended versions of the following run-time filters:
- ScriptClip
- FrameEvaluate
- ConditionalFilter
- WriteFile
- WriteFileIf

The alternative names GScriptClip, GFrameEvaluate, GConditionalFilter, GWriteFile and GWriteFileIf may also be used. However, if running on version 2.57 or earlier of Avisynth, then only the alternative names may be used.
(This restriction is necessary for technical reasons - sorry about that.)

Each filter is 100% backwards compatible with its standard equivalent, but has two additional optional arguments:

• string args: the variables whose values are to be imported into the run-time script, written as a list of names separated by commas. The given variable names are evaluated in the current (compile-time) context, so can include function parameters or local variables.
  Each value becomes the initial value of the corresponding variable at each invocation of the run-time script.
• bool local: if true, the filter will evaluate its run-time script in a new variable scope, avoiding unintended sharing of variables between run-time scripts.
  Default is true if args is also specified, otherwise false (to preserve backwards compatibility).

A short example (based on the original) shows how this greatly simplifies passing function parameters into a run-time script.

function bracket_luma(clip c, float th1, float th2) {
  Assert(0 <= th1 && th1 < th2 && th2 <= 255, "Invalid thresholds!")
  ScriptClip(c, """
    avl = AverageLuma()
    avl <= th1 ? last.BlankClip() : avl >= th2 ? last.BlankClip(color=color_white) : last
  """, args="th1,th2", local=true)
}

This is much easier than the standard approach of dynamically building the runtime script using string concatenation, or passing the values via global variables.

"I really do not like this global variable business with ScriptClip ..." - stickboy

And because the run-time script is evaluated in its own scope, there is now no problem in calling bracket_luma more than once in the same script (previously the variables th1 and th2 of different instances could interfere with each other).

Elements of the args string can also take the form 'name=expression' - the expression is evaluated in the current context and is used to set the value of the named variable in the run-time script.
Example: args="x, y=n+1, c=c.Trim(2, 0)" will provide values for the variables x, y and c.
Here y need not even exist in the current environment (although x, n and c must).

The plugin also provides the following extensions to the run-time functions (eg AverageLuma):

• these functions can now be called inside a user function, when the user function is called from a run-time script
• each function has an new optional int argument, which can be used to get the value from another frame, relative to the current one. For example, AverageLuma(-1) returns the value for the previous frame. (No more assigning to current_frame ...)

For added convenience, there is a new variant of ConditionalFilter which takes a single boolean expression instead of three separate parameters as at present. This is useful when the condition to be tested is a compound one or is already available in boolean form. For example,

ConditionalFilter(c, c1, c2, \
 "AverageLuma(c1) > AverageLuma() && AverageLuma(c1) > AverageLuma(c2)")

where previously you would have to add (...,"=", "true")

Finally, the plugin fixes a fairly serious bug I discovered in the run-time system.
This fix is needed if you are running a version of Avisynth prior to build 080620 of 2.58.

More detailed documentation and examples are provided in the attached download.

EDIT 27-SEP-08: Updated to v1.0.1 to work with Avisynth 2.5.7

Comments and suggestions would still be very welcome.

[martino]

Oh my God! I love you!!! Finally I can do something on which I wasted 2 days trying to achieve (and countless headaches)... I think... xD

[Gavino]

Thanks, martino! I hope you find it solves your problem.

If it does, could you post your resulting script here as an example?
Alternatively, if you still have problems, explain what you're trying to do and I'll see if I can help.

[backtohell]

i am not an expert in avisynth but still will try your GRt

Thanks Gavino for sharing

[Gavino]

Previously, I wrote:

Originally Posted by Gavino

For added convenience, there is a new variant of ConditionalFilter which takes a single boolean expression instead of three separate parameters as at present. This is useful when the condition to be tested is a compound one or is already available in boolean form.

I have now noticed that the standard ConditionalFilter, as well as forcing you to use 3 parameters, does not even support all the boolean operators - you can only use "=", "<" or ">", and not "<=", ">=" or "!=". Of course, you can express ">=" by using "<" and switching round the 'then' and the 'else' clips. But GRunT's version (as well as being slightly shorter) allows you to express the condition in the way you choose, whichever is more convenient/natural to you.

[MOmonster]

@Gavino
I don´t get ScriptClip running.
I get allways the error:

ScriptClip does not have the named argument "show"

I tested your sample scripts and some of my own, but allways the same error (no error without your plugin).
What is wrong?

[Gavino]

@MOmonster: I'm sorry about this - it is a problem with GRunT and Avisynth v2.5.7 that I am still looking into.

At the moment, the workaround is to use v2.5.8, where it works fine.

[MOmonster]

Thanks for the fast answer.
I will do some testing when I adapt srestore to avisynth 2.5.8.

Edit:
Another question. Is it possible to shift values without using global values?

[Gavino]

Originally Posted by MOmonster

Is it possible to shift values without using global values?

What do you mean by 'shift values'?
Can you give me an example?

[Didée]

Its probably related to the fact that R24 uses user defined functions within scriptclip -- Like

function ShiftBackVars() { do_the_shifting_stuff }
...
ScriptClip( c, "ShiftBackVars()" )

When referring to conditional variables in the do_the_shifting_stuff section, they have to be global, don't they?
(Sorry, I'm currently not in the mental "flow" of ScriptClip & Co. ... usually need some run-up when I have to.)

If so, well ... I would NEVER have done R24 without that style of using (lots of) defined functions, called from within the conditional envoronment. That way it was easily possible to get a reasonable level of "modularisation". When having to put all that stuff within one gigantic

ScriptClip(c, """
...
put several hundred lines of code here ...
... 
""" )

then Restore24 never would have come into existance. (Which would be a pity ... think the butterfly effect. )

Howeveritmightbe. The important point is you guys now are heading to do it right, without me needing to get knots in the brain.

[Gavino]

Originally Posted by Didée

When referring to conditional variables in the do_the_shifting_stuff section, they have to be global, don't they?

Yes, to be visible in a function, they do need to be global, as I mentioned above.
And if you're writing to them, you don't have the option of passing them as parameters.

I would NEVER have done R24 without that style of using (lots of) defined functions, called from within the conditional envoronment. That way it was easily possible to get a reasonable level of "modularisation".

GRunT helps in that respect because it allows calling of the run-time functions (such as AverageLuma) from within a user function.

[Gavino]

Version 1.0.1: Bug fix for Avisynth 2.5.7

As MOmonster reported above (#6), and also as reported here, there are problems using GRunT with Avisynth v2.5.7. In fact, I'm embarrassed to say that it just doesn't work on anything prior to v2.5.8.

I have produced a new version (GRunT 1.0.1) that solves the problem. The fix requires users running on 2.5.7 (and earlier) to use alternative names for the run-time filters (GScriptClip instead of ScriptClip and so on). The run-time functions, such as AverageLuma, are not affected.

Users already on 2.5.8 can continue to use the old names - the alternative names may also be used if preferred.

Go to the first post in this thread to download the new version.
Updated documentation, reflecting this change and minor editing, is also included.

[MOmonster]

Thanks for update, but I can´t download it.

[Gavino]

@MOmonster: I assume your problem was that the attachment was still awaiting approval. It should be OK now.

[canuckerfan]

Currently I'm using AviSynth 2.5.8's built-in scriptclip and conditionalreader. anyway I can adapt this code so that I can use Gavino's versions?

Filtered = RemoveNoiseMC(rdlimit=18,rgrain=1,denoise=0,sharp=true) 
ScriptClip("ABC1 ? Filtered : Last")
ConditionalReader("seriously.txt","ABC1",false)

[Gavino]

If you just load the GRunT plugin (or install it in the plugin folder), it will use my version automatically (if running on Avisynth 2.58).

For your example, where the run-time script is very simple, you wouldn't actually gain very much.
However, it would allow you to replace the ScriptClip call by
ConditionalFilter(Filtered, last, "ABC1")
which would be slightly faster.

Even without GRunT, it could also be changed, although with the standard ConditionalFilter you would have to write
ConditionalFilter(Filtered, last, "ABC1", "==", "true")

GRunT does not have its own version of ConditionalReader, so you would continue to use the standard version.

[canuckerfan]

^Thank you

[Forensic]

I have tried everything that I can think of. What I need is a single pass solution to step through a video and only retain frames where AverageLuma(a)>16 (IOW, retain only non-blank frames). Scriptclip can step through the video and conditionally test the AverageLuma value, but I can't get the new video out of the Scriptclip, even with GRunT. Any suggestions?

[Gavino]

Originally Posted by Forensic

I have tried everything that I can think of. What I need is a single pass solution to step through a video and only retain frames where AverageLuma(a)>16 (IOW, retain only non-blank frames). Scriptclip can step through the video and conditionally test the AverageLuma value, but I can't get the new video out of the Scriptclip, even with GRunT. Any suggestions?

As an alternative to StainlessS's solution, you could use my DeleteFrames() function, like this:

DeleteFrames("AverageLuma() <= 16")

(It works out at compile-time which frames to delete, so the script will take a noticeable time to load for a long video.)

BTW Nice to hear my code is being used in real practical applications!

[StainlessS]

Yo dude, long time no see.

So far as I know, Scriptclip must produce same number of output frames as input. (+ same size, colorspace etc).
So 1 single pass solution is unlikely. Although could do it as realtime 1st pass, with auto second pass.
No idea what IOW means.

But see here:- http://forum.doom9.org/showthread.php?t=172904

You might want to use YPlaneMinMaxDifference (Threshold=eg 0.2) to establish 'blank frame', and YPlaneMin or AverageLuma to establish level.