Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd June 2008, 22:22   #1  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Problems using AverageLuma etc in user-defined function

Why can't I use a run-time function (like AverageLuma) inside my own function, as long as my function is only called inside a run-time script (eg via ScriptClip)? Why doesn't this work:
Code:
function AL(clip c) {
  Subtitle(c, string(AverageLuma(c)))
}
...
ScriptClip("AL()")
when it's effectively equivalent to this:
Code:
ScriptClip("Subtitle(string(AverageLuma()))")
which does?

It's clear that run-time functions can only be used inside a run-time script. But here the call to function AL is inside the run-time script. I also understand the distinction between parse-time and run-time, which is usually at the root of problems with ScriptClip and friends. But here both the failing script ("AL()") and the working one ("Subtitle(...)") are at the same level.

Is it a bug, or a feature, or is my code wrong?
Gavino is offline   Reply With Quote
Old 4th June 2008, 20:06   #2  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
This is because runtime variables and functions are made available at the top-level runtime script code only and not in nested scopes (thus they are effectively locals and not globals). Thus although "AL()" and "Subtitle(...)" are at the same level, the AverageLuma call inside your function is not. It is in a nested local scope and not the top-level (local) script scope.

It may not be intuitive, but this is the way things work ; the runtime environment is different from the "normal" script environment.
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 4th June 2008, 21:30   #3  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Thanks gzarkadas, I understand what you are saying.

Do you know if there is a good reason that it has to work that way, or is it "just the way it is"?

For usability, it would be advantageous to allow their use in functions called from within the run-time script. For my simple example it doesn't much matter, but for large/complex run-time scripts, it would make for more readable and maintainable code. Or would there be undesirable consequences that I am not foreseeing?

(Maybe this is now really a question for the development forum?)
Gavino is offline   Reply With Quote
Old 5th June 2008, 23:22   #4  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
Quote:
Originally Posted by Gavino View Post
...
Do you know if there is a good reason that it has to work that way, or is it "just the way it is"?...
Runtime filters construct a script parser on-the-fly at each call to GetFrame to parse the runtime script. Before doing so they set runtime variables -most importantly, current_frame- on the top-level local script code. The runtime functions search for the existent of current_frame variable and exit with error if not found. Since the variable is at local scope, when calling them from within a function, which has its own local scope, the search fails.

Now, about the reason, I can only speculate, since I am not an AviSynth developer , but I see the following difficulty if the choise were made to let runtime filters setting runtime variables at the global scope:
The user script is allowed to do anything inside the very one GetFrame call that has set current_frame. This includes calling other runtime filters with different frame number(s).

The later would set current_frame to their own value(s) and since in our scenario that assigment would be global, they would override current_frame.

If then the script at the initial GetFrame call tried to call another runtime function, the later would operate at a different framenumber than the intended (even a non-existent one). The same is true if just the current_frame variable was used in some way (say to Trim a clip).

That, would certainly resulted in a havoc, sooner or later.
Using local scopes avoids this interaction; and now that I can step back and see my speculation in its entirety, I must admit that there is a good reason .
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 6th June 2008, 00:45   #5  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Thanks for your very comprehensive reply.

I need to study your argument at greater length, but it strikes me that the restriction results from the specific way the run-time features are implemented 'under the hood', rather than something inherent in the concept.

(That's still a 'good reason' if it's really just too hard to do it any other way.)
Gavino is offline   Reply With Quote
Old 6th June 2008, 10:46   #6  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
If I understand gzarkadas correctly, he is saying:
  1. You cannot call AverageLuma (etc) inside a user function because current_frame is not visible there.
  2. This in turn is because current_frame is created as a local variable on entry to the runtime script (inside the GetFrame of ScriptClip).
  3. It needs to be local rather than global to prevent problems with cases like:
    Code:
    ScriptClip("""
       ...
       ScriptClip("...")
       ...
    """)
    in which the inner run-time script could disrupt the correct functioning of the outer one by its own use of current_frame. Therefore it needs its own independent instance of the variable.
I'm not sure that the above example can really cause a problem, because the inner run-time script is executed after the statements of the outer one (which in turn are executed after any others in the top-level (normal) script). Perhaps a more complicated example is needed to illustrate the problem.

But in any case, could this not be fixed just by ScriptClip resetting current_frame to its initial value when it exits GetFrame (just like it does with last)? current_frame could then be allowed to be global, and we could have access to it (and hence AverageLuma etc) in user functions.
Gavino is offline   Reply With Quote
Old 6th June 2008, 19:58   #7  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
Quote:
Originally Posted by Gavino View Post
...
because the inner run-time script is executed after the statements of the outer one (which in turn are executed after any others in the top-level (normal) script).
...
This is not correct; the inner script is evaluated inline, at the point of the call, just as a procedure in a typical procedural language is executed at the point of the call, leaving the caller in a wait state (for its result and/or side-effects) until it returns and the caller resumes to the next statement.

Quote:
Originally Posted by Gavino View Post
...
But in any case, could this not be fixed just by ScriptClip resetting current_frame to its initial value when it exits GetFrame (just like it does with last)? current_frame could then be allowed to be global, and we could have access to it (and hence AverageLuma etc) in user functions.
When GetFrame is exited, the video frame is returned to the caller (either the filter upwards the chain or the rendering application); there is nothing to reset at that time. The entire runtime script has already been evaluated.

The critical point is that the runtime environment has a different, event-driven model than the "normal" script flow. The script execution model page at the Avisynth wiki may be helpful on this.

In summary:
  • After the "normal" script is evaluated the only things that remain are:
    • a chain (more accurately a graph) of filter objects, the filter-graph,
    • the global and script-level-local variable tables,
    • the function table.
    The sole purpose of the script is to create the filter-graph; the tables are built to facilitate that purpose. They also provide functions and variables for runtime scripts.
  • Whenever then the rendering application requests a video frame from Avisynth, Avisynth calls the GetFrame method of the "head" filter object of the filter-graph.
  • Inside its GetFrame method, the "head" filter object of the filter-graph does whatever processing is needed to create and return the requested frame. Typically this includes requesting frames from the filter objects "deeper" in the filter-graph.
  • This results in a chain of synchronous (ie inline at the point of call) GetFrame calls, with arbitrary in the general case frame numbers for each clip, that keeps growing at the Avisynth's stack until the leaf-node filter objects of the filter-graph deliver their frames and return. The stack then starts to unwind until the "head" filter object of the filter-graph returns to the caller application the requested frame.
  • Then the cycle is repeated for a new frame that the caller application requests, as many times as needed before it terminates.

Now, to return in our previous subject, when a runtime filter evaluates its script to return the n-th requested frame it does it in a block-wise manner, using the same mechanism as the "normal" script: the entire script is evaluated, a temporary filter-graph is created and a frame is requested from the temporary filter-graph; the result is returned to the caller.

Thus, the runtime filter cannot know in advance what frame numbers its script will use, neither can it intercept the parsing of the script to adjust the value of current_frame. That would mean another parser, ie another Avisynth implementation; a very resource-costly thing.

So, I would suggest to stick with the alternatives:
  1. Instead of calling a runtime function inside a user function, create an argument where the result of the runtime function can be passed.
  2. Use triple quotes to create multiline runtime scripts; there is no need to try fit everything in a single line (this one is my favorite ).
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 7th June 2008, 02:45   #8  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
gzarkadas, I do already understand the script execution model (though I could easily be mistaken, and it's even more confusing than usual when you have nested run-time scripts).

Let's look again at my example,
Code:
... # prelude
ScriptClip("""  # start of outer
   FilterA()
   ScriptClip("FilterB()") # inner
   FilterC()
""" # end of outer
... # postlude, rest of 'normal' script
where this time I have added some names to refer to.

Because the statements of a run-time script are not evaluated until the frame-serving phase of the enclosing script, I say that the order of statement evaluation is
Code:
prelude, outer ScriptClip, postlude, [FilterA, inner ScriptClip, FilterC, FilterB]*
[]* = repeated for each frame
So although FilterB potentially sees a different value of current_frame to FilterA and FilterC (set in the GetFrame of different instances of ScriptClip), they do not interfere with each other. I still think that this example does not exhibit the problem you had in mind and would work the same whether current_frame was local or global.
(I also conceded a more complex example might be able to demonstrate it. If you've got one, please post it here.)

Where I do agree with you is that my idea of resetting current_frame on exit from GetFrame is pointless, since on the way out of the GetFrame chain, all the evaluations have already happened. Ah well, it was just an idea...

I'm certainly not after anything that would require a new parser or changing the language (I'll leave that to the AviSynth 3.0 crew), just trying to understand exactly what the problem is and explore if there is a simple solution.

If not, your suggested practical approaches are very sound.
Gavino is offline   Reply With Quote
Old 9th June 2008, 13:08   #9  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Quote:
Originally Posted by Gavino View Post
... my idea of resetting current_frame on exit from GetFrame is pointless, since on the way out of the GetFrame chain, all the evaluations have already happened.
Ah, but wait! By the same token, when we enter the GetFrame of the inner ScriptClip (which will go on to evaluate "FilterB"), all other run-time scripts have already been evaluated. So again we see that the inner ScriptClip cannot interfere with the outer script's use of current_frame.

My 'solution' was designed to protect against an inner ScriptClip stamping on the value of current_frame set by an outer one before all statements that needed this value had been evaluated. But this cannot happen. The 'solution' has no effect because there was no 'problem' in the first place.

The key point is that the evaluations of different run-time scripts never overlap in time, even if such scripts are nested. So I say current_frame can be allowed to be global with no ill effects.

If you don't buy this argument, here's a more convincing one:

Although current_frame is a local variable, it is 'local' at the same scope as the top-level script - a run-time script does not start a new scope. Thus there is only ever one instance of current_frame, which just gets re-used in a serial fashion for each run-time script.

So making it truly global cannot break the way run-time scripts work; as far as I can see, the only effect would be to make it visible in functions, which is just what we need to solve my original problem with AverageLuma().
Gavino is offline   Reply With Quote
Old 9th June 2008, 20:12   #10  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
I don't have the time to really check this in the depth that is required. However, a solution that does not require to recompile avisynth in order to use runtime functions inside user functions is to put the following line:

Code:
global current_frame = current_frame
either at the begining of each ScriptClip's runtime script (effectively you will have to use multiline strings only), or inside a FrameEvaluate call that you put after each call to a runtime filter. Say (a quick example):

Code:
...
ScriptClip("some script")
FrameEvaluate("global current_frame = current_frame")
...
# or if clips are assigned to variables
...
c1 = c0.ScriptClip("some script")
c2 = c1.FrameEvaluate("global current_frame = current_frame")
# use c2 instead of c1
...
You can even wrap the last one in custom runtime functions to save the trouble of repeating this stuff again and again:
Code:
function G_ConditionalFilter(clip testclip, clip source1, clip source2, string function, string operator, string value, bool "show") {
    c = ConditionalFilter(testclip, source1, source2, function, operator, value, show)
    return c.FrameEvaluate("global current_frame = current_frame")
}

function G_ScriptClip(clip clip, string function, bool "show", bool "after_frame") {
    c = ScriptClip(clip, function, show, after_frame)
    return c.FrameEvaluate("global current_frame = current_frame")
}

function G_FrameEvaluate(clip clip, string function, bool "after_frame") {
    c = FrameEvaluate(clip, function, after_frame)
    return c.FrameEvaluate("global current_frame = current_frame")
}

function G_ConditionalReader(clip clip, string filename, string variablename, bool "show") {
    c = ConditionalReader(clip, filename, variablename, show)
    return c.FrameEvaluate("global current_frame = current_frame")
}
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 9th June 2008, 20:15   #11  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
The solution works because when Avisynth requests a variable and it is not found at the active local scope, the global scope is then searched.
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 10th June 2008, 15:30   #12  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Good thinking, gzarkadas - that does the trick very nicely indeed.

I may follow up the implementation issue on the development forum, but I think my original question has now been answered. To summarise the conclusions:
  • You can't currently use a run-time function (like AverageLuma) inside a user-defined function, even if that function is only called inside a run-time script (eg via ScriptClip).
  • The explanation in language terms is that current_frame (needed by AverageLuma and set on entry to each run-time script) is a (script-level) local variable, and hence not visible inside the user function.
  • Work-arounds include the following, according to taste:
    1. just do everything directly in the run-time script;
    2. call the run-time function directly in the run-time script and pass the resulting value as a parameter to your function;
    3. set a global current_frame yourself with the same value as the local one, either in the run-time script, or in a (following) FrameEvaluate, or (IMHO the neatest) using the wrapper functions defined above by gzarkadas.

Last edited by Gavino; 10th June 2008 at 15:32. Reason: formatting
Gavino is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.