Delete frames satisfying user-defined condition

[Gavino]

Prompted by this thread, and in particular jmac698's remark:

Originally Posted by jmac698

This type of editing seems tremendously difficult in avisynth.

I have written a generic function that will delete all frames satisfying an arbitrary user-defined condition.

The condition can use any of the Avisynth run-time functions such as AverageLuma(), etc and/or the variable current_frame.

GScript("""
  function DeleteFrames(clip c, string condition) {
  # DeleteFrames (Gavino, 16 Nov 2011)
  # A generic function that will delete all frames satisfying an arbitrary user-defined condition.
  # The condition can use any of the Avisynth run-time functions such as AverageLuma(), etc, 
  # and/or the variable 'current_frame'.
  # (Note - Since the condition is evaluated inside the function,
  # any variables used (except current_frame) must be global.)
  # Example use (delete duplicate frames):
  #   DeleteFrames("YDifferenceFromPrevious < 0.1 && current_frame > 0")

    c    
    fc = FrameCount()
    res = BlankClip(c, length=0)
    current_frame = 0
    
    while (current_frame < fc) {
      while (Eval(condition) && current_frame < fc) {
        current_frame = current_frame+1
      }
      if (current_frame < fc) { # not at end
        start = current_frame # start of wanted section
        while (!Eval(condition) && current_frame < fc) {
          current_frame = current_frame+1
        }
        res = res + Trim(start, start-current_frame)
      }
    }

    return res
  }
""")

For example, jmac698's task (delete duplicates) can be done with:
DeleteFrames("YDifferenceFromPrevious < 0.1 && current_frame > 0")

Notes:
- Since the condition is evaluated inside the function, any variables used (except current_frame) must be global.
- As well as being completely general, the function (unlike the solutions in the referenced thread) works in a single script and does not require linear rendering.
- Since it scans the entire clip at compile time, it may be slow to load on very long clips.
- Requires GScript.

[johnmeyer]

This will be very useful. Thanks!

[Guest]

A naive approach is going to kill static scenes, isn't it?

[Gavino]

Originally Posted by neuron2

A naive approach is going to kill static scenes, isn't it?

You mean the 'delete dups' example?
Yes, of course, that particular example won't work for all sources, if you have real duplicate frames you want to keep. A more complicated condition would have to be devised (perhaps simply adding a check based on current frame if you know you have a static scene at a given point).
It's up to the user calling the function to supply the right condition appropriate for the job in hand.

[Guest]

I'm suggesting that most streams have static scenes, so in most cases, naive deletion is going to be problematic.

Perhaps you can fancy up your script so that it doesn't touch runs of N duplicates, where N is configurable. This static scene protection was the main reason for MultiDecimate.

[jmac698]

Awesome. It's a building block; would be more useful if it could delete at most n in a row, or only when m minimum dupes. This would be beyond any arbitrary condition, which could be checked per frame. Also, what happens to the audio?
Would be good to not only delete, but (in my case) average a string of (cond) down to 1 frame.

[Gavino]

Originally Posted by neuron2

I'm suggesting that most streams have static scenes, so in most cases, naive deletion is going to be problematic.

Perhaps you can fancy up your script so that it doesn't touch runs of N duplicates, where N is configurable. This static scene protection was the main reason for MultiDecimate.

You're right, of course.
However, the purpose of my function is not to 'delete duplicates' - that was just a (naive) example and there are other solutions designed specifically for that. It is intended to provide a user-friendly solution to the general problem "How do I delete all frames that have such-and-such a property?", where the property can be expressed as a boolean condition.

I'll look into the possibility of setting a specified maximum of consecutive frames deleted.

@jmac698:
Audio for the deleted frames gets deleted too. If you want to preserve the original audio, use AudioDub to restore it afterwards. I'll look into your other suggestions.

[Gavino]

By the way, GRunT is not necessary for this function, but if you do have it installed, you can make use of the extensions to the run-time functions, eg AverageLuma(-1) to get the avaerage luma of the previous frame. This makes it easier to write more complex conditions depending on several frames together.

[StainlessS]

Excellent Generic.

[IanB]

I am constantly amazed at what people can actually do with Avisynth scripts. Even when the script language itself fights tooth and nail against the script writers intentions.

I must say I am really impressed but not surprised that Gavino has successfully implemented a script that forces the Avisynth compile phase to do an analysis pass on a video stream. Well done. I rate this up there with the mythical scripts that play tic-tac-toe and the game of life for pure academic ingeniousness.

That said, I don't think this is a particularly productive thing to actually do in practise. And it is actually a bad thing in that encourages the naive to continue to approach Avisynth with the wrong mindset.

Conditionally changing the frame count in Avisynth is a 2 pass operation. Do the 1st pass once, save the result then do the 2nd pass as many times as required.

And I guess the next enhancement we are going to see is a way to save the pass 1 state so the script can skip the 1st pass and load fast for the 2nd and subsequent times.

[StainlessS]

Originally Posted by IanB

And I guess the next enhancement we are going to see is a way to save the pass 1 state so the script can skip the 1st pass and load fast for the 2nd and subsequent times.

Sounds like a challenge to me.

[jmac698]

Sounds like a challenge to me too.
http://forum.doom9.org/showthread.php?t=163121

Thought of a way to do real-time tic-tac-toe game too.. maybe later.

[Gavino]

IanB - Sincere thanks for your comments (a mixture of back-handed compliments and a slap on the wrist ).

I see where you are coming from, but I'm not sure I fully understand your objections.
I agree my method of doing video analysis at compile-time is an abuse of the run-time system (in previous posts I have described the underlying technique as 'cheating') and I would discourage thoughtless use of it where more 'orthodox' methods are available.
Although the code is (if I may say so) beautifully simple, perhaps it should have come with a health warning.

Originally Posted by IanB

Conditionally changing the frame count in Avisynth is a 2 pass operation.

Is there a good reason (practical or even philosophical) why this must be so?
The motivation for my function was to provide a more user-friendly solution (not an academic exercise), so that the whole thing could be done in a single function call, allowing the user to forget about the mechanics of the operation and leaving him free to concentrate solely on defining the appropriate condition for the frames to be deleted.

I don't think an operation like this is something you would normally want to run several times on the same source. (Perhaps if you wanted to experiment with different downstream filters, or just repeatedly view the script output in a player.) And if running on a different source, you would need to run the 1st pass again, so I see no benefit to a two-pass approach.
If using my function in a multi-pass encode, the video analysis is not repeated, since it happens at load-time.

If a user prefers to do it in two passes, then this function can be used instead:

function DeleteFrames2(clip c, string condition, string fileName) {
  keepCondition = "!(" + condition + ")"
  WriteFileIf(c, fileName, keepCondition, """ "0," """, "current_frame", append=false)
}

which will produce a command file to be used with StainlessS's Prune plugin in the 2nd pass.

[JAMESE]

Good
This very useful and easy use.
Thanks for share.

[StainlessS]

Thankyou for that Gavino, Ill nick a copy of it if you dont mind and put it in the Prune() thread [DeleteFrames2()].

Hope you saw the real world app of the YDifferenceFromPrevious noted in Prune update, ie security
cam footage removal of static scenes leaving only the bad guys up to no good.

[Gavino]

Originally Posted by StainlessS

Hope you saw the real world app of the YDifferenceFromPrevious noted in Prune update, ie security
cam footage removal of static scenes leaving only the bad guys up to no good.

Yes, thanks, I noticed that and thought "that's a good practical example", which hadn't occurred to me.

[IanB]

Originally Posted by Gavino

Is there a good reason (practical or even philosophical) why this must be so?

I guess it's a design legacy of being AVIsynth, where the final attributes like frame count need to be all known at start time. If we were a different product, e.g. a STREAMsynth then this would not be such an issue. Although some things you want to plug in to need to have attributes defined as they start. So you end up needing a trip to the end of the stream the see how long it is, so you inherit a de-facto pass1 again.

[Gavino]

Yes, basically my DeleteFrames function is using a trick (exploiting an implementation detail of the run-time system) to do the first pass at compile-time, allowing both passes to be done in the same script (if desired!).