ImageSource(), ImageWriter(), x264-compressed still photos, avisynth improvements

[jmac698]

Like I said, you can use the call plugin to run a batch script or whatever, but it's only run once before the script starts.

Here's how to read and plot pixels:

Function GetPixel(clip clip, int x, int y) {
    #Return luma of a pixel
    current_frame=0
    clip
    IsYUY2 ? ConvertToYV12 : clip#AverageLuma only works on planar.  YV12 is planar.
    pointresize(6,6,x,y,1,1)#Blow up the pixel at x,y by 6 times (minimum possible with resize)
    int(AverageLuma)
}

Function PutPixel(clip clip, int x, int y, int luma) {
    #Plot a pixel
    clip
    ClipIsYV12=IsYV12
    ClipIsYV12 ? ConvertToYUY2 : last
    pointresize(width*2, height)
    u=128
    v=128
    pixel=blankclip(last, color_yuv=luma*65536+u*256+v, width=2, height=1)
    layer(last, pixel, "add", 256, x*2, y)#Layer only works with YUY2
    pointresize(width/2, height)
    ClipIsYV12 ? ConvertToYV12 : last
}

I have to tell you, I'm using an undocumented trick here. You can read a pixel once at the start on a specific frame, by setting current_frame. This lets you use the runtime variables, like AverageLuma. Remove the "current_frame=" line to use it as a runtime function. With the runtime way; averageluma has to appear inside scriptclip or one of the others.

The array implementation in avslib is quite slow. I have the same thoughts of a meta-data per-frame with another clip; but it could be added at the bottom. The garbage in the last few lines can be be cropped out.

You can extend this to RGB with showred.converttoyv12 for example. You could also make a kind of array in a clip by plotting pixels and reading them.

My slicer and addcode plugins can also read and plot int's on a line of video.
http://forum.doom9.org/showthread.ph...74#post1582574

for myself: keywords pset, plot, plot a pixel, plot(x,y), pset(x,y)

edit: thank you me! I swear, this is the 5th time I've searched for this post.

[TheFluff]

Originally Posted by jmac698

And saying it's a filter graph is just semantics; I can write a loop that does nothing but calculate a factorial, if I want to. You could even argue that it's Turiing complete and therefore like any other language. There's even GUI plugins to Avisynth; you can get keyboard input and interact with it just like a real program. Just let the video play forever and forget that it's a video, cause it's really a program

The fact that the scripting language allows you to write external functions in a general purpose language that gives you arbitrary side-effects does not change the fact that the scripting language just describes a graph. The entire script has a return value, which is the end point of that graph.

Originally Posted by turbine

Object-oriented has been the standard for a long time due to its advantages. On the contrary it's the intuitive way to way to represent video and audio, the standard "is a" and "has a" questions we ask ourselves when creating classes. A video "has a" set of frames. A frame "has" one or more images (1080p vs. 480p, 1.77 AR vs. 1.33 for example), an image "has a" set of pixels.

Object oriented programming focuses on the interaction between data structures, but video frame buffers don't exactly need to talk to each other. The data structures you describe are all very simple and there's nothing particularly object oriented about them.

I don't deny that object oriented programming is very useful in many scenarios, but pretty much none of its big features (inheritance, encapsulation, polymorphism, data abstraction) are of any particular use for video processing. Complain about Avisynth script all you want, but the dataflow/graph paradigm was definitely the correct choice for the kind of video clip manipulation Avisynth was intended for. As for writing actual video filtering functions, you pretty much have to do that in a low-level compiled and unmanaged language in order to get it fast enough, and when you're on that level you pretty much work with plain byte arrays, so there's really no point to object-orienting anything.

Doing individual pixel manipulations in an interpreted language/managed code environment isn't really very useful in practice because even on modern hardware it's REALLY SLOW for any resolution bigger than a postage stamp, and at the time when Avisynth was designed it was, of course, a completely ridiculous idea, so the scripting language really doesn't try to help you when it comes to that kind of stuff.

Originally Posted by turbine

Performance would take a hit, but isn't really a big concern when the weak link in the chain is typically the codec (x264 usually) and it's not realtime. Realtime frame-serving would arguably be a different story.

looks like you've never used a real-world avisynth script to actually do anything productive (hint: the frameserver is almost always the biggest bottleneck by far)

Originally Posted by turbine

I realized it could be useful to create a parallel clip to store metadata, a RGB32 pixel can store a 32bit float or four ASCII characters, for example.

The complete inability to store any sort of per-frame or even per-clip metadata is one of Avisynth's greatest weaknesses.

Originally Posted by turbine

Uncompressed audio could work if it's possible to read/write individual samples...

It is, if you write a plugin. Start to see a pattern emerging here?

Spoiler: the pattern is "don't use a simplistic graph description language if what you really want is an unmanaged environment that lets you poke arbitrary byte addresses". If you want C, write C.

Originally Posted by turbine

On the other hand, if it is still commonly used, there's probably still value in adding a new run mode so that a user could choose to have it execute all commands as written, instead of it's current "lazy" execution model. Doesn't it seem like that would solve so many common problems?

Okay, let me try to explain this one final time.
Avisynth scripts are not executed. The script is compiled, yes, but the result of the compilation is a chain of filter functions, not a program that is executed. Absolutely nothing happens with it until the host application (which loaded the script) requests a video frame. At that point, Avisynth pulls the frame through the function chain, and it's only at this point that things happen in "filter" functions that have side effects (i.e. ImageWriter). In other words, if you want to call a function on each frame of a clip, you need to make sure all frames of that clip are actually requested, either directly by the host application or indirectly via something in your script (careful with that though, you may or may not run into funny issues if frames are requested multiple times).

Your idea that the environment should execute scripts imperatively is, of course, completely ridiculous. Just try thinking it through for a while and you'll probably realize why it's completely wrong to try to process an entire video clip procedurally instead of via a per-frame callback-driven data flow (just think of what kind of memory footprint a source filter that returns an entire clip at once would have). Incidentally, your idea would require changing so much of Avisynth that it'd be easier to just rewrite the entire thing from scratch.

[StainlessS]

@TheFluff,
Excellent Tutorial

[gyth]

Originally Posted by TheFluff

I don't really understand what you want to accomplish with your script

Tar with built-in lossy compression???

[StainlessS]

Originally Posted by TheFluff View Post
I don't really understand what you want to accomplish with your script

Originally Posted by gyth

Tar with built-in lossy compression???

I think TheFluff was replying to Turbine, and you were referring to your post:

a=ImageSource("sm444.png")

a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

a.subtitle("rendered").ImageWriter("rendered", type="png")

In the 1st ImageWriter line the result is thrown away (temp assigned to Last and then overwritten
by following line), so the result is never used (neither is the source of the result, ie 1st ImageWriter,
nor the source of that ie the 'not rendered' Subtitle) as it plays no part in the output.

a simple

a=a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

would solve the problem but for the two different outputs required by your script, one with
"not_rendered" on it and the other with "rendered", assuming that you actually want two different
outputs with two different subtitles, then you would need to somehow use the result of the 1st
ImageWriter line in the second.
(The problem here is there is only one output, but you want
to have a clip with two different subtitles, and not one overwriting
the other nor one without and one with a subtitle.
[in the words of a famous movie script, 'There Can Be Only One'])

There are any number of ways this could be done, eg

a=ImageSource("sm444.png")

Tmp = a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

dummy = Tmp.FrameCount()
dummy = dummy - dummy

a.subtitle("rendered").ImageWriter("rendered",start=0+dummy, type="png")

Not tested but should I think work. (There are probably better ways)

EDIT: The dummy arg forces the 1st Imagewrite chain to be processed
as the output is reliant on both subtitled paths through the
graph, the advantages are that it is a highly efficient way
of selecting what is and is not necessary due to various conditions,
but results in seemingly silly kludges to force 'side effects' to occur.
Avisynth could well do some things better, but is quite remarkable in what
is does and the filter graph model itself is in no need of change.

[gyth]

Tar with built-in lossy compression???

That was my guess as to what turbine was trying to do.

Originally Posted by StainlessS

Not tested but should I think work. (There are probably better ways)

And these better ways should be collected and linked to a faq and ImageWriter.

This one works for me

a = ImageSource("sm444.png")

a0 = a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

a1 = a.subtitle("rendered").ImageWriter("rendered", type="png")

stackhorizontal(a1, a0).crop(0, 0, a1.width, a1.height)

[StainlessS]

Perhaps a "Return me a zero", and "Return me a one" should be added to the clip status returns
so you dont have to be silly and be so creative.
Was hard to tell what comment you were replying to, sorry, got it wrong.
The '???' was a clue but I EVENTUALLY got it wrong just the same, Shoot me!

[gyth]

Maybe add a function designed to produce side effects easily. (while minimizing overhead)

function render_both_return_first(clip c1, clip c2) {
    ????
}

[turbine]

Thanks, for the replies, guys.... I have had my hands full with a big event, I'll have more time later today or tomorrow.

Fluff, As far as script execution goes you're arguing terminology -- I get it; it's event driven, the processing or "execution" of the various commands happens when a frame is requested.

[jmac698]

And here's how to "execute" an Avisynth script without showing any frames:
http://forum.doom9.org/showthread.php?t=165528

You could wrap this all up into a program that processes text files, and never know it had anything to do with video. Rename it "sh.exe" and no one would be the wiser!

[TheFluff]

Originally Posted by jmac698

And here's how to "execute" an Avisynth script without showing any frames:
http://forum.doom9.org/showthread.php?t=165528

You could wrap this all up into a program that processes text files, and never know it had anything to do with video. Rename it "sh.exe" and no one would be the wiser!

That still retrieves all frames, even if it just discards them immediately after retrieving them.

[pbristow]

Originally Posted by turbine

Fluff, As far as script execution goes you're arguing terminology -- I get it; it's event driven, the processing or "execution" of the various commands happens when a frame is requested.

Turbine, no he isn't! He's pointing out fundamental design concepts that have to do with achieving the best possible video processing performance on barely adequate hardware. Many of us are doing complex, heavyweight processing that absolutely dwarfs the CPU time and memory allocation required by a mere codec, and in that context, Avisynth's *fundamental design concept* of only processing the frames that will actually make a difference to the output is absolutely the right one *in most cases*. In the rare cases that it causes a problem, there are various workarounds available.

Avisynth has some flaws, sure, but "not being a totally different animal" isn't one of them!

I'm going to have to agree with the overall concensus here and say, if you want a tool that works in a completey different way to AviSynth, then use a tool that isn't AviSynth. If the tool you want doesn't exist yet, then start recruiting a team to create one. (Hint: Annoying the people who might very well be glad to join such a team by denigrating the work they've already done on another project is probably not conducive to succesful recruitment!).

If you can figure out how to make the new tool compatible with Avisynth plug-ins, you (and your users) would get the benefit of all the work that's already been done over the last decade in creating a huge variety of awesomely powerful video processing tools. You could even give it an Avisynth-like syntax, if you think that would be helpful... But you would probably find that designing an appropriate syntax to suit the way your new tool works would be better.

Good luck!

[Gavino]

[Catching up on recent threads after a long absence]

Originally Posted by StainlessS

a=ImageSource("sm444.png")

Tmp = a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

dummy = Tmp.FrameCount()
dummy = dummy - dummy

a.subtitle("rendered").ImageWriter("rendered",start=0+dummy, type="png")

Not tested but should I think work. (There are probably better ways)

EDIT: The dummy arg forces the 1st Imagewrite chain to be processed
as the output is reliant on both subtitled paths through the
graph ...

That doesn't work, since Framecount is known at compile time and does not require run-time access to frame data.
You need something that forces rendering of the frames, such as gyth's example:

Originally Posted by gyth

a = ImageSource("sm444.png")

a0 = a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

a1 = a.subtitle("rendered").ImageWriter("rendered", type="png")

stackhorizontal(a1, a0).crop(0, 0, a1.width, a1.height)

[StainlessS]

Thank you Maestro,
glad you're back, think D9 was about to go into mourning.

[Chikuzen]

why don't you use ImmaAVS?
ImmaRead() has 'size_x' and 'size_y' options to resize/pad automatically to the size you specified.