Vapoursynth

[feisty2]

Originally Posted by Myrsloik

r51:
using get_core() in python now generates a deprecation warning since it's been deprecated for years

how do I specify constants like vs.GRAY or vs.RGB if I switch

import vapoursynth as vs
core =  vs.get_core()

to

from vapoursynth import core

?

[ChaosKing]

You can use this

from vapoursynth import core, YUV, GRAY

and then use just GRAY instead of vs.GRAY
OR

use

core = vs.core

instead of

core = vs.get_core()

EDIT
whoops I was too slow

[feisty2]

is there any filter that actually uses VSActivationReason::arFrameReady and queryCompletedFrame()? I haven't been able to find any example for this

[Myrsloik]

Originally Posted by feisty2

is there any filter that actually uses VSActivationReason::arFrameReady and queryCompletedFrame()? I haven't been able to find any example for this

Not that I know of. I'm thinking about removing it completely for the next api revision since I couldn't think of a single useful filter that needs it.

QueryCompletedFrame is also half broken...

[Myrsloik]

R51-RC2

Found a few more trivially fixable bugs when developing the glorious new branch. So here's another RC. If you have far too much time on your hands you could compare the speed and memory usage with R50.

Changes

r51:
updated visual studio 2019 runtime version
fixed a cache shrinking issue
fixed a crash when removing a message handler without a free function
fixed an extremely rare threading issue only affecting fmparallelrequests filters and arframeready events
fixed compilation when avs+ master is used
fixed lut and lut2 triggering a fatal error when invalid planes were specified
fixed property append operations on non-empty keys not properly copying the underlying data
fixed wave64 headers generated by avfs
fixed crash in averageframes with odd number of clips (sekrit-twc)
scale averageframes for integer chroma by distance from grey (sekrit-twc)
several fixes and improvements regarding handling of the active script environment in python (stuxcrystal)
plugin loading now has better error messages (jackoneill)
using get_core() in python now generates a deprecation warning since it's been deprecated for years

[HuBandiT]

Originally Posted by feisty2

is there any filter that actually uses VSActivationReason::arFrameReady and queryCompletedFrame()? I haven't been able to find any example for this

Originally Posted by Myrsloik

Not that I know of. I'm thinking about removing it completely for the next api revision since I couldn't think of a single useful filter that needs it.

QueryCompletedFrame is also half broken...

A few years ago there was talk about potentially getting GPUs involved into VS and VS filters, at which point it might suddenly make sense to hide RAM->GPU transfer latencies by initiating transfer of each individual incoming frame as soon as it becomes available (arFrameReady), instead of initiating transfer of all frames in one large batch (arAllFramesReady). The same would hold in the hypothetical case of writing a multi-node distribution layer for VS (think renderfarm for video filtering).

[Myrsloik]

Originally Posted by HuBandiT

A few years ago there was talk about potentially getting GPUs involved into VS and VS filters, at which point it might suddenly make sense to hide RAM->GPU transfer latencies by initiating transfer of each individual incoming frame as soon as it becomes available (arFrameReady), instead of initiating transfer of all frames in one large batch (arAllFramesReady).

GPUs are kinda dead in this space. You can get 64 cores in a single socket UMA configuration for almost nothing now. I plan to focus on making things run as well as possible with many threads.

You also don't need arFrameReady events unless your filter implementation is a monolithic monstrosity. Have a separate filter to do transfers to the GPU an one after processing to transfer back if needed. That's more or less how I'd handle GPU support if I ever officially added it anyway.

[HuBandiT]

Purely academic as I agree it is not worth pursuing this for now:

Originally Posted by Myrsloik

GPUs are kinda dead in this space. You can get 64 cores in a single socket UMA configuration for almost nothing now. I plan to focus on making things run as well as possible with many threads.

Processing paralellism (cores) yes - but memory paralellism...? Plus you do tie up your CPUs while leaving your GPUs idle, which might be a factor for certain workloads.

Originally Posted by Myrsloik

You also don't need arFrameReady events unless your filter implementation is a monolithic monstrosity. Have a separate filter to do transfers to the GPU an one after processing to transfer back if needed. That's more or less how I'd handle GPU support if I ever officially added it anyway.

For a very forward-looking discussion: OpenCV went through similar CPU memory <-> GPU memory teething pains some years back, and as I recall, they decided to go for making the core (built-in) image structure/class/type/object aware of in which memory domain (CPU or GPU - and for that matter, which GPU if there are several) it currently resides on or has copies in, and having both CPU and GPU versions of most built-in filters, so that transfers between memory domains could be elided in many cases. (Not sure how far they went on optimizing those filter graphs, whether that was statically decided by the programmer, or dynamically optimized during runtime, etc.)

[Myrsloik]

CPUs with more cores have more memory channels and bandwidth too. Unless it's a crippled server cpu being sold to enthusiasts.

The current filter model of operating on whole frames is also very memory bandwidth inefficient. Adding support for filters that work per scanline/tile will allow for huge improvements since then L3 and L2 cache speed would be the limit and not ram.

It's it's the curse of having a simple and developer friendly filter model.

Writing two versions of filters, even the basic ones, is unsustainable for a project of this size. And I don't only mean in terms of development time but also the number of users that test things. It would most likely constantly end up broken.

[Cary Knoop]

Originally Posted by Myrsloik

CPUs with more cores have more memory channels and bandwidth too. Unless it's a crippled server cpu being sold to enthusiasts.

The current filter model of operating on whole frames is also very memory bandwidth inefficient. Adding support for filters that work per scanline/tile will allow for huge improvements since then L3 and L2 cache speed would be the limit and not ram.

It's it's the curse of having a simple and developer friendly filter model.

Writing two versions of filters, even the basic ones, is unsustainable for a project of this size. And I don't only mean in terms of development time but also the number of users that test things. It would most likely constantly end up broken.

Or, have an interface that allows CUDA filters using a CUDA source file that gets compiled on Vapoursynth invocation?


DaVinci Resolve uses this model, you can write a CUDA file with a given interface like so:

 __DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, __TEXTURE__ p_TexR, __TEXTURE__ p_TexG, __TEXTURE__ p_TexB)
{
       float3 x = make_float3(_tex2D(p_TexR, p_X, p_Y),
                                          _tex2D(p_TexG, p_X, p_Y),
                                          _tex2D(p_TexB, p_X, p_Y));
        // Do something here with a 3-pixel:
        x.x += x.x + 0.1f;
      return x;
}

A more fancy extension would be to have a prior and next frame pointer as well.

[Myrsloik]

Achooo

I'm allergic to proprietary crap

[Cary Knoop]

Originally Posted by Myrsloik

Achooo

I'm allergic to proprietary crap

Then use OpenCL.

I think having such an interface would be a great addition to Vapoursynth. And once you have the inferface spec one could write other adapters like for CUDA or other GPU languages.

[HuBandiT]

Originally Posted by Myrsloik

The current filter model of operating on whole frames is also very memory bandwidth inefficient. Adding support for filters that work per scanline/tile will allow for huge improvements since then L3 and L2 cache speed would be the limit and not ram.

Amen! One day perhaps the JIT in VS will be enhanced enough to allow users to write filter chains and then do Expression Templates-type optimization on them, fully exploiting available locality.

(Until then maybe vsFilterScript will get there first.)

[stax76]

VapourSynth is not mentioned in the Wikipedia Frameserver article. I don't have much time and Wikipedia experience.

[feisty2]

vsFilterScript is now ready for general purpose use, and ready to retire vsxx as the modern vs C++ api wrapper.
it has successfully achieved zero cost abstraction, the test filter runs even faster than the corresponding core filter bundled with vaporsynth.
and it comes with much prettier syntax and higher level of abstraction FOR FREE!

[Myrsloik]

R51 released!

Not that exciting. Get ready for the considerably more exciting audio and api development builds soon.

[mastrboy]

Originally Posted by Myrsloik

Get ready for the considerably more exciting audio and api development builds soon.

Nice, I might finally switch over from Avisynth after audio support has been released.

Will there be source filters for audio available when the core gets audio support?

[Myrsloik]

Originally Posted by mastrboy

Nice, I might finally switch over from Avisynth after audio support has been released.

Will there be source filters for audio available when the core gets audio support?

BestAudioSource is already available. How else could I test things?

I encourage the the curious among you to take a look at the audio test builds.

[lansing]

I have problem setting the color of blankclip referenced from a YUV444P16 clip.

clip_yuv16 = core.resize.Bicubic(clip, format=vs.YUV444P16)
color_clip = core.std.BlankClip(clip_yuv16, color=[0, 0, 220])
color_clip.set_output()

This gives me a dark green frame

[Myrsloik]

Originally Posted by lansing

I have problem setting the color of blankclip referenced from a YUV444P16 clip.

clip_yuv16 = core.resize.Bicubic(clip, format=vs.YUV444P16)
color_clip = core.std.BlankClip(clip_yuv16, color=[0, 0, 220])
color_clip.set_output()

This gives me a dark green frame

Sounds right with those values. Why do you think it should be something else?