Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th March 2021, 09:53   #4261  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@Myrsloik
is it safe to delete the user data passed to registerFunction() in Create() like the following?
Code:
auto RegisterFunction(auto&& Signature, auto&& Function) {
    using FunctionType = std::decay_t<decltype(Function)>;
    auto FunctionHandle = new auto{ std::forward<decltype(Function)>(Function) };
    auto Create = [](auto in, auto out, auto FunctionHandle, auto core, auto...) {
        auto& RegisteredFunction = *reinterpret_cast<FunctionType*>(FunctionHandle);
        Console{ out }.Receive(RegisteredFunction(ArgumentList{ in }, CoreProxy{ core }));
        delete &RegisteredFunction;
    };
    ::vsapi->registerFunction(DeduceName(Signature), ExtractParamList(Signature), Create, FunctionHandle, ::PluginHandle);
}

RegisterFunction("f(x: int[])", [](auto args, auto Core) { for (auto y : args["x"]) Core.Print(static_cast<int>(y)); return 42; });
I guess what I am actually asking is that will Create() run on mutiple threads which then leads to the risk of use-after-free or double-free?
feisty2 is offline   Reply With Quote
Old 28th March 2021, 11:50   #4262  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by feisty2 View Post
@Myrsloik
is it safe to delete the user data passed to registerFunction() in Create() like the following?
[code]
...
No. Filter creation can be multithreaded too (mostly happens when you create and destroy filters in a getframe function).
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th March 2021, 12:17   #4263  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
then who deletes the user data? I searched thru avisynth_compat.cpp and didn't find where the WrappedFunction object gets deleted
feisty2 is offline   Reply With Quote
Old 28th March 2021, 12:29   #4264  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by Patman View Post
Hello everybody. I need your help with the following problem:

I have this code in my avs.c:

Code:
    h->bit_depth = h->func.avs_bits_per_component(vi);
    FAIL_IF_ERROR( h->bit_depth < 8 || h->bit_depth > 16, "unsupported bit depth `%d'\n", h->bit_depth );
    if( h->bit_depth & 7 )
    {
        AVS_Value arg_arr[2];
        arg_arr[0] = res;
        arg_arr[1] = avs_new_value_int( 16 );
        const char *arg_name[] = { NULL, "bits" };
        AVS_Value res2 = h->func.avs_invoke( h->env, "ConvertBits", avs_new_value_array( arg_arr, 2 ), arg_name );
        FAIL_IF_ERROR( avs_is_error( res2 ), "couldn't convert to 16 bits: %s\n", avs_as_error( res2 ) );
        res = update_clip( h, &vi, res2, res );
    }
I would now like to store a code with an identical function in my vpy.c, but unfortunately it doesn't work that well. I have the following approach:

Code:
    h->bit_depth = vi->format->bitsPerSample;
    FAIL_IF_ERROR( h->bit_depth < 8 || h->bit_depth > 16, "unsupported bit depth `%d'\n", h->bit_depth );
    FAIL_IF_ERROR( vi->format->sampleType == stFloat, "unsupported sample type `float'\n" );
    if (h->bit_depth & 7)
    {
        VSMap* args = h->vsapi->createMap();
        const char* error;
        VSPlugin* fmtcPlugin = h->vsapi->getPluginById("fmtconv", core);

        h->vsapi->propSetNode(args, "clip", h->node, paReplace);
        h->vsapi->freeNode(h->node);
        const VSFormat* new_format = h->vsapi->registerFormat(vi->format->colorFamily, vi->format->sampleType, 16, vi->format->subSamplingW, vi->format->subSamplingH, core);
        h->vsapi->propSetInt(args, "csp", new_format->id, paReplace);
        VSMap* ret = h->vsapi->invoke(fmtcPlugin, "bitdepth", args);
        error = h->vsapi->getError(ret);
        if (error)
        {
            FAIL_IF_ERROR(1, "failed to convert node to 16 bits: `%s'\n", error);
            h->vsapi->freeMap(args);
            h->vsapi->freeMap(ret);
            return -1;
        }
        h->node = h->vsapi->propGetNode(ret, "clip", 0, NULL);
        h->vsapi->freeMap(ret);
    }
The aim is to consider certain plugins when processing. With avs it is 'ConvertBits' and with vpy 'fmtconv'. Does somebody has any idea?
It looks like it should function. What problem did you encounter?

Things I noticed:
- You're leaking the args VSMap. You should free it after you call invoke.
- getPluginById can return NULL if the plugin you want is not loaded. fmtconv is not included with VapourSynth. You could use the built-in resizer (com.vapoursynth.resize), then you don't really have to check what getPluginById returns.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 28th March 2021, 12:41   #4265  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by feisty2 View Post
then who deletes the user data? I searched thru avisynth_compat.cpp and didn't find where the WrappedFunction object gets deleted
Nobody deletes it. It's mostly a bad inherited avisynth-ism and should probably never be used.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th March 2021, 12:44   #4266  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
R53-RC1
Code:
updated visual studio 2019 runtime version
updated to python 3.9 for windows
fixed length calculation in y4m header (djatom)
added long path support to vspipe (stax76)
fixed crash in text filter with small resolution clips (jackoneill)
fixed calling wrapped functions through python (IFeelBloated)
Finally doing a bit of development again. Note that the included vsrepo has some fixes too.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th March 2021, 13:48   #4267  |  Link
l00t
Where's my loot?
 
Join Date: May 2019
Posts: 63
Thanks for the new release!

With the portable version combined with python 3.9.2 embedded I get
Code:
Failed to initialize VapourSynth environment
(both with vspipe -v and also in vseditor)

Previously I had R52 with 3.8.8 working just fine (portable+embedded). I copied over the fresh files and I get this.

The installed version works fine with the installer version of python 3.9.2...

What did I do wrong? Any suggestions?

I'm on Win10 20H2 x64. Both python and vs are 64-bit variants.

Last edited by l00t; 28th March 2021 at 13:50.
l00t is offline   Reply With Quote
Old 28th March 2021, 13:55   #4268  |  Link
l00t
Where's my loot?
 
Join Date: May 2019
Posts: 63
Hmm, placing vapoursynth.cp39-win_amd64.pyd under Lib\site-packages seems to solve this issue...

Last edited by l00t; 28th March 2021 at 14:00.
l00t is offline   Reply With Quote
Old 28th March 2021, 13:58   #4269  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by l00t View Post
Thanks for the new release!

With the portable version combined with python 3.9.2 embedded I get
Code:
Failed to initialize VapourSynth environment
(both with vspipe -v and also in vseditor)

Previously I had R52 with 3.8.8 working just fine (portable+embedded). I copied over the fresh files and I get this.

The installed version works fine with the installer version of python 3.9.2...

What did I do wrong? Any suggestions?

I'm on Win10 20H2 x64. Both python and vs are 64-bit variants.
Doh, typo in the portable build script so the actual module didn't get included. Redownload it.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th March 2021, 14:02   #4270  |  Link
l00t
Where's my loot?
 
Join Date: May 2019
Posts: 63
Ah, now it works like a charm without messing with precompiled files. Thank you!
l00t is offline   Reply With Quote
Old 28th March 2021, 15:28   #4271  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
Quote:
Originally Posted by Myrsloik View Post
Nobody deletes it. It's mostly a bad inherited avisynth-ism and should probably never be used.
I guess it might be somewhat useful for "bridge" plugins (bridging foreign plugins developed for other video processing software to VS, like core.avs.LoadPlugin), and it allows the user to register stateful lambdas as plugin functions.

I guess I could tie the lifetime of user data to a static variable like the following
Code:
struct GlobalRecyler {
    using RecordType = std::tuple<auto(*)(void*)->void, void*>;
    std::vector<RecordType> Records = {};

    GlobalRecyler() = default;
    GlobalRecyler(const GlobalRecyler&) = delete;
    GlobalRecyler(GlobalRecyler&&) = delete;
    auto& operator=(const GlobalRecyler&) = delete;
    auto& operator=(GlobalRecyler&&) = delete;

    auto Collect(auto Garbage) {
        using GarbageType = std::decay_t<decltype(*Garbage)>;
        Records.push_back({ [](auto PointerToGarbage) { delete reinterpret_cast<GarbageType*>(PointerToGarbage); }, Garbage });
    }

    ~GlobalRecyler() {
        for (auto [Recyler, Garbage] : Records)
            Recyler(Garbage);
    }
};

auto RegisterFunction(auto&& Signature, auto&& Function) {
    using FunctionType = std::decay_t<decltype(Function)>;
    static auto Evil = GlobalRecyler{};
    auto FunctionHandle = new auto{ std::forward<decltype(Function)>(Function) };
    auto Create = [](auto in, auto out, auto FunctionHandle, auto core, auto...) {
        auto& RegisteredFunction = *reinterpret_cast<FunctionType*>(FunctionHandle);
        Console{ out }.Receive(RegisteredFunction(ArgumentList{ in }, CoreProxy{ core }));
    };
    Evil.Collect(FunctionHandle);
    ::vsapi->registerFunction(DeduceName(Signature), ExtractParamList(Signature), Create, FunctionHandle, ::PluginHandle);
}
and the user data will be recycled along with the static variable when the plugin is unloaded, that should work, right?
feisty2 is offline   Reply With Quote
Old 31st March 2021, 10:09   #4272  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by feisty2 View Post

and the user data will be recycled along with the static variable when the plugin is unloaded, that should work, right?
Stop the madness!

You have to ask yourself if you're writing a wrapper to create new good plugins or if it's only a programming exercise to sexually please yourself using naughty code. The correct pattern is for the plugin itself to have an internal filter initialization counter or properly free everything on library unload. That pointer is useless if you have any idea what you're doing when writing code.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 31st March 2021, 14:57   #4273  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
well I'm not doing it just because I can, I'm doing it because I think there is indeed a valid use case. I've excluded quite a few rarely used or deprecated things or things that are not meant to be used by plugin developers from the C API (setMessageHandler, queryCompletedFrame, releaseFrameEarly, arFrameReady, nfIsCache, nfMakeLinear, cmCompat and all Compat formats, also all "map" stuff is invisible to the user). I would have excluded the use of this pointer too if I didn't already find a use case for it.

a while ago @lansing requested an example of a bridge plugin to load Vdubfilters, while I never had the time to finish the example he/she asked for, I did take a quick glance over how it's done in avisynth. there's this configuration object (fdl) representing the state of a virtualdub filter that you must pass to a filter instance, I suppose the skeleton of the bridge plugin would be something like
Code:
struct VirtualDubProxy {
    VirtualDubProxy(auto Arguments, auto& Configurations, auto Core) {
        auto& [ReferenceCounter, VirtualDubFunctionHandles, _] = Configurations;
        ...
    }

    static auto SpecifySignature(auto& Configurations) {
        auto& [_, __, RawSignature] = Configurations;
        ...
        return ... // converts RawSignature to VS signature format
    }

    ....
};

for (auto& x : VirtualDubPlugin.ListFunctions())
    PluginAPI::RegisterFilter<VirtualDubProxy>(x.ZipConfigurations());
apparently RegisterFilter() is just a wrapper for vsapi->registerFunction() and Create(), and the only place that allows me to pass that configuration object seems to be that userData pointer. so I don't know how this is possible without using that pointer. the configuration object will have a static lifetime so it can be accessed by all filter instances across threads, and itself may contain a reference counter and some resources may be released as soon as the reference count drops to 0, I don't understand how any of this is madness.
feisty2 is offline   Reply With Quote
Old 10th April 2021, 14:50   #4274  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
R53-RC2
Code:
r53:
updated visual studio 2019 runtime version
updated to python 3.9 for windows
added scale argument to text filters (AkarinVS)
fixed length calculation in y4m header (djatom)
added long path support to vspipe (stax76)
fixed crash in text filter with small resolution clips (jackoneill)
fixed calling wrapped functions through python (IFeelBloated)
Test it a bit. Probably the final RC so this thing gets released.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 10th April 2021, 16:20   #4275  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
I tested portable with Python 3.9.4, it works fine.

Long path support might not be useful currently because original/vanilla x265 builds don't support long path, most builds are modded, mods from DJATOM, Patman and MeteorRain have a vs reader included, Lighs builds are based on synth readers from DJATOM and MeteorRain, links are here:

https://github.com/staxrip/staxrip/wiki/x265
stax76 is offline   Reply With Quote
Old 20th April 2021, 19:13   #4276  |  Link
unix
Registered User
 
Join Date: Aug 2015
Posts: 47
Guys I have an issue, I installed placebo plugin but vsp editor cant recognize it!

Note: I'm using portable Vapoursynth just for tests
Attached Images
 

Last edited by unix; 21st April 2021 at 11:09.
unix is offline   Reply With Quote
Old 21st April 2021, 17:59   #4277  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
R53 is released!

Code:
updated visual studio 2019 runtime version
updated to python 3.9 for windows
added scale argument to text filters (AkarinVS)
fixed length calculation in y4m header (djatom)
added long path support to vspipe (stax76)
fixed crash in text filter with small resolution clips (jackoneill)
fixed calling wrapped functions through python (IFeelBloated)
Hopefully I'll have time to finish the mythic audio branch this summer.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 22nd April 2021, 14:21   #4278  |  Link
Nico8583
Registered User
 
Join Date: Jan 2010
Location: France
Posts: 851
Thank you Myrsloik ! The Windows portable installation is the same than previous versions (except Python 3.9 instead of 3.8 - Decompress R53 into Python 3.9 embeddable folder) ? Thank you.
Nico8583 is offline   Reply With Quote
Old 23rd April 2021, 08:05   #4279  |  Link
vcmohan
Registered User
 
Join Date: Jul 2003
Location: India
Posts: 890
It is a great convenience that vapousynth input script allows arrays. I am however curious whether an array of arrays for example filt:int[[],[],[],...]: opt; can be used in the code of plugin. If its possible how to get input number of arrays and number of elements in each array.
__________________
mohan
my plugins are now hosted here
vcmohan is offline   Reply With Quote
Old 23rd April 2021, 09:21   #4280  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by vcmohan View Post
It is a great convenience that vapousynth input script allows arrays. I am however curious whether an array of arrays for example filt:int[[],[],[],...]: opt; can be used in the code of plugin. If its possible how to get input number of arrays and number of elements in each array.
No, only 1D arrays are allowed to keep the api simple. If you have a fixed number of dimensions simply add a second argument to specify the row size to get around it. Or implicitly figure out the array dimensions like the convolution filter.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.