Avisynth / Vapoursynth - Package Management System

[ChaosKing]

Hi,

I've been thinking lately about how cool it would be, if Avisynth/Vapoursynth would have its own package management system.
Or more precisely a downloader which also downloads all necessary dependencies for you. Like apt-get etc, but far more simple like the downloader in megui, only better

I already done some testing and my current "solution" would look like this:
- A package.json file (structure would be similar to package.json files from https://npmjs.org/). This file holds all information about the author, maintaner, dependencies...

An example {
    "filtername": "flash3kyuu_deband",  // I like your filter :)
    "version": "1.5",
    "category": "Restoration Filters",
    "sub-category": "Debanding",
    "filetype": "dll",
    "platform": "x86",
    "usesGPU": "false",
    "description": "This avisynth plugin debands the video by replacing banded pixels with average value of referenced pixels, and optionally add grain (random dithering) to them.",
    "mediawiki": "http://avisynth.org/mediawiki/flash3kyuu_deband",
    "homepage": "http://forum.doom9.org/showthread.php?t=161411",
    "doomX-link": "http://forum.doom9.org/showthread.php?t=161411",
    "keywords": [],
    "plugin-author": {
        "name": "SAPikachu",
        "email": "idk@somefantasymail.com",
        "url": "http://www.google.com"
    },
    "package-maintainer": {
        "name": "ChaosKing",
        "email": "thechaoscoder@gmail.com",
        "url": "http://chaosking.de/avisynth-filter-db"
    },
    "dependencies": {  // I know flash3kyuu_deband doesn't have any dependencies :P
        "Fastlinedarkenmod": "1.3", 
        "sangnom": "1.0"
    }
}

- The filter itself + doc ( + maybe source code too) will be compressed with zip/7zip to keep the file size small.

/bin/filter.dll
/doc/{filtername}/filter.html
/src/{filtername}/[sourcecode]
/some other folder/

- The package will be uploaded to a website, where the package information will be stored in a mysql database. (but the package.json will stay in the package). You can search online ...
- An update file will be generated which holds the download url, filter name and version of every uploaded package, so the desktop app knows that to download.
- The desktop app downloads all necessary packages, extracts them and save them in a folder of your choice (my filters are not stored in the avisynth folder)

Structure:
/[filters]
/doc/{filtername}/doc.html, txt ... package.json
/src/{filtername}/[source code]

Basically it consists of two parts. The desktop app (downloads/extracts/updates your files) and a web server + db with all packages + an generated update file.

So, that do you think of this idea. Is it a good idea or waste of time?
I'm open to all suggestions.


Problems:
- Not every filter has a version number. Some uses date, some nothing :O
- How can I trust a package or verify its creator? (signing files with gpg + list of trust?)
- much work xD


// Interesting Sites
coapp.org (THE apt-get for windows, but still alpha/beta)
chocolatey.org
npmjs.org <-- seems to be a good solution, but needs a CouchDB and other stuff

[Adub]

I have to admit, this certainly is a cool idea.

Writing the client is going to be the interesting part, especially because you want to support multiple OSes (especially now with Vapoursynth). It will be very important for the user to be able to specify a custom install location, as a lot of people install them in weird locations.

Still, something that runs across systems and has support for Avisynth/Vapoursynth/Avisynth for Linux/Future Scripting Language is a very cool idea.

[qyot27]

In the interests of being cross-platform, I'd suggest using actual tarballs, not zip or 7zip archives. There's a much longer history of package management dealing with either tarballs (usually for source, sometimes for standalone binary distro) and distinct packaging formats (for binaries; deb, rpm, etc.). If LZMA2 is desired, it's not like tar.xz is at all exotic anymore (and tar.gz is nigh-ubiquitous); or even tar.bz2, for that matter.

There's going to be an issue either way: Windows doesn't natively have an untar program, and OSX and Linux usually only go as far as zip (and that can be kind of iffy, depending on which distro you're talking about here). But due to the typical conventions of Windows software distribution, it's probably a lot simpler/less of a hassle to just include a copy of tar itself with the package manager than having to resolve the necessary dependencies in the *nixen first.

Not to mention that for OSX/Linux you're going to need to deal with filesystem permissions (or even the possibility of symlinks), and so tarballs are more well-suited to the task.

[Myrsloik]

Note that I'm not against the idea I just think it's kinda complicated. A big compressed file could accomplish most of it.

Personally I intend to bundle most open source plugins in the vapoursynth installer in the future. It just seems like the most reasonable way to do things. Every not too buggy/insanely big filter is welcome. Scripts too I guess once that gets going.

As for not windows, binaries are such a mess to distribute you'll just have to do it the source code way. The only exception being osx.

[ChaosKing]

I don't think the client would be much of a problem. I thought about to code it in Qt5 with QML or c# (mono)... every external software will be bundled, like in megui -> portable

It will be very important for the user to be able to specify a custom install location, as a lot of people install them in weird locations.

like me

@Myrsloik, but the user still would need to download newer filters himself. Except you offer regular updates.

My goal is that a filter developer could/should create a package himself. Once he has created a package, all he needs to do is change the version string in his next release and repack it, done! ... One minute effort. Every one is happy ^^


The idea of a big compressed file is also interesting. Then you could also offer incremental file updates to keep the download size small.

[ChaosKing]

Ideally a filter developer would do this himself and set the package config accordingly. Since he knows his filter best. (What would be the case for newer filters)
For older ones, well I guess the most popular plugins will be added quick (by me)

[Adub]

Indeed, for Windows users this will definitely require a GUI of some kind, simply because of historical reasons.

Linux users (like myself) should certainly be able to get by without a GUI, but I'm sure one would be appreciated nonetheless.

OSX, it's probably 50/50. As a Mac Book Pro owner, I use Homebrew enough for package management that I could easily do without the GUI. But again, considering the history of graphical applications in OS X and Windows, a GUI would likely be appreciated.

I believe that the Qt QML approach is a strong bet, especially considering that a number of systems have these libraries installed already, versus something like Mono. In addition, Vapoursynth already uses some of the Qt libraries, so there is a nice parallel there.

[Caroliano]

I wonder if something like ZeroInstall could be adapted for a plugin system...

The idea of big compressed file is simpler, but more difficult to maintain up to date. And it is already kinda being done in avisynth with filters like QTGMC that have lots of dependencies.

[ChaosKing]

Looks promising, will look into it. thx for the link