ffmpeg using .vpy scripts directly one day, like avisynth ?

[amichaelt]

Originally Posted by MysteryX

Isn't this whole conversation because it isn't enabled by default?

Even if it was, it not being enabled by default has nothing to do with your FUD about Vapoursynth being a potential a malware vector. Plenty of ffmpeg features are optional and disabled by default. For example, Avisynth support is also disabled by default.

[amichaelt]

Also a malicious Avisynth plugin can be written to delete everything off your hard druve, download malware, etc. What you're talking about is what any piece of malicious arbitrary-code can do. Avisynth also has no sandboxing to prevent this. So you're really no more safe with Avisynth.

And if you use autoload, a malware-ridden plugin could have its init function called and do malicious things without you even needing to ever explicitly call it.

[MysteryX]

What about this? I haven't made anything up, but I understand their decision.

Originally Posted by hydra3333

Here's the ffmpeg commit
https://git.ffmpeg.org/gitweb/ffmpeg...182aa0b3778b79

--enable-vapoursynth in the ffmpeg configure

[foxyshadis]

Originally Posted by MysteryX

There are indeed risks with downloading DLLs from unknown sources. However, one needs to do special work to run code in a DLL, it won't happen automatically, and most viruses within DLLs will be auto-detected by a good anti-virus.

Running a Python script by double-clicking on a file (for users who may not even know what VapourSynth is), however, can seriously limit the adaptation of VapourSynth. If that's the reason FFMPEG hasn't added native support for it, then I fully understand. In a business or production environment, have to be *VERY* careful where the scripts are coming from and where they are running. I don't think many people realize that. I also don't think Kaspersky will scan the Python raw script for malicious code.

I would say that this, combined with the lack of audio support, are the 2 things most limiting the adaptation of VapourSynth.

In Avisynth, all you have to do is get someone to load your DLL by saying it does something cool, and it turns out that pretty much everyone would. You can't even inspect that without being an expert programmer/reverser.

[ChaosKing]

And If you are really evil you could try to overwrite files with http://avisynth.nl/index.php/ImageWriter

[MysteryX]

I see a clear difference between having security risks on a development machine for someone using Avisynth/VapourSynth and downloading various DLLs, compared to a production server using FFMPEG without any awareness of what VapourSynth/Avisynth even are.

If you take risks on your local machine, that's one thing.

If a server environment is at risk simply by using FFMPEG because it auto-loads a risky script engine (without admin awareness), that's something else.

[poisondeathray]

Using Wolfberry's FFMpeg build with vpy support , direct vapoursynth input is about 50% speed compared to vspipe to the same ffmpeg . I would have expected marginally faster for direct vpy demuxer. Script is just QTGMC . I tried specifying different ffmpeg -threads values as an input argument. Any ideas, or am I doing something wrong? Could it be a cache issue ?

https://forum.doom9.org/showthread.p...31#post1866931

vspipe --y4m input.vpy - | "ffmpeg" -f yuv4mpegpipe -i - -an -f null NUL

"ffmpeg" -f vapoursynth -i input.vpy -an -f null NUL

EDIT #1: but another test of just reading a MP4 video with lsmash (no other filters, just source filter), direct vapoursynth is about 1.4-1.5x faster than vspipe to ffmpeg

EDIT #2: tested other source filters too lsmash, ffms2, other file types (mpeg2, avc); pretty consistent observation - direct read with only source filter is faster, but as soon as you add any additional filter in script (not just QTGMC, it can be anything like a denoiser, or even a simple resize only) it becomes slower than vspipe method at about 50% speed

Is it specific to this ffmpeg build, or do other people get different results with their own build ?

[HolyWu]

Originally Posted by poisondeathray

Is it specific to this ffmpeg build, or do other people get different results with their own build ?

It seems that there is something wrong with wm4's vapoursynth demuxer implementation. Below ffmpeg_1 uses the official implementation, while ffmpeg_2 uses the implementation from Stephen.

Test1

from vapoursynth import core
core.max_cache_size = 12 * 1024
clip = core.ffms2.Source(r'test.mp4')
clip.set_output()

>vspipe -y test.vpy - | ffmpeg_1 -hide_banner -benchmark -f yuv4mpegpipe -i - -f null -
Input #0, yuv4mpegpipe, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1920x1080, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
Output 1250 frames in 7.06 seconds (177.05 fps)24.20 bitrate=N/A speed=3.68x
frame= 1250 fps=181 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=3.62x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=0.109s stime=4.516s rtime=6.912s
bench: maxrss=16848kB


>ffmpeg_1 -hide_banner -benchmark -f vapoursynth -i test.vpy -f null -
Input #0, vapoursynth, from 'test.vpy':
  Duration: 00:00:25.00, start: 0.000000, bitrate: 0 kb/s
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
frame= 1250 fps=238 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=4.76x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=27.969s stime=0.312s rtime=5.256s
bench: maxrss=975760kB


>ffmpeg_2 -hide_banner -benchmark -f vapoursynth -i test.vpy -f null -
Input #0, vapoursynth, from 'test.vpy':
  Duration: 00:00:25.00, start: 0.000000, bitrate: 0 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
frame= 1250 fps=201 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=4.02x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=34.578s stime=3.328s rtime=6.224s
bench: maxrss=984464kB

Test2

from vapoursynth import core
core.max_cache_size = 12 * 1024
clip = core.ffms2.Source(r'test.mp4')
clip = clip.std.BoxBlur(hradius=2, vradius=2).misc.AverageFrames(weights=[1] * 5)
clip.set_output()

>vspipe -y test.vpy - | ffmpeg_1 -hide_banner -benchmark -f yuv4mpegpipe -i - -f null -
Input #0, yuv4mpegpipe, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1920x1080, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
Output 1250 frames in 19.06 seconds (65.57 fps)24.80 bitrate=N/A speed=1.34x
frame= 1250 fps= 66 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=1.33x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=0.203s stime=4.469s rtime=18.845s
bench: maxrss=16848kB


>ffmpeg_1 -hide_banner -benchmark -f vapoursynth -i test.vpy -f null -
Input #0, vapoursynth, from 'test.vpy':
  Duration: 00:00:25.00, start: 0.000000, bitrate: 0 kb/s
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
frame= 1250 fps= 24 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=0.478x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=84.891s stime=0.922s rtime=52.273s
bench: maxrss=1152720kB


>ffmpeg_2 -hide_banner -benchmark -f vapoursynth -i test.vpy -f null -
Input #0, vapoursynth, from 'test.vpy':
  Duration: 00:00:25.00, start: 0.000000, bitrate: 0 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 50 fps, 50 tbr, 50 tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080, q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.35.100 wrapped_avframe
frame= 1250 fps= 68 q=-0.0 Lsize=N/A time=00:00:25.00 bitrate=N/A speed=1.37x
video:654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=137.797s stime=3.359s rtime=18.304s
bench: maxrss=1189088kB

[qyot27]

Post #26 in this thread linked to a copy of it on pastebin.

I think I'll probably have to switch to it also, since I could never seem to get the official implementation to cross-compile right, despite following the steps provided earlier to get the Python lib encapsulated correctly.

[poisondeathray]

Thanks Wolfberry ; does this recent one have wm4 or Stephen patch ?

[Wolfberry]

I'm sure this build uses Stephen's implementation, but I haven't done any tests.

[poisondeathray]

Originally Posted by Wolfberry

I'm sure this build uses Stephen's implementation, but I haven't done any tests.

Some quick tests - in terms of speed wise it seems to be working as expected

Why isn't Stephen implementation the official one ?

[Pat357]

Originally Posted by qyot27

Post #26 in this thread linked to a copy of it on pastebin.

I think I'll probably have to switch to it also, since I could never seem to get the official implementation to cross-compile right, despite following the steps provided earlier to get the Python lib encapsulated correctly.

Unfortunately I can't help you with this specific issue, but I have a request :
could you consider to make a new FFMS2 compile with the libaom decoder replaced by dav1d ?
On relative simple scripts, the just decoding from AV1 takes a lot of CPU with libaom, while dav1d is at least 4x faster on modern CPU's.

With the following videos, I can't get even real-time speed when using liboam :

https://www.elecard.com/storage/vide..._13.9mbps.webm

https://www.elecard.com/storage/vide..._22.7mbps.webm

[poisondeathray]

Originally Posted by Pat357;1873061I have a request :
could you consider to make a new FFMS2 compile with the libaom decoder replaced by dav1d ?
On relative simple scripts, the just decoding from AV1 takes a lot of CPU with libaom, while dav1d is at least 4x faster on modern CPU's.

With the following videos, I can't get even real-time speed when using liboam :

[url

https://www.elecard.com/storage/video/Stream3_AV1_4K_13.9mbps.webm[/url]

https://www.elecard.com/storage/vide..._22.7mbps.webm

Wolfberry posted ffms2 builds with libdav1d in the other thread

https://forum.doom9.org/showthread.php?t=176198

FFMS2:

ffmpeg 4.2-dev N-93683-g163bb087f8
libdav1d 0.2.2-0d7aa95d
libopenjpeg 2.3.1
mbedTLS 2.16.1

[qyot27]

Originally Posted by Pat357

Unfortunately I can't help you with this specific issue, but I have a request :
could you consider to make a new FFMS2 compile with the libaom decoder replaced by dav1d ?
On relative simple scripts, the just decoding from AV1 takes a lot of CPU with libaom, while dav1d is at least 4x faster on modern CPU's.

You mean like the builds I released in January and March of this year?

[hydra3333]

Originally Posted by qyot27

... since I could never seem to get the official implementation to cross-compile right, despite following the steps provided earlier to get the Python lib encapsulated correctly.

Originally Posted by Wolfberry

ffmpeg-4.2-dev-N-93696-g45048ece81-win64-static

ffmpeg version N-93696-g45048ece81 Copyright (c) 2000-2019 the FFmpeg developers

built with gcc 9.0.1 (Built by Wolfberry) 20190426 (prerelease)

configuration: --enable-amf --enable-avisynth --enable-bzlib --enable-cuda --enable-d3d11va --enable-ffnvcodec --enable-frei0r --enable-gray --enable-iconv --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libcdio 
--enable-libdav1d --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-liblensfun --enable-libmfx --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband 
--enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtesseract --enable-libvidstab --enable-libvmaf --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg 
--enable-libzmq --enable-lzma --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-openssl --enable-pthreads --enable-sdl2 --enable-vapoursynth --enable-zlib --enable-gpl --enable-version3

Would it be possible to post the build scripts ? That'd likely help the many.

[richardpl]

That build is unsafe for using.

[Wolfberry]

Do you mean the auto-detection of .vpy extension?

That's in the original implementation found at post #26

I can try revert that part to the official implementation by wm4.

Or do you mean something else?

If you feel unsafe then don't use it. It's simple

[Pat357]

Originally Posted by richardpl

That build is unsafe for using.

Why is that ?

[hydra3333]

Thanks !