Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th March 2022, 21:02   #1  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
vs-dpir onnxruntime portable?

Trying to get a portable Vapoursynth running with onnxruntime to use latest vs-dpir and vs-realesrgan:

Okay, here's how I setup Vapoursynth to get it working for CUDA (atm.):
  • I created a new empy Vapoursynth-folder
  • downloaded 'Windows embeddable package (64-bit)' from https://www.python.org/downloads/release/python-3912/
  • extracted the Python download into the download into the 'Vapoursynth'-folder
  • downloaded 'VapourSynth64-Portable-R57' from https://github.com/vapoursynth/vapoursynth/releases
  • extracted the Vapoursynth portable download into the 'Vapoursynth'-folder
  • downloaded get-pip.py from https://bootstrap.pypa.io/get-pip.py and save it into the 'Vapoursynth'-folder
  • opened a 'Windows Command Prompt'-window and navigate into the 'Vaporusynth'-folder
  • installed pip by calling :
    Code:
    pyhton get-pip.py
  • opened the python39._pth in a text addition and added the following to lines above anything else in that file and saved the file
    Code:
    Scripts
    Lib\site-packages
  • installed VSGAN
    Code:
    python -m pip install vsgan
    python -m pip install torch===1.11.0+cu113 torchvision==0.12.0 -f https://download.pytorch.org/whl/torch_stable.html
  • installed BASICVSR++
    Code:
    python -m pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.11/index.html
    python -m pip install tqdm
    python -m pip install opencv-python
    python -m pip install --upgrade vsbasicvsrpp
    python -m vsbasicvsrpp
    installted RIFE
    Code:
    python -m pip install --upgrade vsrife
  • installed SWINIR
    Code:
    python -m pip install --upgrade vsswinir
    python -m vsswinir
  • installed DPIR and onnxruntime-gpu
    Code:
    python -m pip install --upgrade vsdpir
    python -m pip install --upgrade onnxruntime-gpu
  • from cudnn-11.4-windows-x64-v8.2.4.15.zip and NVIDIA CUDA SDK 11.4.1 runtimes
    I copied:
    cublas64_11.dll
    cublasLt64_11.dll
    cudart64_110.dll
    cudnn64_8.dll
    cudnn_cnn_infer64_8.dll
    cudnn_ops_infer64_8.dll
    cufft64_10.dll
    cufftw64_10.dll
    into Vapoursynth/Lib/site-packages/onnxruntime/capi and then uninstalled the sdk and cudnn and cleared all __pycache__-folders inside the Vapoursynth subfolders.
  • downloaded the vsdpir modules
    Code:
    python -m vsdpir
  • installed REALESRGAN (which also uses onnxruntime)
    Code:
    python -m pip install --upgrade vsrealesrgan
    python -m vsrealesrgan

When I add the path to "Vapoursynth/Lib/site-packages/onnxruntime/capi" where the dlls are to my systemwide PATH variable vsdpir works fine.
(I can also move the dlls into a separate folder, as long as that folder is listed in the PATH variable, it works.)

Problem is: I want it portable without having to change the PATH variable on the system.

I tried adding:
Code:
import os
import site
# Import scripts folder
os.environ["PATH"] += site.getsitepackages()[0]+'/Lib/site-packages/onnxruntime/capi'
to the start of my Vapoursynth scripts, but that didn't help.
I also tried to expand the PATH, just for the process that runs vsViewer (which I use to open the script), but that didn't help either.

=> does anyone have an idea how to get the onnxruntime working without having to adjust the PATH variable or copy the dlls to a path that is already in the PATH variable?

Cu Selur

Ps.: I'm struggling with this for a few days and since I can't find a solution, I thought may be someone here knows a way.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 31st March 2022, 02:12   #2  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
You could manually load these dlls in .vpy by
Code:
from ctypes import WinDLL
WinDLL(dll1)
WinDLL(dll2)
...
The ordering matters.
WolframRhodium is offline   Reply With Quote
Old 31st March 2022, 04:10   #3  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Thanks, I will give it a try.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 31st March 2022, 04:15   #4  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Code:
import os
import site
# load libraries for onnxruntime
from ctypes import WinDLL
path = site.getsitepackages()[0]+'/onnxruntime_dlls/' 
WinDLL(path+'cublas64_11.dll')
WinDLL(path+'cudart64_110.dll')
WinDLL(path+'cudnn64_8.dll')
WinDLL(path+'cudnn_cnn_infer64_8.dll')
WinDLL(path+'cudnn_ops_infer64_8.dll')
WinDLL(path+'cufft64_10.dll')
WinDLL(path+'cufftw64_10.dll')
seems to work! (moved the libraries to a separte folder 'onnxruntime_dlls')
Hurray!

Thanks !

Cu Selur
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 31st March 2022 at 04:29.
Selur is offline   Reply With Quote
Old 31st March 2022, 08:38   #5  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Maybe calling this function is more elegant?
https://docs.microsoft.com/en-us/win...tdlldirectoryw
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 31st March 2022, 13:48   #6  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
@mysloik: you are right using
Code:
import site
# Import libraries for onnxruntime
import ctypes
path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
is more elegant.

@all:
For TensorRT users:
Code:
import site
# Import libraries for onnxruntime
from ctypes import WinDLL
path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
WinDLL(path+'cublas64_11.dll')
WinDLL(path+'cudart64_110.dll')
WinDLL(path+'cudnn64_8.dll')
WinDLL(path+'cudnn_cnn_infer64_8.dll')
WinDLL(path+'cudnn_ops_infer64_8.dll')
WinDLL(path+'cufft64_10.dll')
WinDLL(path+'cufftw64_10.dll')
WinDLL(path+'nvinfer.dll')
WinDLL(path+'nvinfer_plugin.dll')
WinDLL(path+'nvparsers.dll')
WinDLL(path+'nvonnxparser.dll')
seems to work.

Cu Selur
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 1st April 2022 at 14:15.
Selur is offline   Reply With Quote
Old 31st March 2022, 18:58   #7  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Argh,... vs-dpir in combination with vs-rife fails.

When using just vs-rife:
Code:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'C:\Users\Selur\Desktop\VTS_01_1.VOB'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\VTS_01_1.VOB using D2VSource
clip = core.d2v.Source(input="E:/Temp/vob_941fdaaeda22090766694391cc4281d5_853323747.d2v")
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=10, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 10
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = core.misc.SCDetect(clip=clip,threshold=0.150)
from vsrife import RIFE
# adjusting color space from YUV420P8 to RGBS for VsTorchRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# adjusting frame count&rate with RIFE (torch)
clip = RIFE(clip, multi=3, device_type='cuda', device_index=0) # new fps: 20
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 30.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1)
# Output
clip.set_output()
everything works.
But when I add latest vs-dpir:
Code:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
import os
import site
# Import libraries for onnxruntime
from ctypes import WinDLL
path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
WinDLL(path+'cublas64_11.dll')
WinDLL(path+'cudart64_110.dll')
WinDLL(path+'cudnn64_8.dll')
WinDLL(path+'cudnn_cnn_infer64_8.dll')
WinDLL(path+'cudnn_ops_infer64_8.dll')
WinDLL(path+'cufft64_10.dll')
WinDLL(path+'cufftw64_10.dll')
WinDLL(path+'nvinfer.dll')
WinDLL(path+'nvinfer_plugin.dll')
WinDLL(path+'nvparsers.dll')
WinDLL(path+'nvonnxparser.dll')
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'C:\Users\Selur\Desktop\VTS_01_1.VOB'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\VTS_01_1.VOB using D2VSource
clip = core.d2v.Source(input="E:/Temp/vob_941fdaaeda22090766694391cc4281d5_853323747.d2v")
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=10, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 10
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
from vsdpir import DPIR
# adjusting color space from YUV420P8 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=15.000, task="denoise", provider=1, device_id=0)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
clip = core.misc.SCDetect(clip=clip,threshold=0.150)
from vsrife import RIFE
# adjusting color space from YUV444P16 to RGBS for VsTorchRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# adjusting frame count&rate with RIFE (torch)
clip = RIFE(clip, multi=3, device_type='cuda', device_index=0) # new fps: 20
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 30.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1)
# Output
clip.set_output()
I get:
Code:
Python exception: [WinError 127] Die angegebene Prozedur wurde nicht gefunden. Error loading "I:\Hybrid\64bit\Vapoursynth\Lib/site-packages\torch\lib\cudnn_cnn_train64_8.dll" or one of its dependencies.
Using just vs-dpir:
Code:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
import os
import site
# Import libraries for onnxruntime
from ctypes import WinDLL
path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
WinDLL(path+'cublas64_11.dll')
WinDLL(path+'cudart64_110.dll')
WinDLL(path+'cudnn64_8.dll')
WinDLL(path+'cudnn_cnn_infer64_8.dll')
WinDLL(path+'cudnn_ops_infer64_8.dll')
WinDLL(path+'cufft64_10.dll')
WinDLL(path+'cufftw64_10.dll')
WinDLL(path+'nvinfer.dll')
WinDLL(path+'nvinfer_plugin.dll')
WinDLL(path+'nvparsers.dll')
WinDLL(path+'nvonnxparser.dll')
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'C:\Users\Selur\Desktop\VTS_01_1.VOB'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\VTS_01_1.VOB using D2VSource
clip = core.d2v.Source(input="E:/Temp/vob_941fdaaeda22090766694391cc4281d5_853323747.d2v")
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip, mode=7, rate=10, dupThresh=0.04, vidThresh=3.50, sceneThresh=15.00)# new fps: 10
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
from vsdpir import DPIR
# adjusting color space from YUV420P8 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=15.000, task="denoise", provider=1, device_id=0)
# adjusting output color from: RGBS to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 10.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=10, fpsden=1)
# Output
clip.set_output()
works fine.

-> Doe anyone have an idea how to could fix this?

Cu Selur

Ps.: also posted it to the vs-rife bug tracker https://github.com/HolyWu/vs-rife/issues/15
PPs.: Same issue happens when having vs-realesrgan and vs-rife in the same script.
PPPs.: using
Code:
import site
# Import libraries for onnxruntime
import ctypes
path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
instead of loading the libraries one at a time doesn help.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 1st April 2022 at 14:14.
Selur is offline   Reply With Quote
Old 1st April 2022, 17:05   #8  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
... wrong thread
_Al_ is offline   Reply With Quote
Old 2nd April 2022, 19:51   #9  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
No clue, what I did to my system, but now vs-dpir and vs-rife work together,...
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 24th April 2022, 16:17   #10  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Small warning to others: Current onnxruntime 3.11 (https://github.com/microsoft/onnxruntime) does not support Python 3.10 so if you want to use vs-dpir or vs-realesrgan do not switch to Vapoursynth R58 (which uses Python 3.10) until onnxruntime 3.12 (which should support Pyhton 3.10) is out.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 24th April 2022, 18:07   #11  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
I thought R58 supports 3.10 and 3.8 ?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 24th April 2022, 18:12   #12  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Yes, you are right. One might get it working with Python 3.8, but since I used 3.9 before with Vapoursynth R53 (switched to 3.10 since 3.9 isn't supported by Vapoursynth R54), I don't really plan to downgrade Python.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 25th April 2022, 21:26   #13  |  Link
knumag
Registered User
 
Join Date: Oct 2018
Posts: 7
Had to change it to this, to make it work with R57 and python 3.9:

Code:
python -m pip install torch===1.11.0+cu113 torchvision==0.12.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html
python -m pip install vsrife==1.3.0
python -m pip install vsdpir==1.6.0
python -m pip install vsrealesrgan==1.2.0
With R58 and python 3.8 I got the newest versions of vsrife, vsdpir and vsrealesrgan with onnx to work.

Same FPS when converting in both installs, but GPU use is 90-100% with latest versions and 10% with the above versions..
Do not understand why it is so slow and why the GPU usage differs so much when fps is the same.
With Topaz and other similar programs it is around 10x as fast as this.
And also got sharper ouput from VSGAN with same model. Very strange. Some other filters working there?

Last edited by knumag; 25th April 2022 at 21:31.
knumag is offline   Reply With Quote
Old 26th April 2022, 17:39   #14  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
With R58 and python 3.8 I got the newest versions of vsrife, vsdpir and vsrealesrgan with onnx to work.
Good to know.

No clue about speed&co
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 10th June 2022, 13:47   #15  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
btw. R58&R59 with Python 3.10 work fine with the ort-nightly, so instead of using:
Code:
python -m pip install --upgrade onnxruntime-gpu
using
Code:
python -m pip install --upgrade ort-nightly
works fine so far.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 18th June 2022, 07:02   #16  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 303
...a Colab Notebook would be great...

https://github.com/AlphaAtlas/VapourSynthColab
https://github.com/kodxana/VapourSynthColab
__________________
HYbrid Multimedia Production Suite project @ GitHub
PatchWorKs is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.