vs-basicvsr

[Selur]

Cuda plugins very slow now

Not really Cuda, more like most of the ai model based stuff.

[kedautinh12]

You forgot "Cu Selur"

[Selur]

¯\_(ツ)_/¯

[Selur]

A few more examples of vs-femasr:
not bad:
https://imgsli.com/MTQ1ODI3
https://imgsli.com/MTQ1ODI4
https://imgsli.com/MTQ1ODI5
https://imgsli.com/MTQ1ODM1
nice detail recovery, but partially too sharp (too much contrast) for my liking, and if you have noise&co that might get enhanced too.
https://imgsli.com/MTQ1ODM5

Cu
Selur

[Selur]

High Performance GMFSS with RIFE and GAN for Anime Video Frame Interpolation: https://github.com/HolyWu/vs-gmfss_union

[kedautinh12]

Wow, VS development very fast. When i think Avs+ can reach to VS but after Pytorch, NumPy releases VS development more than Avs+ very much

[ChaosKing]

Some quick performance numbers with RTX 3070 TI & 720x480 clip (noisy source):
tested with vsedit, so number should be slighlty higher bcs of the editors small overhead.

gmfss_union(clip) # ~14fps
gmfss_union(clip, trt=True) # ~17fps, tensor cache took a long time to build
gmfss_union(clip, num_streams=2) # ~17.5fps
gmfss_union(clip, num_streams=3) # ~18.9fps
gmfss_union(clip, trt=True, num_streams=2) # ~21.5fps
gmfss_union(clip, trt=True, num_streams=3) # ~22.5fps

[Selur]

What speed do you get with RIFE and the anime model?

[ChaosKing]

https://github.com/HolyWu/vs-rife runs with 175 fps
clip = RIFE(clip)

https://github.com/HomeOfVapourSynth...FE-ncnn-Vulkan
rife.RIFE(clip, model=3) # ~78fps

Look at this perfect frame
https://i.imgur.com/r2GAnRI.jpeg

[Selur]

Okay, so it's really slow.
(for RIFE enabling sc and setting sc_threshold to something like 0.1 might help a bit)

...

Played around with it and probably won't add this to Hybrid since I don't see any gain from it over RIFE&co atm.

[~ VEGETA ~]

tried installing vsbasicvsr++ but got this error: https://pastebin.com/6VWwaXRd

how can i solve it?

[Selur]

Probably an issue with the cuda dependencies,
here's what I did last time I setup the environment:

• I created a new empy Vapoursynth-folder
• downloaded 'Windows embeddable package (64-bit)' from https://www.python.org/downloads/release/python-3108/ (3.11 isn't supported atm. by Vapoursynth)
• extracted the Python download into the download into the 'Vapoursynth'-folder
• downloaded 'VapourSynth64-Portable-R60.7z' from https://github.com/vapoursynth/vapoursynth/releases
• extracted the Vapoursynth portable download into the 'Vapoursynth'-folder
• downloaded get-pip.py from https://bootstrap.pypa.io/get-pip.py and save it into the 'Vapoursynth'-folder
• opened a 'Windows Command Prompt'-window and navigate into the 'Vaporusynth'-folder
• installed pip by calling :
  

  pyhton get-pip.py

• opened the python310._pth in a text addition and added the following to lines above anything else in that file and saved the file
  

  Scripts
  Lib\site-packages

• installed VSGAN
  • python -m pip install vsgan
  • python -m pip install torch===1.13.0+cu117 torchvision==0.14.0 -f https://download.pytorch.org/whl/torch_stable.html
  • installed BASICVSR++
    

    python -m pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.13/index.html
    python -m pip install tqdm
    python -m pip install opencv-python
    python -m pip install --upgrade vsbasicvsrpp
    python -m vsbasicvsrpp

• installted RIFE
  • downloaded https://github.com/HolyWu/vs-realesr...5.2.2_win64.7z (there's a newer versio now)
    
  • extractedtensorrt-8.5.2.2-cp310-none-win_amd64.whl into my Vapoursynth folder
  • python -m pip install tensorrt-8.5.2.2-cp310-none-win_amd64.whl
  • deleted tensorrt-8.5.2.2-cp310-none-win_amd64.whl
  • python -m pip install --upgrade vsrife
  • created a folder 'vsrife_dependencies' inside the Vapoursynth folder and copied the other files form the CUDA-11.7_cuDNN-8.6.0_TensorRT-8.5.1.7_win64.7z into it. This folder need to be added to when using vs-rife using:
    

    import site
    # add vs-rife dependencies to the path
    path = site.getsitepackages()[0]+'/torch_dependencies/'
    path = path.replace('\\', '/')
    import os
    os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]

  • downloaded the models using:
    

    set PATH=i:\Hybrid\64bit\Vapoursynth\torch_dependencies\bin;%PATH
    python -m vsrife

  • installed SWINIR
    

    python -m pip install --upgrade vsswinir
    python -m vsswinir

• installed DPIR and onnxruntime-gpu
  

  python -m pip install --upgrade vsdpir
  python -m pip install --upgrade onnxruntime-gpu
  python -m vsdpir

• created a onnxruntime_dlls folder inside the Vapoursynth folder.
  I copied from cudnn-11.4-windows-x64-v8.2.4.15.zip and NVIDIA CUDA SDK 11.4.1 runtimes the following files into the folder:
  • cublas64_11.dll
  • cublasLt64_11.dll
  • cudart64_110.dll
  • cudnn_cnn_infer64_8.dll
  • cudnn_ops_infer64_8.dll
  • cudnn64_8.dll
  • cufft64_10.dll
  • cufftw64_10.dll
  • msvcp140.dll
  • nvinfer.dll
  • nvinfer_builder_resource.dll
  • nvinfer_plugin.dll
  • nvonnxparser.dll
  • nvparsers.dll
  • vcruntime140.dll
  • vcruntime140_1.dll
• When using vsdir or realesrgan in a Vapoursynth script:
  

  # Import libraries for onnxruntime
  import site
  import ctypes
  path = site.getsitepackages()[0]+'/onnxruntime_dlls/'
  ctypes.windll.kernel32.SetDllDirectoryW(path)

  needs to be added to the script to load the dlls.
• installed REALESRGAN (which also uses onnxruntime)
  

  python -m pip install --upgrade vsrealesrgan
  python -m vsrealesrgan

• Installed HINet
  

  python -m pip install --upgrade vshinet
  python -m vshinet

Cu Selur

Ps.: also send you a link to my current Hybrid dev and torch addon, going to bed now. You might want to checkout

[Selur]

Anyone managed to get BasicVSR++ v2.0.0 running in a portable Vapoursynth setup?
I'm stuck https://github.com/HolyWu/vs-basicvsrpp/issues/29 installing mmcv-full.

Cu Selur

[Selur]

HolyWu could help, using:

python -m pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
python -m pip install -U openmim
python -m mim install "mmcv-full>=1.7.1"
python -m pip install -U vsbasicvsrpp
python -m vsbasicvsrpp

worked

Cu Selur

[Selur]

With the release of Vapoursynth R62, it's either Python 3.11 or Python 3.8. (Python 3.8 is still there to support old Windows versions and Python 3.10 support was dropped not to having to support multiple Python versions.)
So Python 3.11 is probably what most folks want to go for, but since there are no cp311 pytorch versions over at https://download.pytorch.org/whl/torch/ for Windows I was wondering if someone got a way to get all the ai stuff working with Python 3.11 and Vapoursynth R62?

Cu Selur

[Selur]

btw. did anyone test https://github.com/HolyWu/vs-codeformer ?
https://imgsli.com/MTY2MTM5
Suzie: https://imgsli.com/MTY2MTQz

[ChaosKing]

lol they get entirely new faces

[Selur]

Yes, but in combination with realesrgan this might have its uses.
=> I have to check what happens if this is applied to different actors,...
https://imgsli.com/MTY2MTQ0
https://imgsli.com/MTY2MTQ2
https://imgsli.com/MTY2MTQ3
https://imgsli.com/MTY2MTQ5
seems more like a gimmick filter than really usable atm., but its impressive.

Cu Selur

[Dogway]

Some people mix it with GFPGAN or other face models, haven't played with NN but it seems there's more work to it than a simple button push.
Usually whenever I use NN upscaling I always follow a downscale, to hide the pristine AI look and blend everything together.
Very excited to get into this AI craze soon.

[HolyWu]

Originally Posted by Selur

Yes, but in combination with realesrgan this might have its uses.
=> I have to check what happens if this is applied to different actors,...
https://imgsli.com/MTY2MTQ0
https://imgsli.com/MTY2MTQ2
https://imgsli.com/MTY2MTQ3
https://imgsli.com/MTY2MTQ5
seems more like a gimmick filter than really usable atm., but its impressive.

Cu Selur

You should also try weight=1 if you prefer a higher-fidelity result.