Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th August 2021, 06:28   #21  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,148
The interessting part are the screenshots, so the language shouldn't be an issue. Not much text so, Google Translate or similar should work too.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 28th August 2021, 14:10   #22  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Quote:
Originally Posted by Selur View Post
Btw. has anyone an example where BasicVSR show some impressive results when starting from a 480p or 576p source?
Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.
Only thing that I found is that it does do well is upscaling QCIF content, and I wonder whether that is it's goal.

BasicVSR works ok in simular situations to where the ESRGAN base model works ok - ie. fairly clean, downsampled sources - really an oversampled SD source... Which is NOT what people typically have - it's usually some noisy, soft DVD source, or SD crappy home video that people want to upscale. But those clean datasets are what most publically available algorithms were trained on: Clean HR ground truth source, downscaled by 4. Usually with bicubic. BasicVSR used selections from the Reds4 and Vimeo90k datasets. qcif "suzie" is really an oversampled source. If you take some ok quality UHD, and downscale it, that will work too. What's needed is training of specific models that take account for noise, degredation, compression artifacts like mpeg2 artifacts, etc... stuff like bad YT compression and degredation after the downscaling step. There are specific datasets that include noise - but they are synthetic noise - and they weren't used for these default models .

So in those situations where the model is applicable - BasicVSR is going to produce more detailed images than , say lanczos3 + any form of sharpen, but less detailed results than default ESRGAN or ESRGAN+ models on single image examination. On single image, ESRGAN+ is going to look more "wow" in applicable source situations. The main difference is BasicVSR is more consistent temporally, when you look at actual video, or more than 1 image. It resembles tecogan in that regard, but produces better results detail wise than the default tecogan model (which isn't the same as the tecogan published paper model). For all single image SR approaches you often need to use some type of temporal smoothing to make it decent for video purposes (and that smoothing reduces the fine details too) , otherwise you are prone to flickering aliasing artifacts, objects changing shapes (as single image prediction is slightly different on different frames) .

So the question is can you use something like ESRGAN, add some QTGMC in progressive mode - to get similar temporally consistent results to BasicVSR, but hopefully keep more fine details? The answer is generally no. The artifacts are too "baked" in - that's the strength of basicvsr
poisondeathray is offline   Reply With Quote
Old 28th August 2021, 17:52   #23  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Some apng comparisons. They should animate and loop in most browsers

1:1 non-AR corrected crops, unfiltered. 720x480 => 2880x1920.

Note the temporal flickering artifacts (aliasing flicker, and prediction flicker).

You can see the earrings look like random lightning in the ESRGAN+ results.



No amount of sharpening will transform lanczos3 into as detailed as BasicVSR. No amount of temporal filtering will transform lanczos3, or ESRGAN+ results into as temporally consistent results as BasicVSR




The license plate, you can make out the numbers/letters with lanczos, but it's much more clear in the BasicVSR. It doesn't take a stretch of the imagination that slightly smaller or a more distant plate might be illegible when processed with lanczos , but partially readable by BasicVSR



ESRGAN and psnr model mess up letters/numbers/text (many GAN, NN models do), I won't include in the animated temporal demo, but here is a still image compare animated demo. ESRGAN default does good on BG foliage textures, but messes up text. None of the GAN scalers have "fuzzy" lines or bad aliasing - that's what they excel at. Early gen NNEDI/1/2/3 is NN (neural net) based, but signficantly worse in terms of aliasing artifacts for upscaling 4x than modern GAN scalers





But these are cleanly downsample source examples, much like the training set. ie. They don't have MPEG2 macroblocks or or grain plate added on top like you might with some film DVD

Some trained degraded jpeg ESRGAN models work ok for single image jpeg interpolation - I would like to see more BasicVSR models that utilize some typical degraded aspects of common video
poisondeathray is offline   Reply With Quote
Old 28th August 2021, 19:49   #24  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,654
The car plate image is amazing
lansing is offline   Reply With Quote
Old 28th August 2021, 19:55   #25  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Keep in mind those are "textbook" clean sources . Normal dashcam / security cam footage would have a bunch of data loss, compression artifacts. The training set for the 3 base models do not account for that

There is an improved version, BasicVSR++ , but the code has not been released yet . I hope HolyWu brings it to vapoursynth when it's released
poisondeathray is offline   Reply With Quote
Old 28th August 2021, 22:55   #26  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,148
@poisondeathray: Thanks for the explaination, that explains why I didn't get and impressive results from my usual content.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 28th August 2021, 23:51   #27  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,148
Quote:
There is an improved version, BasicVSR++ ,..
Looks impressive: https://ckkelvinchan.github.io/projects/BasicVSR++/

Quote:
I hope HolyWu brings it to vapoursynth when it's released
fingers crossed

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 29th August 2021, 02:27   #28  |  Link
edcrfv94
Registered User
 
Join Date: Apr 2015
Posts: 84
Quote:
Originally Posted by poisondeathray View Post
Keep in mind those are "textbook" clean sources . Normal dashcam / security cam footage would have a bunch of data loss, compression artifacts. The training set for the 3 base models do not account for that

There is an improved version, BasicVSR++ , but the code has not been released yet . I hope HolyWu brings it to vapoursynth when it's released
released 10 days ago
https://github.com/open-mmlab/mmedit...basicvsr_pp.py

MMEditing v0.10.0 Release
Support LIIF-RDN (CVPR'2021)
Support BasicVSR++ (NTIRE'2021)
https://github.com/open-mmlab/mmedit...es/tag/v0.10.0
edcrfv94 is offline   Reply With Quote
Old 29th August 2021, 03:01   #29  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Quote:
Originally Posted by edcrfv94 View Post
released 10 days ago
https://github.com/open-mmlab/mmedit...basicvsr_pp.py

MMEditing v0.10.0 Release
Support LIIF-RDN (CVPR'2021)
Support BasicVSR++ (NTIRE'2021)
https://github.com/open-mmlab/mmedit...es/tag/v0.10.0
Nice!!
poisondeathray is offline   Reply With Quote
Old 30th August 2021, 04:39   #30  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 478
@poisondeathray: Could I ask what sort of speed are you getting in your test? Upscaling just from 720x480 -> 1280x720 I'm getting like 0.05fps with a Ryzen 5950X + RTX 3060Ti, and I'm wondering if I'm doing something wrong.
aegisofrime is offline   Reply With Quote
Old 30th August 2021, 14:09   #31  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Quote:
Originally Posted by aegisofrime View Post
@poisondeathray: Could I ask what sort of speed are you getting in your test? Upscaling just from 720x480 -> 1280x720 I'm getting like 0.05fps with a Ryzen 5950X + RTX 3060Ti, and I'm wondering if I'm doing something wrong.
Similar speeds - it's very slow.
poisondeathray is offline   Reply With Quote
Old 1st September 2021, 11:21   #32  |  Link
gugglu
Registered User
 
Join Date: Jul 2012
Location: Nottingham
Posts: 44
Hi folks, i am trying to test the vsbasicVSR but whenever i try to load the BasicVSR it says cuda not available. any hint what i am missing, tia.

script using to test
Code:
import vapoursynth as vs
import numpy
import torch
from vsbasicvsr import BasicVSR

core = vs.get_core()

src = core.dgdecodenv.DGSource(r'C:\Users\User89\Desktop\test\00000.dgi')
src = core.fmtc.bitdepth(src, bits=32)
src = core.fmtc.resample(src, kernel='spline36', css='444')
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)
ret = BasicVSR(src)

ret.set_output()
I get this error when checking the above script
Code:
Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2242, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 2243, in vapoursynth.vpy_evaluateScript
File "C:/Users/User89/Desktop/VapourSynthEditor-r19-64bit/Untitled.vpy", line 12, in 
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)
File "C:\Users\User89\AppData\Local\Programs\Python\Python39\lib\site-packages\vsbasicvsr\__init__.py", line 56, in BasicVSR
raise vs.Error('BasicVSR: CUDA is not available')
vapoursynth.Error: BasicVSR: CUDA is not available

Last edited by gugglu; 1st September 2021 at 20:37.
gugglu is offline   Reply With Quote
Old 1st September 2021, 15:58   #33  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Quote:
Originally Posted by gugglu View Post
vapoursynth.Error: BasicVSR: CUDA is not available
Drivers ?

Did you install pytorch with cuda ?

Does cuda work ok for you in other applications, or vapoursynth ? eg. try eedi2cuda
https://github.com/AmusementClub/VapourSynth-EEDI2CUDA
poisondeathray is offline   Reply With Quote
Old 1st September 2021, 17:16   #34  |  Link
gugglu
Registered User
 
Join Date: Jul 2012
Location: Nottingham
Posts: 44
Quote:
Originally Posted by poisondeathray View Post
Drivers ?

Did you install pytorch with cuda ?

Does cuda work ok for you in other applications, or vapoursynth ? eg. try eedi2cuda
https://github.com/AmusementClub/VapourSynth-EEDI2CUDA
Hi poisondeathray, i've installed cuda (cuda_11.0.2_451.48_win10) but on its own here are some scrn.shots of process please have a look.

Cuda installation:
https://prnt.sc/1qyobff
https://prnt.sc/1qyocvh
https://prnt.sc/1qyodye
https://prnt.sc/1qyp184
https://prnt.sc/1qyp3ly
installed Drivers:
https://prnt.sc/1qyp9ae
and pytorch was installed with vsbasicvsr via pip as instructed on a github .
https://prnt.sc/1qyrk22
yes EEDi2CUDA works fine
https://prnt.sc/1qyrd78

gugglu is offline   Reply With Quote
Old 1st September 2021, 18:06   #35  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285
Quote:
Originally Posted by gugglu View Post
H
and pytorch was installed with vsbasicvsr via pip as instructed on a github .
https://prnt.sc/1qyrk22
You're missing pytorch with cuda . You only have pytorch cpu

There should be a folder torch-xxx-cuxxx

Where "xxx" is the version number, cu is for cuda

If you go here, and select the boxes that apply to you, it will give you command line to install correct version with cuda
https://pytorch.org/get-started/locally/
poisondeathray is offline   Reply With Quote
Old 1st September 2021, 18:39   #36  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,148
Also, if you want I can send you a link to the 'addon' for Hybrid I just uploaded, it's basically a portable Vapoursynth&Python install with:packed size ~9GB
if you want a link to the download send me a pm.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 1st September 2021, 20:12   #37  |  Link
gugglu
Registered User
 
Join Date: Jul 2012
Location: Nottingham
Posts: 44
Quote:
Originally Posted by poisondeathray View Post
You're missing pytorch with cuda . You only have pytorch cpu

There should be a folder torch-xxx-cuxxx

Where "xxx" is the version number, cu is for cuda

If you go here, and select the boxes that apply to you, it will give you command line to install correct version with cuda
https://pytorch.org/get-started/locally/
thanks a lot poisondeathray its working after downloading pytorch with cuda with windows command line from a pytorch link

Code:
Microsoft Windows [Version 10.0.19042.1165]
(c) Microsoft Corporation. All rights reserved.

C:\Users\User89\AppData\Local\Programs\Python\Python39\Scripts>pip3 install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.9.0+cu102
  Downloading https://download.pytorch.org/whl/cu102/torch-1.9.0%2Bcu102-cp39-cp39-win_amd64.whl (1440.4 MB)
     |████████████████████████████████| 1440.4 MB 2.8 kB/s
Collecting torchvision==0.10.0+cu102
  Downloading https://download.pytorch.org/whl/cu102/torchvision-0.10.0%2Bcu102-cp39-cp39-win_amd64.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 3.3 MB/s
Collecting torchaudio===0.9.0
  Using cached torchaudio-0.9.0-cp39-cp39-win_amd64.whl (215 kB)
Requirement already satisfied: typing-extensions in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torch==1.9.0+cu102) (3.10.0.2)
Requirement already satisfied: pillow>=5.3.0 in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torchvision==0.10.0+cu102) (8.3.1)
Requirement already satisfied: numpy in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torchvision==0.10.0+cu102) (1.21.2)
Installing collected packages: torch, torchvision, torchaudio
  Attempting uninstall: torch
    Found existing installation: torch 1.9.0
    Uninstalling torch-1.9.0:
      Successfully uninstalled torch-1.9.0
  Attempting uninstall: torchvision
    Found existing installation: torchvision 0.10.0
    Uninstalling torchvision-0.10.0:
      Successfully uninstalled torchvision-0.10.0
Successfully installed torch-1.9.0+cu102 torchaudio-0.9.0 torchvision-0.10.0+cu102
gugglu is offline   Reply With Quote
Old 1st September 2021, 20:20   #38  |  Link
gugglu
Registered User
 
Join Date: Jul 2012
Location: Nottingham
Posts: 44
Quote:
Originally Posted by Selur View Post
Also, if you want I can send you a link to the 'addon' for Hybrid I just uploaded, it's basically a portable Vapoursynth&Python install with:packed size ~9GB
if you want a link to the download send me a pm.

Cu Selur
that seems quite interesting, Selur thank you
gugglu is offline   Reply With Quote
Old 4th September 2021, 20:15   #39  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,285


Thanks HolyWu

vs-BasicVSR++
https://github.com/HolyWu/vs-basicvsrpp
poisondeathray is offline   Reply With Quote
Old 5th September 2021, 09:43   #40  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,148
nice!
Code:
 Support either x4 upsampling (for model 0-2) or same size output (for model 3-5).
    For model 0-2, input resolution must be at least 64.
    For model 3-5, input resolution must be at least 256 and mod-4.
    Parameters:
        clip: Clip to process. Only planar format with float sample type of 32 bit depth is supported.
        model: Model to use.
            0 = REDS
            1 = Vimeo-90K (BI)
            2 = Vimeo-90K (BD)
            3 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 1
            4 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 2
            5 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 3
Is there a difference between BasicVSR and BasicVSR++ if model 0-2 are used, or is it the same resizing as BasicVSR (model 0-2) and additional models for cleaning?
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 5th September 2021 at 09:53.
Selur is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:03.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.