Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th October 2019, 21:00   #1  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
VSGAN - VapourSynth GAN Implementation, based on ESRGAN's implementation

https://github.com/imPRAGMA/VSGAN

This is a port of ESRGAN
It supports both new and old "arch", any scale option, any model.
Input clip can be of any format, but it will be converted to RGB24 to be compatible with pytorch. (perhaps someone can figure out a workaround?)

Last edited by PRAGMA; 22nd October 2019 at 23:52.
PRAGMA is offline   Reply With Quote
Old 16th October 2019, 00:45   #2  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database

Last edited by poisondeathray; 16th October 2019 at 00:51.
poisondeathray is offline   Reply With Quote
Old 16th October 2019, 07:28   #3  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
Quote:
Originally Posted by poisondeathray View Post
Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database
Yep, take a look at:
https://github.com/WolframRhodium/Super-Resolution-Zoo
PRAGMA is offline   Reply With Quote
Old 16th October 2019, 07:34   #4  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Quote:
Originally Posted by PRAGMA View Post
Yes, I remember this, from the other thread - and I could never get that working

But these are converted models (params , json ; not pth), are they drop in compatible ?

(And some of the links to the original projects and papers have pth models)

Last edited by poisondeathray; 16th October 2019 at 07:42.
poisondeathray is offline   Reply With Quote
Old 16th October 2019, 10:30   #5  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,030
Quote:
Originally Posted by poisondeathray View Post
Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database
I tested some filters and the most usefull (and safe) one which also does not generate artifacts was 1x_DeSharpen.

The "spongebob model" looked also not bad for some older animes but had some blurry spots for some reason.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 16th October 2019, 10:40   #6  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,030
I get out of memory errors for 1080p content using a GTX 1070. Does it really need what much ram? used model 1x_DeSharpen

CUDA out of memory. Tried to allocate 1.48 GiB (GPU 0; 8.00 GiB total capacity; 2.56 GiB already allocated; 72.05 MiB free; 3.65 GiB cached)

Restarting vsedit didn't help.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 16th October 2019, 16:04   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
I think so...I can only use 1x_Desharpen on smaller dimensions . You can look at GPU caps viewer or similar utilities and it looks like it's using all

I wonder if there is a way to share system memory with CUDA memory ? eg. Although it's slower, some 3D/CG renderers can offload graphics card memory to system memory when doing calculations (like a shared pool) enabling you to complete if the scene is too large
poisondeathray is offline   Reply With Quote
Old 16th October 2019, 22:36   #8  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 65
Quote:
Originally Posted by poisondeathray View Post
I think so...I can only use 1x_Desharpen on smaller dimensions . You can look at GPU caps viewer or similar utilities and it looks like it's using all

I wonder if there is a way to share system memory with CUDA memory ? eg. Although it's slower, some 3D/CG renderers can offload graphics card memory to system memory when doing calculations (like a shared pool) enabling you to complete if the scene is too large
I believe graphics drivers automatically do this in 3D programs, but that might not be the case with CUDA.

PyTorch does have a "empty cache" function. Maybe you could call it every few frames with FrameEval.

https://pytorch.org/docs/stable/cuda...ory-management
brucethemoose is offline   Reply With Quote
Old 17th October 2019, 16:08   #9  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Quote:
Originally Posted by brucethemoose View Post
I believe graphics drivers automatically do this in 3D programs, but that might not be the case with CUDA.
Definitely CUDA for some 3D renderers like redshift . It's known as "out of core" rendering. I don't think it's an "automatic" feature for CUDA, some programmer figured it out. It's a big selling point for some types of renderers

Quote:

PyTorch does have a "empty cache" function. Maybe you could call it every few frames with FrameEval.

https://pytorch.org/docs/stable/cuda...ory-management
Thanks, but I can't even do 1 1920x1080 frame, it probably won't help calling it periodically. It's only that 1x_DeSharpen, the other models work ok (I didn't test them all , just a few) . So maybe something is up with that model

Last edited by poisondeathray; 17th October 2019 at 16:14.
poisondeathray is offline   Reply With Quote
Old 20th October 2019, 14:03   #10  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
Quote:
Originally Posted by ChaosKing View Post
I get out of memory errors for 1080p content using a GTX 1070. Does it really need what much ram? used model 1x_DeSharpen

CUDA out of memory. Tried to allocate 1.48 GiB (GPU 0; 8.00 GiB total capacity; 2.56 GiB already allocated; 72.05 MiB free; 3.65 GiB cached)

Restarting vsedit didn't help.
Quote:
Originally Posted by ChaosKing View Post
I tested some filters and the most usefull (and safe) one which also does not generate artifacts was 1x_DeSharpen.

The "spongebob model" looked also not bad for some older animes but had some blurry spots for some reason.
This is an ESRGAN port, so it will generate results the exact same as ESRGAN, and use the same amount of memory as ESRGAN.
But yes, generally speaking x1080 resolution content require quite a bit of memory, i'm trying to work out a way to split up the input frame into multiple split images, to then run them all separately and then combine them back to 1 frame and return, that way using the same amount of memory, but less memory simultaneously, would be slower, but would be the only viable option for most GPU's.
I'm having a bit of trouble working it out right now but hopefully I or someone else, figures out a way to do so.
PRAGMA is offline   Reply With Quote
Old 20th October 2019, 14:52   #11  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 110
Quote:
Originally Posted by PRAGMA View Post
i'm trying to work out a way to split up the input frame into multiple split images, to then run them all separately and then combine them back to 1 frame and return
Example from EDSR: https://github.com/thstkdgus35/EDSR-...init__.py#L105
WolframRhodium is offline   Reply With Quote
Old 20th October 2019, 15:54   #12  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
Quote:
Originally Posted by WolframRhodium View Post
I updated VSGAN with chunk support, it will chunk into 4 sub images, and render them separately instead of all at once, then re-compile it back into 1 image and returns to frame server.

I didn't end up using anything like EDSR though, instead I simply cropped the frame, and then ran them.
PRAGMA is offline   Reply With Quote
Old 20th October 2019, 17:33   #13  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Thanks for the update

Request: If you have time would it be possible to port EDVR ? Looks great and deals with video specific issues, temporal and motion blur

https://xinntao.github.io/projects/EDVR
https://github.com/xinntao/EDVR
poisondeathray is offline   Reply With Quote
Old 22nd October 2019, 15:46   #14  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 462
Hey PRAGMA,

Just wanted to let you know that the __init__.py file installed by pip and by the zip file provided by you in the manual installation steps have different file sizes. Is this intended?
aegisofrime is offline   Reply With Quote
Old 22nd October 2019, 23:35   #15  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Request#2 , another pytorch implementation, PPON

https://github.com/Zheng222/PPON

Someone forked it and wrote some demo code , would it be possible to implement in vapoursynth ?
https://github.com/vlad3996/PPON/blo...rence_utils.py
https://github.com/vlad3996/PPON/blob/master/demo.ipynb

Thanks
poisondeathray is offline   Reply With Quote
Old 22nd October 2019, 23:59   #16  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
Quote:
Originally Posted by aegisofrime View Post
Hey PRAGMA,

Just wanted to let you know that the __init__.py file installed by pip and by the zip file provided by you in the manual installation steps have different file sizes. Is this intended?
I believe when uploaded/setup for PyPI is gets optimized/minified or something before being sent off to PyPI's repo, but I could be wrong, regardless, they are both up to date both the repo and PyPI pip package.
PRAGMA is offline   Reply With Quote
Old 23rd October 2019, 06:00   #17  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 462
Quote:
Originally Posted by PRAGMA View Post
I believe when uploaded/setup for PyPI is gets optimized/minified or something before being sent off to PyPI's repo, but I could be wrong, regardless, they are both up to date both the repo and PyPI pip package.
Well, maybe it's something silly I did somewhere, but I got an unable to import VSGAN error using pip, but was able to use it with the manual installation zip you provided.

As an aside, I wonder what speeds are you folks getting? With a GTX 1070, upscaling 480p content using the "PSNR_x4_DB.pth" model, I'm getting like... 1fps :/
aegisofrime is offline   Reply With Quote
Old 24th October 2019, 01:34   #18  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 11
Quote:
Originally Posted by aegisofrime View Post
Well, maybe it's something silly I did somewhere, but I got an unable to import VSGAN error using pip, but was able to use it with the manual installation zip you provided.

As an aside, I wonder what speeds are you folks getting? With a GTX 1070, upscaling 480p content using the "PSNR_x4_DB.pth" model, I'm getting like... 1fps :/
Yeah on a GTX 1080ti it takes about 1.8s per frame for a 720x480 input image.
RTX 2080ti takes about 1.4s per frame for the same input.
PRAGMA is offline   Reply With Quote
Old 24th October 2019, 03:06   #19  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 204
Quote:
Originally Posted by PRAGMA View Post
Yeah on a GTX 1080ti it takes about 1.8s per frame for a 720x480 input image.
RTX 2080ti takes about 1.4s per frame for the same input.
That's strange, why would the 2080ti be slower than the 1080ti?
Cary Knoop is offline   Reply With Quote
Old 24th October 2019, 03:12   #20  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
Quote:
Originally Posted by Cary Knoop View Post
That's strange, why would the 2080ti be slower than the 1080ti?
lower sec / frame means it 's faster
poisondeathray is offline   Reply With Quote
Reply

Tags
esrgan, gan, upscale, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.