Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th May 2021, 14:44   #41  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
I hope I didnīt talk you into too much work Really appreciate this addition, thanx !!

My amateur suspection is, that the ...GAN folders should be somehow reduceable in size (since some other similar implementations come in smaller packages), but my skills are far from doing that
ReinerSchweinlin is offline   Reply With Quote
Old 16th May 2021, 15:11   #42  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Yeah, my guess too is that one probably can delete some of the large cuda files inside VapourSynth64\Lib\site-packages\torch\lib since probably not all are needed, but haven't tested and probably won't spend time on it.
If someone has the time and motivation to test which dlls are really needed let me know.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 12th June 2021, 08:42   #43  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Just wondering: Can the .params&.json files from https://github.com/WolframRhodium/Super-Resolution-Zoo
be used in VSGAN somehow?

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 29th July 2021, 20:57   #44  |  Link
Zarxrax
Registered User
 
Join Date: Dec 2001
Posts: 1,219
I have installed VSGAN yesterday on windows, and the installation seems to have gone smoothly, but when I try the example script from the documentation, I receive this error:
Code:
raise EnvironmentError(f"VSGAN: Either NVIDIA CUDA or the device ({device}) isn't available.")
OSError: VSGAN: Either NVIDIA CUDA or the device (cuda) isn't available.
Any ideas what this error means? My video card is GTX 750 ti, which I know is quite old, but I am able to use other implementations of ESRGAN on it. I'm not sure if this is telling me my video card is not compatible, or something else.

Last edited by Zarxrax; 29th July 2021 at 22:03.
Zarxrax is offline   Reply With Quote
Old 30th July 2021, 19:26   #45  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Wild guess would be a driver or rights issue.
Here is how I set it up.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 30th July 2021, 20:05   #46  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
You can check cuda support with gpuz.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 30th July 2021, 20:35   #47  |  Link
Zarxrax
Registered User
 
Join Date: Dec 2001
Posts: 1,219
Quote:
Originally Posted by Selur View Post
Wild guess would be a driver or rights issue.
Here is how I set it up.
It was the torch version. I uninstalled torch and then installed an older version using the command you provided, and it works now.
Zarxrax is offline   Reply With Quote
Old 14th August 2021, 04:14   #48  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 182
Would anyone be so kind to post some benchmark numbers for 1080p to 2160p with one model for anime and one model for real life? Please name your gpu and time in seconds per frame. Thanks.
takla is offline   Reply With Quote
Old 19th August 2021, 15:10   #49  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Can't do 1080p to 2160p with my Geforce GTX 1070ti (8GB VRAM) even when using a the 2x_PSNR model, when using one of the models with a 4x resize factor my guess it that you at least need a 30xx card with 12GB or more VRAM.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 19th August 2021, 15:40   #50  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Selur View Post
Can't do 1080p to 2160p with my Geforce GTX 1070ti (8GB VRAM) even when using a the 2x_PSNR model, when using one of the models with a 4x resize factor my guess it that you at least need a 30xx card with 12GB or more VRAM.
Yes it would help if PRAGMA introduced the tiling script portion that some of the ESRGAN scripts and some GUI's use .

When using decent size margins, you don't see any seams or artifacts (but they are "microscopically" detected with amplified differences)
poisondeathray is offline   Reply With Quote
Old 19th August 2021, 16:16   #51  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Yup, some tiling would to keep the memory consumption low would be nice.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 4th October 2021, 19:01   #52  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Hmm, wrote a small script that splits an image into tileCount*tileCount tiles and applies vsgan on each of them.
Code:
import vapoursynth as vs
from vapoursynth import core
from vsgan import VSGAN
import torch

# clip: the video source
# model: path to the model to callable
# tileCount number of times source will be splitt horizontally and vertically
def tileVSGAN(clip, tileCount = 8, model = None) -> vs.VideoNode:
  if not isinstance(clip, vs.VideoNode):
     raise vs.Error('tileVSGAN: clip must be an instance of vs.VideoNode')
  if clip.format.id != vs.RGB24:
    raise vs.Error('tileVSGAN: clip must be RGB24')
  if model == None:
    raise vs.Error('tileVSGAN: model needs to be specified')
  if tileCount%2 != 0:
    raise vs.Error('tileVSGAN: tileCount needs to dividiable by 2.')
  if clip.width%(tileCount*2) != 0:
    raise vs.Error('tileVSGAN: clip.width must be dividable by 2*tileCount without rest')
  if clip.width%(tileCount*2) != 0:
    raise vs.Error('tileVSGAN: clip.height must be dividable by 2*tileCount without rest')
  
  tileWidth = clip.width/tileCount
  tileHeight = clip.height/tileCount
  tiles = [[None for x in range(tileCount)] for y in range(tileCount)]
  for index1 in range (tileCount): # vertical 
    for index2 in range (tileCount): # horizontal
      tile = core.std.CropRel(clip=clip, left=tileWidth*index2, right = tileWidth*(tileCount-index2-1), top=tileHeight*index1, bottom=tileHeight*(tileCount-index1-1))
      #tiles[index1][index2] = core.text.Text(clip=tile,text=str(index1)+'/'+str(index2),scale=1)
    
   
  # apply VSGAN on each tile
  
  for index1 in range (tileCount): # vertical 
    for index2 in range (tileCount): # horizontal
      vsgan = VSGAN("cuda")
      vsgan.load_model(model)  
      tiles[index1][index2] = vsgan.run(clip=tiles[index1][index2])
      del vsgan # delete vsgan object
      torch.cuda.empty_cache() # trigger pytorch to free gpu memory cache
  
  horizontal = [None for x in range(tileCount)]
  for index in range (tileCount):
     horizontal[index] = core.std.StackHorizontal(tiles[index])
 
  clip = core.std.StackVertical(horizontal)
   
  return clip
the intention was that running vsgan on lower resolution images should require less vram.
Problem is that is doesn't seem to work, even with tileCount 64 on 4k content all my 8GB VRAM is full. :/

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 5th October 2021, 02:02   #53  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
unfortunately I do not have VSGAN to test it, but what about this, trying to process one clip at a time, put it more into functions so things get garbage collected:
Code:
def clipVSGAN(clip, model):        
    vsgan = VSGAN("cuda")
    vsgan.load_model(model)  
    new_clip = vsgan.run(clip=clip)
    del vsgan # delete vsgan object
    torch.cuda.empty_cache() # trigger pytorch to free gpu memory cache
    return new_clip

def tileVSGAN(clip, tileCount = 8, model = None) -> vs.VideoNode:
    if not isinstance(clip, vs.VideoNode):
        raise vs.Error('tileVSGAN: clip must be an instance of vs.VideoNode')
    if clip.format.id != vs.RGB24:
        raise vs.Error('tileVSGAN: clip must be RGB24')
    if model == None:
        raise vs.Error('tileVSGAN: model needs to be specified')
    if tileCount%2 != 0:
        raise vs.Error('tileVSGAN: tileCount needs to dividiable by 2.')
    if clip.width%(tileCount*2) != 0:
        raise vs.Error('tileVSGAN: clip.width must be dividable by 2*tileCount without rest')
    if clip.width%(tileCount*2) != 0:
        raise vs.Error('tileVSGAN: clip.height must be dividable by 2*tileCount without rest')

    def fetch_new_tile(clip, tileWidth, tileHeight, tileCount):
        for index1 in range (tileCount): # vertical 
            for index2 in range (tileCount): # horizontal
                tile = core.std.CropRel(clip   = clip,
                                        left   = tileWidth*index2,
                                        right  = tileWidth*(tileCount-index2-1),
                                        top    = tileHeight*index1,
                                        bottom = tileHeight*(tileCount-index1-1)
                                        ) 
                yield clipVSGAN(tile, model)
                del tile
    
    tileWidth = clip.width/tileCount
    tileHeight = clip.height/tileCount
    tiles = [[None for x in range(tileCount)] for y in range(tileCount)]
    new_tile_generator = fetch_new_tile(clip, tileWidth, tileHeight, tileCount)

    for index1 in range (tileCount):
        for index2 in range (tileCount):
            tiles[index1][index2] = next(new_tile_generator)
     
    horizontal = [None for x in range(tileCount)]
    for index in range (tileCount):
        horizontal[index] = core.std.StackHorizontal(tiles[index])

    clip = core.std.StackVertical(horizontal)
    return clip

Last edited by _Al_; 5th October 2021 at 03:59.
_Al_ is offline   Reply With Quote
Old 5th October 2021, 14:31   #54  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
There are several tiling scripts with variable input size and padding/margins out there

eg
https://github.com/Oriode/ESRGAN-Til...ter/upscale.py

They just need to be "massaged" into vapoursynth form .

Several of HolyWu's scripts have tiling/padding too. HolyWu could probably add it easily
poisondeathray is offline   Reply With Quote
Old 5th October 2021, 18:00   #55  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
@_Al_: doesn't seem to behave differntly
Code:
RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 8.00 GiB total capacity; 6.75 GiB already allocated; 0 bytes free; 6.90 GiB reserved in total by PyTorch)
So if someone comes up with a working way to use this please share.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 5th October 2021, 21:34   #56  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
https://github.com/rlaphoenix/VSGAN/...an/__init__.py
PRAGMA had implemented that already, his parameter chunk in run()
Code:
def run(self, clip: vs.VideoNode, chunk: bool = False) -> vs.VideoNode:
chunk=True would split clip to quadrants

Code:
    def chunk(self, clip: vs.VideoNode) -> Iterable[vs.VideoNode]:
        """
        Split clip down the center into two clips (a left and right clip)
        Then split those 2 clips in the center into two clips (a top and bottom clip).
        Resulting in a total of 4 clips (aka chunk).
        """
        return itertools.chain.from_iterable([
            self.split(x, axis=1) for x in self.split(clip, axis=0)
        ])
that function would need to be changed to return list of tiles

then he is reassembling clips doing this:
Code:
        # if chunked, rejoin the chunked clips otherwise return the result
        clip = core.std.StackHorizontal([
            core.std.StackVertical([results[0], results[1]]),
            core.std.StackVertical([results[2], results[3]])
        ]) if chunk else results[0]
that would need to be changed to reassemble it same way as it was chopped off

but there is a concern, because that poissondeathray link workflow uses margins, which they are cut off before assembling tiles, probably if not done, borders could be visible with some artifacts

Last edited by _Al_; 5th October 2021 at 21:44.
_Al_ is offline   Reply With Quote
Old 6th October 2021, 00:08   #57  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by _Al_ View Post

but there is a concern, because that poissondeathray link workflow uses margins, which they are cut off before assembling tiles, probably if not done, borders could be visible with some artifacts
Yes, you need padding, otherwise you get artifacts. Quality loss is negligible if padding is large enough (but detectable with metrics or amplified differences - you see the split lines, but invisible to human eye)
poisondeathray is offline   Reply With Quote
Old 6th October 2021, 21:35   #58  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 73
Quote:
Originally Posted by _Al_ View Post
https://github.com/rlaphoenix/VSGAN/...an/__init__.py
PRAGMA had implemented that already, his parameter chunk in run()
Code:
def run(self, clip: vs.VideoNode, chunk: bool = False) -> vs.VideoNode:
chunk=True would split clip to quadrants

Code:
    def chunk(self, clip: vs.VideoNode) -> Iterable[vs.VideoNode]:
        """
        Split clip down the center into two clips (a left and right clip)
        Then split those 2 clips in the center into two clips (a top and bottom clip).
        Resulting in a total of 4 clips (aka chunk).
        """
        return itertools.chain.from_iterable([
            self.split(x, axis=1) for x in self.split(clip, axis=0)
        ])
that function would need to be changed to return list of tiles

then he is reassembling clips doing this:
Code:
        # if chunked, rejoin the chunked clips otherwise return the result
        clip = core.std.StackHorizontal([
            core.std.StackVertical([results[0], results[1]]),
            core.std.StackVertical([results[2], results[3]])
        ]) if chunk else results[0]
that would need to be changed to reassemble it same way as it was chopped off

but there is a concern, because that poissondeathray link workflow uses margins, which they are cut off before assembling tiles, probably if not done, borders could be visible with some artifacts
Hi, yeah, it has support for this but generally speaking I advise people not to use it because the current way it works isnt as unnoticeable as other solutions can return.

I could probably work on adding some black padding fairly easily to each one, and then crop if that works well enough. I see selur made an issue post asking for it, so if I ever work on it I will push updates there. https://github.com/rlaphoenix/VSGAN/issues/9
PRAGMA is offline   Reply With Quote
Old 6th October 2021, 21:43   #59  |  Link
PRAGMA
Registered User
 
Join Date: Jul 2019
Posts: 73
I updated the Installation docs page to add some small information about how to use python/pip on a portable installation. Some users here struggled with that.
I also created a new troubleshooting page with some solutions or advice to problems posted in pages 2 and 3.
PRAGMA is offline   Reply With Quote
Old 6th October 2021, 22:53   #60  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
I believe padding with black is not going to help, rather it needs the surrounding image data. Kind of like MVTools overlap.
zorr is offline   Reply With Quote
Reply

Tags
esrgan, gan, upscale, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.