Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 8th July 2016, 15:35   #1  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Waifu2x-caffe

https://github.com/HomeOfVapourSynth...-Waifu2x-caffe

Another implementation of waifu2x, based on the waifu2x-caffe library. It can only run on NVIDIA GPU.
HolyWu is offline  
Old 11th July 2016, 17:05   #2  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Update r6.
  • Update waifu2x-caffe library to 1.1.6.
  • Add noise reduction level 0 and make it the default level.
HolyWu is offline  
Old 11th September 2016, 10:00   #3  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Update r7.
  • Update upconv_7 models.
  • Change the default model to upconv_7_anime_style_art_rgb.
  • Update waifu2x-caffe library to 1.1.7.1.
HolyWu is offline  
Old 19th September 2016, 17:40   #4  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 67
Out of curiosity, why does Waifu2x caffe support non-integer scaling when this version doesn't? 2.25x scaling would be perfect for 480p -> 1080p.

EDIT: Also, great filter BTW. This is by far the best animation scaling I've seen.

Last edited by brucethemoose; 19th September 2016 at 20:37.
brucethemoose is offline  
Old 20th September 2016, 13:56   #5  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Quote:
Originally Posted by brucethemoose View Post
Out of curiosity, why does Waifu2x caffe support non-integer scaling when this version doesn't? 2.25x scaling would be perfect for 480p -> 1080p.
Internally the image is always enlarged by a power of 2. So even you specify a scale ratio of 2.25x, the program still enlarges the image to 4x first. It then downscales the 4x image to the final resolution afterwards. I simply don't want to hardcode the downscaling part in my filter. It's up to the users to decide which resizer to use for downscaling.
HolyWu is offline  
Old 23rd May 2017, 19:42   #6  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Update r8.
  • Update waifu2x-caffe library to 1.1.8.3.
  • Now uses cuDNN 6.0.
  • Add configure script. It's compilable on Linux now. Thanks Are_ for testing.
HolyWu is offline  
Old 1st June 2017, 11:22   #7  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
I try to replicate an exception posted to the staxrip issue tracker, the user is getting E_ACCESSDENIED after staxrip calls TaskDialog to show an error returned from VapourSynth, I get following error:

Python exception: Waifu2x-caffe: failed open model file at initialization

Where can I find this model file?

Code:
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r"D:\Projekte\VS\VB\StaxRip\bin\Apps\Plugins\vs\fmtconv\fmtconv.dll")
core.std.LoadPlugin(r"D:\Projekte\VS\VB\StaxRip\bin\Apps\Plugins\vs\vslsmashsource\vslsmashsource.dll")
clip = core.lsmas.LWLibavSource(r"D:\Temp\staxrip\ash.mkv")
core.std.LoadPlugin(path=r'D:\Temp\Waifu2x-caffe-r8\Win64\Waifu2x-caffe.dll')
clip = core.fmtc.bitdepth(clip,bits=32)
clip = core.caffe.Waifu2x(clip, noise=1, scale=2, block_w=512, block_h=512, model=3, cudnn=True, processor=0, tta=False)
clip.set_output()
https://github.com/stax76/staxrip/issues/190
stax76 is offline  
Old 1st June 2017, 13:04   #8  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Quote:
Originally Posted by stax76 View Post
Python exception: Waifu2x-caffe: failed open model file at initialization

Where can I find this model file?
They are at the GitHub page as well, just in previous release.
HolyWu is offline  
Old 1st June 2017, 18:12   #9  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
Thanks, got it working now.
stax76 is offline  
Old 3rd June 2017, 00:34   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Thanks HolyWu,

I tested on some low to midrange cards, and this seems much faster than the vpy Waifu2x-w2xc implementation. Playing with the blocksize can make a big difference in the speed

Offhand do you know of any big differences quality wise ? or would there be any reason NOT to switch to using the caffe version if you had a compatible GPU ?

Last edited by poisondeathray; 3rd June 2017 at 00:40.
poisondeathray is offline  
Old 3rd June 2017, 03:45   #11  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Quote:
Originally Posted by poisondeathray View Post
Offhand do you know of any big differences quality wise ? or would there be any reason NOT to switch to using the caffe version if you had a compatible GPU ?
The difference in quality is quite minor. Actually the models 0~2 in caffe version are the same as w2xc version. The major difference is that caffe version has upconv_7 models and cuDNN support while w2xc version doesn't. Both of them primarily concern speed. There is really no reason to use w2xc version when you have a compatible NVIDIA GPU though.
HolyWu is offline  
Old 12th June 2017, 13:12   #12  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
What is the recommended card for this? I'm using the default setting on a clip scaled to 1440x1020 after the filter, and I'm getting 0.66fps...
lansing is offline  
Old 12th June 2017, 16:48   #13  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Quote:
Originally Posted by lansing View Post
What is the recommended card for this? I'm using the default setting on a clip scaled to 1440x1020 after the filter, and I'm getting 0.66fps...
There is no officially recommended card. The only requirement is a NVIDIA GPU of compute capability 3.0 or higher.

I guess the color family of your clip is YUV? You will get slightly faster speed if you convert to RGB first before invoking the filter. Furthermore, tweak block_w/block_h setting so that it can divide the width/height of the clip with no remainder. Larger value isn't necessarily better because it has to do with the card's capability and/or memory size. You have to do experiments with some different values to find out the optimal pair for the specific resolution.
HolyWu is offline  
Old 12th June 2017, 17:19   #14  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by HolyWu View Post
Furthermore, tweak block_w/block_h setting so that it can divide the width/height of the clip with no remainder. Larger value isn't necessarily better because it has to do with the card's capability and/or memory size. You have to do experiments with some different values to find out the optimal pair for the specific resolution.
Do you mean the width/height of the video before the filter or after?
lansing is offline  
Old 13th June 2017, 06:37   #15  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by lansing View Post
Do you mean the width/height of the video before the filter or after?
no, the block_w , block_h are filter arguments

https://github.com/HomeOfVapourSynth...-Waifu2x-caffe
Code:

block_w: The horizontal block size for dividing the image during processing. Smaller value results in lower VRAM usage, while larger value may not necessarily give faster speed. The optimal value may vary according to different graphics card and image size.

block_h: The same as block_w but for vertical.
It can make a big difference in speed , play with different values

Don't complain, this is much faster than the waifu w2xc implementation . And that was much much faster than the avisynth waifu2x implementation
poisondeathray is offline  
Old 13th June 2017, 18:00   #16  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
I guess lansing was asking whether he should divide the image of the original size or the upscaled size. The answer is the original size though.
HolyWu is offline  
Old 28th October 2018, 11:07   #17  |  Link
HolyWu
Registered User
 
Join Date: Aug 2006
Location: Taiwan
Posts: 392
Update r10.
  • Update waifu2x-caffe library to 1.1.9.2.
  • Add UpResNet10 model.
  • Add batch parameter.
  • Now uses CUDA Toolkit v10.0.130 and cuDNN v7.3.1.
HolyWu is offline  
Old 28th October 2018, 21:44   #18  |  Link
amayra
Quality Checker
 
amayra's Avatar
 
Join Date: Aug 2013
Posts: 284
so this work only in windows ?
__________________
I love Doom9

Last edited by amayra; 28th October 2018 at 21:48.
amayra is offline  
Old 29th October 2018, 00:38   #19  |  Link
Blovesx
Registered User
 
Join Date: May 2015
Posts: 14
Okay, where can I find any explanation what's UpResNet10? How much is it different from other models?

What's batch option for?

I'm using GUI version for Windows. How can I encode video using this program? You posted the link for vapoursynth (whatever it is) but I don't get it how to compile it. Can't you upload it as you do with caffe-gui releases?
Blovesx is offline  
Old 29th October 2018, 02:09   #20  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Quote:
Originally Posted by Blovesx View Post
Okay, where can I find any explanation what's UpResNet10? How much is it different from other models?
Personally, UpResNet10 combines the advantages of residual block and squeeze-and-excitation block (the former one eases the training while the later one boosts the representation power):
The network is made up of 5 "residual SE blocks" with global skip connection. Each residual SE block includes 2 convolution layers (5 * 2 =10, which gives the name UpResNet10) and a channel attention mechanism.
Previous waifu2x models do not make use of any of these techniques.

Similar structure can be found in recent state-of-the-art super-resolution models like EDSR, RCAN, etc.

Last edited by WolframRhodium; 29th October 2018 at 02:24.
WolframRhodium is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:09.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.