Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th May 2020, 12:20   #1  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
VS Port of TOFLOW possible?

Hello,

I am twisting my head around AI Upscaling lately, trying to get a grasp of what is developed at the moment.. Some Companies pick up the topic and release Software (Topaz AI Video Enhancer for example) and some stuff already has found itĀ“s way into open source / free tools (Waifu2x in Selur's hybrid as vapoursynth variants using Vulkan or Cuda, Video2x has waifu2x and lately incorporated realsr)... A lot going on... Playing around with ESRGAN Resizers, Topaz Beta Software, etc. is fun and some results are really astonishing compared to traditional filters we all know.

Would someone have the skill and have fun building a vapoursynth variant of one of the more mature models out there developed for video? I know of an VS - ESRGAN Port, but it is only "single image" for now...

As far as I know, Task Oriented Flow seems to be a candidate which is especially for video and is able to recover details, denoise and upscale - So my thinking is that it could be a good candidate..

https://github.com/anchen1011/toflow
pytorch version:
https://github.com/Coldog2333/pytoflow

I am aware that some of the dependencies probably aren“t "portable", so some stuff like CUDA or Python might have to be installed seperately, but that's just my uneducated guess... And wouldn“t hurt..

Actually, I really don`t care if TOFLOW would be ported or something similar or even better... I am just dreaming of a vapoursynth variant of some AI-Upscale/Denoise/whatsoever But I am lacking the skills to do so - maybe someone with more knowlegde than me wants to give it a shot?
ReinerSchweinlin is offline   Reply With Quote
Old 28th May 2020, 14:45   #2  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Have you seen this? https://github.com/WolframRhodium/Super-Resolution-Zoo (mostly upscaler denoiser models)
I didn't like any of the denoiser models bcs you can't really tweak them with paramteres. So they are often too strong or too weak or just look bad. I think the ByNet upscaler looked good and had some decent fps.

Maybe it is possible to converted these models to "SuperResZoo models"

The x2 toflow fps looked not bad. I wonder how fast it is.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is online now   Reply With Quote
Old 28th May 2020, 19:44   #3  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
I stumpled across the Wholfram Rhodium one - Since it said "models", I always thought these simply where the trained models for each framework and let it be - looking at it again it seems the frameworks are also included (ported to mxnet).. Am I getting it right?

Up till now, I only managed to install some of the windows GUIs for ESRGAN or waifu2x like:
http://forums.qhimm.com/index.php?topic=18798.50
https://github.com/ptrsuder/IEU.Winforms

and have been browsing through various sites like
https://awesomeopensource.com/projects/super-resolution

to get a grasp of whats out there and compare stuff (only by looking at the demos mostly - ESRGAN with different models and waifu2x I managed to get running and have been playing around with them on my RTX2060 in windows using the above mentioned GUIs. So far, my skills are not refined enough to compile stuff myself, I am only getting up to the point where someone else has done work and managed to put it into a GUI..)

Since the Game-upscale-community is very active, I found a guide for training own models for Esrgan (https://www.reddit.com/r/GameUpscale...d_setup_guide/) which I thought to be a good idea (making a training set out of bluray frames and record them onto VHS - then train ESRGAN on it) - but my last attempt to Linux has been a few years ago and my skills are very basic. If I have time, I manage to find out stuff, but I am no programmer or "daily Linux user"... fiddling a Linux on yn ancient G3 Power Mac I manage, installing different distros on a PC and configuring some basic stuff - ok... But thatĀ“s about it for the moment.

Shortly after taking many close looks at ESRGAN, I learned that it is only single frame capable and that there are some newer techniques (like TOFLOW) out there which are able to recover some details out of the motion of a video, taking several frames into account.

one thing I encountered which seems to use vapoursynth is:
https://github.com/AlphaAtlas/Vapour...olution-Helper
But somehow its not running correctly on my system. I must say that I grew up with Avisynth and still have to get my head wrapped around Vapoursynth more.

So I'm afraid I canĀ“t contribute much on the actual programming here, I am simply pointing out a suggestion and giving my amateur thoughts on the matter.

IĀ“d love to learn to write stuff like this - but this takes a lot of time and sadly, a noob google "write your own vapoursynth filter" brings up stuff over my head
ReinerSchweinlin is offline   Reply With Quote
Old 23rd June 2020, 20:16   #4  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
New upscaler / denoiser https://github.com/Kiyamou/VapourSynth-SRMD-ncnn-Vulkan

vsrepo install srmd
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is online now   Reply With Quote
Old 24th June 2020, 08:40   #5  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
hm, nice Thanks
ReinerSchweinlin is offline   Reply With Quote
Old 25th June 2020, 15:49   #6  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,255
Is there a portable version of this available?
-> scratch that missed the release on github
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 5th July 2020 at 09:57.
Selur is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.