Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st June 2020, 23:32   #501  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by zapp7 View Post
Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!
Oh, I thought you were saying it could get to 6x in 1 step. If it can't output 8x as is it's a bit of a mess then.
Stereodude is offline   Reply With Quote
Old 2nd June 2020, 05:56   #502  |  Link
zapp7
Registered User
 
Join Date: May 2020
Location: Canada
Posts: 49
Quote:
Originally Posted by Stereodude View Post
Oh, I thought you were saying it could get to 6x in 1 step. If it can't output 8x as is it's a bit of a mess then.
So this leads to my next question:

Which do you think is better: upscaling to 4K with 2 resampling steps or upscaling to 1080p with 1 resampling step?
zapp7 is offline   Reply With Quote
Old 4th June 2020, 02:08   #503  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by zapp7 View Post
So this leads to my next question:

Which do you think is better: upscaling to 4K with 2 resampling steps or upscaling to 1080p with 1 resampling step?
No idea. You'd have to try it and compare them.
Stereodude is offline   Reply With Quote
Old 7th June 2020, 05:23   #504  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Did anyone check out my nnedi3 upscales here?
https://forum.doom9.org/showthread.p...91#post1914291

I think upscaling to 4k is over-kill myself, but aside from needing to find a way to deal with the remaining aliasing, I still think they look just as good, if not better, than any of the clever upscaling methods I've seen so far... for upscaling the CGI, at least.

I'm also not sure you couldn't achieve much the same thing by simply applying some decent sharpening and encoding, and letting the player do the upscaling, but I haven't compared that yet.
hello_hello is offline   Reply With Quote
Old 11th June 2020, 17:58   #505  |  Link
zapp7
Registered User
 
Join Date: May 2020
Location: Canada
Posts: 49
Quote:
Originally Posted by hello_hello View Post
Did anyone check out my nnedi3 upscales here?
https://forum.doom9.org/showthread.p...91#post1914291

I think upscaling to 4k is over-kill myself, but aside from needing to find a way to deal with the remaining aliasing, I still think they look just as good, if not better, than any of the clever upscaling methods I've seen so far... for upscaling the CGI, at least.

I'm also not sure you couldn't achieve much the same thing by simply applying some decent sharpening and encoding, and letting the player do the upscaling, but I haven't compared that yet.
I had a look at your nnedi3 and it appears to have less detail than my method of Gaia-HQ (followed by TemporalDegrain2(grainLevel=false)) generated with 16 bit tiffs and dithered down to 10 bit HEVC.
Sample here

Comparison here, GaiaHQ on left, nnedi3 on right

I will say that the nnedi3 is appealing since the whole workflow is far less tedious and takes about 4ish hours per episode versus the 24+ hours for this GaiaHQ workflow. (On my system, at least)

The aliasing doesn't bother me too much. It's not noticeable when I watch on my 4K TV from my couch.

Last edited by zapp7; 11th June 2020 at 18:07.
zapp7 is offline   Reply With Quote
Old 13th June 2020, 02:30   #506  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
zapp7,
It surprises me you're upscaling before TemporalDegrain2. I assume it looks better than de-graining first? I'm not saying it's the wrong way to do it, because whatever looks better....

The tiny bit of live action at the end of your sample looks quite good, and while most of the fuss here has been how to handle the CGI, I assume there's a lot more live action than CGI in most episodes. I watched the show years ago but I don't remember it all that well. I still think upscaling to 4k might be overkill though. How does it look if you just upscale to 1080p and let the TV do the rest? I do wonder if that'd make the GaiaHQ version look a little less "sharpened", while still looking better than the DVD.

The GaiaHQ sample does generally look a bit more detailed than my nnedi3 sample, but I think it's mostly in places where your upscaling fixes problems (aliasing) that needi3 doesn't. I'm not sure I don't prefer the GaiaCG sample JoelHruska uploaded though because the HQ version looks a bit over-sharpened in places to me. Mind you I'm sitting close to the monitor and nitpicking over comparisons of individual frames of CGI for a show that'd be mostly live action. It's what you prefer when the picture is moving and you're sitting on the couch that matters.
hello_hello is offline   Reply With Quote
Old 14th June 2020, 05:31   #507  |  Link
zapp7
Registered User
 
Join Date: May 2020
Location: Canada
Posts: 49
Quote:
Originally Posted by hello_hello View Post
zapp7,
It surprises me you're upscaling before TemporalDegrain2. I assume it looks better than de-graining first? I'm not saying it's the wrong way to do it, because whatever looks better....

The tiny bit of live action at the end of your sample looks quite good, and while most of the fuss here has been how to handle the CGI, I assume there's a lot more live action than CGI in most episodes. I watched the show years ago but I don't remember it all that well. I still think upscaling to 4k might be overkill though. How does it look if you just upscale to 1080p and let the TV do the rest? I do wonder if that'd make the GaiaHQ version look a little less "sharpened", while still looking better than the DVD.

The GaiaHQ sample does generally look a bit more detailed than my nnedi3 sample, but I think it's mostly in places where your upscaling fixes problems (aliasing) that needi3 doesn't. I'm not sure I don't prefer the GaiaCG sample JoelHruska uploaded though because the HQ version looks a bit over-sharpened in places to me. Mind you I'm sitting close to the monitor and nitpicking over comparisons of individual frames of CGI for a show that'd be mostly live action. It's what you prefer when the picture is moving and you're sitting on the couch that matters.
I compared the amount of detail with TemporalDegrain2 before vs after upscaling. It was pretty clear that applying it after upscaling was preserving more detail. My guess is that leaving the frame untouched before upscaling leaves more detail for the upscaler to work with, so that was my rationale. I like how the live action scenes look - and you're right, if I had to guess I would say 95%+ of the series is live action stuff.

I've flip-flopped a lot on the 1080p vs 4k. I might run some more tests, but I'm at the point where I feel like I'm reaching diminishing returns on quality improvement vs time spent fiddling.
zapp7 is offline   Reply With Quote
Old 15th June 2020, 08:08   #508  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Quote:
Originally Posted by JoelHruska View Post
What would you call an approach like ESRGAN? According to the description of the model:

"how do we recover the finer texture details when we super-resolve at large upscaling factors? ... Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks."

They use phrases like "recover" as opposed to "approximate" or "repaint." Are they obfuscating their own approach, or are they one of the models that actually recovers data?
Hey Joel,

sorry for the long delay, "life to do"

ESRGAN actually "makes stuff" up. It uses models which are trained on a certain type of content and tries to recognize what it sees and then repaints what it thinks it sees. Itīs pretty obvious if we take a look at some of the authorīs example pictures:



If you compare "HR" (high resolution = the original) or "ground truth" to the ESRGAN Example and look at the fine details, itīs eays to see, that the bamboo's beard looks good - but the hair are not the same ones as in the original. Also the grass looks rather detailed and good, but the single straws are not all the ones in the Original.

ESRGAN also only uses single frames, so itīs not aware of the prevoius and following frames in a video.
Therefore, it works quite good on single images IF your model is trained for the kind of content you want to upscale. There are quite a few trained models out there, named "faces", "Wood", "maps", etc... some are for realistic fotografs, others for special content, many for old 8 Bit Games etc...

So when they claim to "recover", what they actually are saying is: "we recognize hair, so we put hair there"... To be fair, itīs just teh terminology used in this field.. if one looks up the research papers, the distinction of which mechanism does what is not hung up on terms like "recover"

Although some outputs are very impressive, the limits of this apporach is obvious as soon as you have such low resolution that the guesswork at hand simply is making up false details. A very small shot of an actor shot from the side simply isnīt enough detail to pick up. Also, the model should be trained for the specific content.. Which most arenīt.. VEAI uses generic models, not "Scifi from the 90s" trained ones...

My proposal at Topaz was to train models for specific usecases - There is "DS9 - like looking" footage ou there which exists in HD, so a training set could be compiled and trained upon... In the case if ESRGAN, we could do it ourselfs, there are many tutorials out there how to set up training.

Of course, there are other "AI-Upscalers/denoisers/etc..." out there, in fact, the list ist enormous (the following list is just "a few" examples):
https://awesomeopensource.com/projects/super-resolution

With video, we have another source of detail, we can "tap into"... The previous and following frames often contain the same objects as the current one. Often so called "super resolution" methods can take many frames into account and mostly by subsampling are able to recover details. One good illustration of the basic principal in my oppinion is the somewhat older software photoaccute, which works on multiple shots from the same scene in order to process Fotos. This screenshot frmo the manufacturers website illustrates what can be done if several pictures are combined:

With video, itīs a little more complicated, but research as gone a long way, so there are algos which are able to recover some lost details in video - the screenshots in the above post with the NNEEDI comparison are a good example. The red marked line on the spaceship is a recovered detail which only appears in the AI-Version of the upscale.
One example out of my tests (SG1, taken from NTSC DVD):



Some of the numbers are revealed, some look a little more "generic", maybe I get some more detail out of this (challenge accepted... hehe)... The workflow chain for this is... ugly... but VEAI is involved at some stage to do the upscaling (Artemis LQ in this example, a version from a few weeks ago...).

One IMO powerfull technique which is able to do this is SPMC:
https://eng.uber.com/research/detail...er-resolution/
A good example (taken from the author's paper):


There is a lot going on in this field - personaly I am always torn between "finish it up" and "oh, ah better method arose, letīs start from scratch all over again"

I hope my long writing made some sense and helped a little..
Attached Images
  
ReinerSchweinlin is offline   Reply With Quote
Old 17th October 2020, 03:52   #509  |  Link
neo_sapien
Registered User
 
Join Date: Jan 2002
Location: USA
Posts: 249
I'm experimenting with DS9, both at 480p 23.976fps and upscaled to 1080p at 23.976fps. For some reason Vapoursynth runs 10x faster than Avisynth for me (I'm using Staxrip 2.1.3.0), I get 2.6fps with Avisynth and 26fps with Vapoursynth. Because of that, I'm trying to convert this Avisynth script to Vapoursynth:

Code:
TFM()
TDecimate()
QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, TR2=4)
Repair(QTGMC2,QTGMC3, 9)
This is what I've come up with in Vapoursynth, but I don't know if it's correct or not:

Code:
clip = core.std.SetFieldBased(clip, 2) # 1 = BFF, 2 = TFF
clip = core.vivtc.VFM(clip, 1)
clip = core.vivtc.VDecimate(clip)
QTGMC1 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC2 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", InputType=3, TR2=4)
clip = core.rgvs.Repair(QTGMC1,QTGMC2, 9)

Doing a test encode shows that the Vapoursynth version needs an extra 100kbps at CRF-14 in x265, so it sounds like it's not producing the exact same output. Can someone check to see if I have it right?

And I tried using nnedi3cl to upscale, but it seems like I have to use that filter to deinterlace as well, unless I'm misunderstanding. You have to use the "field" function:

Code:
field: Controls the mode of operation (double vs same rate) and which field is kept.

    0 = same rate, keep bottom field
    1 = same rate, keep top field
    2 = double rate (alternates each frame), starts with bottom
    3 = double rate (alternates each frame), starts with top
So this is what I'm coming up with in nnedi3cl, given that I can't figure out how to turn disable deinterlacing and just use it to upscale:

Code:
clip = core.nnedi3cl.NNEDI3CL(clip, field = 1, dh=True, dw=True)
Is there a better way to use NNEDI3CL to just upscale? Preferably to 2.25x the original resolution?
neo_sapien is offline   Reply With Quote
Old 17th October 2020, 08:30   #510  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
Is there a better way to use NNEDI3CL to just upscale? Preferably to 2.25x the original resolution?
You could use nedi3cl_rpow2 (https://github.com/Selur/Vapoursynth...r/edi_rpow2.py) to resize to 4x and then downscale using the downscaler you like most.
Also note that is the Avisynth part of the forum, not Vapoursynth.

Quote:
There is a lot going on in this field - personaly I am always torn between "finish it up" and "oh, ah better method arose, letīs start from scratch all over again"
Yup, feels like every other week a new method pops up,.. hopefully more of them add Vapoursynth (and/or Avisynth) support, since it's a paint to have to create those temporal images.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 17th October 2020 at 08:47.
Selur is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.