Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th February 2021, 07:13   #41  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by markfilipak View Post
But in an effort to find primary sources, do you know how I can query the capabilities of InterFrame? I tried a textual search of all the files for "havsfunc" hoping to extract strings from the file (much as you might extract the version number from an executable's StringFileInfo), but the search didn't get any hits. I have 2, slightly differing copies of 'InterFrame2.avsi' (same version #: 2.8.2 by SubJunk) and that's helped, but it seems from them that the 'Cores=<n>' option is not there. I don't know whether you diddle with InterFrame.

for "capabilities" did you mean settings or switches?

cores=<n> is for the avisynth version, running on CPU , with GPU=false

For vapoursynth, using CPU, there is automatic threading so cores=<n> switch is irrelevant. Interframe for vapoursynth was a direct port from the avisynth version, and the avisynth version has a html documentation file for the settings , and for the most part it's applicable and "human readable"

But the best way for that type of info is to look at the havsfunc.py for any information because that's the actual version you are using. Open it up in notepad++ or similar text edtior. Yes, it's not exactly user friendly readable, but it gives you the actual switches you can use. Look at the InterFrame section and compare with the avs html docs

Interframe is a convenience wrapper function, to simply commands . It does not give you access to all the individual settings - if wanted explictly change the horizontal or vertical block size or subpixel motion estimation, forward or backward vectors etc.. you'd have to use svpflow or mvtools outside of interframe
poisondeathray is offline   Reply With Quote
Old 11th February 2021, 09:27   #42  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Yep. Easiest way is too look at the parameters here https://github.com/HomeOfVapourSynth...sfunc.py#L3834

Code:
def InterFrame(Input, Preset='Medium', Tuning='Film', NewNum=None, NewDen=1, GPU=False, InputType='2D', OverrideAlgo=None, OverrideArea=None, FrameDouble=False):
....
    if Preset not in ['medium', 'fast', 'faster', 'fastest']:
        raise vs.Error(f"InterFrame: '{Preset}' is not a valid preset")

    if Tuning not in ['film', 'smooth', 'animation', 'weak']:
        raise vs.Error(f"InterFrame: '{Tuning}' is not a valid tuning")

    if InputType not in ['2D', 'SBS', 'OU', 'HSBS', 'HOU']:
        raise vs.Error(f"InterFrame: '{InputType}' is not a valid InputType")
And at the avisynth doc https://www.spirton.com/uploads/Inte...terFrame2.html
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 21st April 2021, 11:27   #43  |  Link
markfilipak
Registered User
 
markfilipak's Avatar
 
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 277
Quote:
Originally Posted by poisondeathray View Post
for "capabilities" did you mean settings or switches?
I guess what I meant was how do I 'talk' to SVSuper, SVAnalyse, SVConvert, and SVSmoothFps (though I didn't know their names at the time).
Quote:
Originally Posted by poisondeathray View Post
Interframe is a convenience wrapper function, to simply commands . It does not give you access to all the individual settings - if wanted explictly change the horizontal or vertical block size or subpixel motion estimation, forward or backward vectors etc.. you'd have to use svpflow or mvtools outside of interframe
Yes, thanks, I understand.

I've made a lot of slow progress. FYI, I've diddled InterFrame extensively... maybe 50 differing tests. I've hit some limits...

You undoubtedly tread my path long ago, and I don't want to burden you, so read the '====='-delimited section quickly just to get the flavor.

=====

The source video is 23.976fps, 1920x1080. It has strong 'picket fence'-type backgrounds during panning shots that track a walking man -- the man appears stationary as the background pans behind him -- then he stops. There appears to be 3 problems:

1, After the man stops, the 'picket fence' then exhibits large, flashing, 'phantoms': single-frame, blocky reverse-image areas that are somewhat like afterimages. The 'phantoms' are nowhere near the man but are way off to the right, overlaying parts of the 'picket fence' -- it's actually a wooden Chinese screen but 'picket fence' is the term I've read in Doom9 Avisynth posts from 2017. Curiously, the 'phantom' affects only the verticals in the 'picket fence', not the horizontals (which remain sharp and well defined). During the 'phantom' image distortions, the 'picket fence' is either stationary or is moving very slightly (almost pel-fractionally).

2, Before the man stops (i.e. as he 'walks' during the panning shot), he seems to 'drag' the patterned background with him, but only within the approximately 20 to 30 pels that directly surround his body. This is what the SVSmoothFps documentation calls 'halo'.

The 2 problems above exist for all values of 'SVSmoothFps.algo:' (i.e. 'algo:0', 'algo:1', 'algo:11', 'algo:13', 'algo:21', and 'algo:23').

If I raise 'SVSmoothFps.mask.area:' (default: 0) to 100, then 200, then 300, then 400, the 'phantom' and the 'halo' are both substantially less noticeable at 'area:200', then are overcome (but not really) at 'area:300'. However, doing so exposes the 3rd problem.

3, At higher 'area' settings, pixel blending seems to kick in (even for 'algo:0' and 'algo:1'). Inside the blending, shiny highlights on vertical columns in the scene begin to judder at 'area:300' and get worse at 'area:400'.

At 'area:300', single stepping frame-by-frame shows that the judder is contained within the general area of the vertical highlights -- that the highlights are 'vibrating' -- but the judder doesn't affect the sides of the columns.

Then, if 'area:' is raised to 400, the actual bodies and sides of the columns themselves exhibit small but substantial judder. Oddly, the judder is at 24 judders/s., not at 60 judders/s.

So it appears that I can either avoid judder at 'area:' values below 300 (and than suffer some 'phantoms'), or I can avoid 'phantoms' at 'area:' values above 300 (and then suffer some judder), but there is no 'sweet spot' that avoids both 'phantoms' and judder. Both 'phantoms' and 'judder' (and to a lesser extent, 'halo's) appear to depend on video image content, not rates or other frame mechanics, so trying to find a 'sweet spot' that works for all videos is probably a fool's errand.

Regarding the 'halo's, higher 'area:' values don't have much effect. What does have effect is 'algo' values. 'algo:11', 'algo:13', 'algo:21', and 'algo:23' fuzz the 'halo's, but they are there for all 'algo:'s.

For what it's worth, I don't like the fuzzing that 'algo:11', 'algo:13', 'algo:21', and 'algo:23' do. I consider motion vector interpolation to be a mechanical approach while the fuzzing is a cosmetic approach. I don't know why the higher 'algo:' settings apply any cosmetics and I wish they didn't.

=====

Here's the 'meat' of this meal:

Issue #1:
Can I call SVSmoothFps (and perhaps SVSuper, SVAnalyse, and SVConvert) myself, outside of InterFrame? Would I need to write my own InterFrame-style python script? Would I put that script into Scripts\havsfunc.py? Would I need to 'install' it? Or would I 'compile' it into Scripts\__pycache__\havsfunc.cpython-38.pyc? Generally, how would I make VapourSynth 'aware' of its existence? (A link to documentation would do, and if it's already there, I apologize for asking but I'm getting pretty wasted groping in the dark).

Issue #2:
It appears that SVSmoothFps.scene.mode = '1m' interpolates (via MVs) from 24fps to 48fps (and then repeats some of the interpolated frames to get to 60fps), or that SVSmoothFps.scene.mode = '2m' interpolates 24fps to 72fps (and then discards some of the interpolated frames to get to 60fps). I think something better can be done: 1-to-5, direct interpolation from 24fps to 60fps via scaled, temporal vector interpolation (like a weighted 'mix' filter in ffmpeg, and without 'dynamic medians'). I think that doing so could eliminate/avoid all of the 'phantom' 'halo' and judder problems I've seen.

In that regard, do you know of anyone who would like to help and who might like a bounty?

Warm Regards,
Mark.
markfilipak is offline   Reply With Quote
Old 21st April 2021, 17:04   #44  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by markfilipak View Post
Both 'phantoms' and 'judder' (and to a lesser extent, 'halo's) appear to depend on video image content, not rates or other frame mechanics, so trying to find a 'sweet spot' that works for all videos is probably a fool's errand.
Yes, those are typical "fail" scenarios for optical flow - all kinds - even expensive professional OF solutions

One set of settings that improves frames from one section, will make another section worse.

When it works, it can be ok. But sometimes artifacts are too distracting. Either live with it, or don't do anything.



Quote:

Here's the 'meat' of this meal:

Issue #1:
Can I call SVSmoothFps (and perhaps SVSuper, SVAnalyse, and SVConvert) myself, outside of InterFrame? Would I need to write my own InterFrame-style python script? Would I put that script into Scripts\havsfunc.py? Would I need to 'install' it? Or would I 'compile' it into Scripts\__pycache__\havsfunc.cpython-38.pyc? Generally, how would I make VapourSynth 'aware' of its existence? (A link to documentation would do, and if it's already there, I apologize for asking but I'm getting pretty wasted groping in the dark).

The SVPFlow plugins need to be loaded; you already have them loaded if you used interframe before

There are example scripts on the site
https://www.svp-team.com/wiki/Manual...ursynth_script


Quote:
Issue #2:
It appears that SVSmoothFps.scene.mode = '1m' interpolates (via MVs) from 24fps to 48fps (and then repeats some of the interpolated frames to get to 60fps), or that SVSmoothFps.scene.mode = '2m' interpolates 24fps to 72fps (and then discards some of the interpolated frames to get to 60fps). I think something better can be done: 1-to-5, direct interpolation from 24fps to 60fps via scaled, temporal vector interpolation (like a weighted 'mix' filter in ffmpeg, and without 'dynamic medians'). I think that doing so could eliminate/avoid all of the 'phantom' 'halo' and judder problems I've seen.

Repeating patterns aka the protoypical "picket fence" scenario - causes problems with ALL optical flow algorithms. The solution in other programs is some manual guidance - motion tracking and masking. Motion tracking to guide the estimation, masking to indicate object boundaries and reduce boundary artifacts. It's just not possible with some automatic script - You will always get some artifacts.

"AI" research methods such as DAIN, RIFE can sometimes produce cleaner results, especially on rotational vectors, but the "picket fence" issues are still very much a problem with them too .

Object boundaries and occlusions are still very much a problem for all algorithms. It's just impossible for computer at this point in time to track and distinguish when objects change shape or go behind other objects and reappear



For cadence issues, svpflow is closed plugin. You can ask in the svpflow forum.

But MVTools2 has "time" parameter for functions like mflowinter, where you should be able to adjust the phase for interpolated frames

Quote:
time

Interpolation time position between frames, in percent. Default value of 50.0 is half-way.
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.