Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th May 2019, 05:54   #1  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Frame Blending with Max 80/20 Ratio

Has anyone ever written a version of frame blending that blends with a maximum ratio of 80/20 so that each resulting frame shows clear details of a source frame? And it could be set with a configurable ratio. I don't see any reason why that couldn't be done, and I expect it should actually look pretty neat.

If someone can find or write such a script, I'd put it into FrameRateConverter to see how it performs. I really don't see any reason why that wouldn't work. I'd want a plugin that takes a parameter value between 0 and 50. If 0, it simply repeats source frames. If 50, it behaves like frame blending (blending some frames 50/50). Every other value would provide a result in-between.

It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.

Has this been done before? If not, any easy way to implement it?

Otherwise the easiest is probably to take ConvertFps' source code and edit to change the blending ratio calculation.
MysteryX is offline   Reply With Quote
Old 27th May 2019, 14:32   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,158
Your requirements are a little unclear, methinks.

Quote:
maximum ratio of 80/20
Does that mean, at most 1 in 5 frames blended ?

Quote:
(blending some frames 50/50)
Quote:
It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.
Seems contradictory, does that mean at least 80% of output frames are unblended ? [EDIT: Same as 'at most 1 in 5 blended']

Quote:
Every other value would provide a result in-between.
Based on what ?

I'm not volunteering for this, just want clarification.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2019 at 21:02.
StainlessS is offline   Reply With Quote
Old 28th May 2019, 02:29   #3  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,571
My interpretation is that he is asking for the new intermediate frame to not be a 50/50 blend of the two adjacent frames, but to have more weight from either the preceding or succeeding frame so that the result would be sharper, but at the risk of losing some temporal smoothness. In other words, provide just a "hint" of what that intermediate frame would look like.

Interesting idea.
johnmeyer is offline   Reply With Quote
Old 28th May 2019, 09:35   #4  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,915
Perhaps some ideas for weighted blending can be taken from deinterlacers?
Sharc is offline   Reply With Quote
Old 28th May 2019, 13:14   #5  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,158
So,
Arg=00) = repeat frames only, no blends.
Arg=50) = unrestricted color blending ratio.

Quote:
It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.
Assuming Arg=25 for above, where 'source frame' should perhaps read something like 'one or other source frame', as there are obviously gonna be 2 source frames to any blend,
ie between (100:0) and (80:20)% OR (0:100) and (20:80%)

Maybe ? [even with above, still unclear]

EDIT: Oh, think I got it, Arg, always applies to say, earlier of the two blend src frames [or always later of the two blend frames, depends on implementation].
(100:0) -> (80:20)% OR (0:100) -> (20:80%), amount to the same thing:-

EDIT: Nah, was confused again.
maybe where arg range 0.0 -> 50.0

Code:
if (Ratio < Arg ||  (100.0-Ratio) < Arg) Then Blend, else Copy.
Is above as required MX ?

EDIT:
Also, what if decision COPY ie dupe, any benefit where dupe source is Src1 OR Src2, based on luma diff between (Src1-1)<->Src1 and Src1<->Src2),
any benefit, or dont care ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2019 at 14:26.
StainlessS is offline   Reply With Quote
Old 28th May 2019, 18:40   #6  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Quote:
Originally Posted by StainlessS View Post
Is above as required MX?

EDIT:
Also, what if decision COPY ie dupe, any benefit where dupe source is Src1 OR Src2, based on luma diff between (Src1-1)<->Src1 and Src1<->Src2),
any benefit, or dont care ?
You're confusing me. Johnmeyer got it. An interpolated frame is based on the previous and next frame. The only question is with what ratio you blend them together.

As to how that would be calculated, that's a different story.

I have no clue what you're saying about dupes.
MysteryX is offline   Reply With Quote
Old 28th May 2019, 18:58   #7  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,668
80:20 is going to look like a jerky mess.
videoh is offline   Reply With Quote
Old 28th May 2019, 20:15   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,158
Quote:
Otherwise the easiest is probably to take ConvertFps' source code and edit to change the blending ratio calculation.
Sorry MX. above ChangeFPS got me thinking you were talking bout simple blending FrameRate change, Not interpolation, hence confusion. Just ignore me, not feeling too chippa.

EDIT:
Quote:
Has anyone ever written a version of frame blending that blends with a maximum ratio of 80/20
Quote:
If 0, it simply repeats source frames. If 50, it behaves like frame blending
Of course above (+ more) also assisted in the confusion. [EDIT: Repeats == Dupes]

EDIT: And if Interpolating, would be Time related rather than blend ratio, not quite the same thing.

Think maybe VideoH nailed it [right into the coffin ]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2019 at 22:35.
StainlessS is offline   Reply With Quote
Old 29th May 2019, 06:34   #9  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Quote:
Originally Posted by videoh View Post
80:20 is going to look like a jerky mess.
I don't think so. Not more than watching a video at... 24fps!! with duplicate frames.

Or yes jerky mess but much less than interpolating with ChangeFps.

It's about finding a compromise between ChangeFps (blend) and ConvertFps (copy) when interpolating 24fps to 60fps.

This will be especially useful with FrameRateConverter as it does a really good job at identifying interpolation artifacts but none of the 2 does a good job at filling those zones. ConvertFps makes the image "jump" around while ChangeFps gives "shadow" effects by blending an arm 50/50 between 2 frames. If we're displaying 80/20, we eliminate the double-shadow effect.

Last edited by MysteryX; 29th May 2019 at 06:37.
MysteryX is offline   Reply With Quote
Old 29th May 2019, 13:41   #10  |  Link
Andouille
Registered sausage
 
Andouille's Avatar
 
Join Date: May 2012
Posts: 78
Average plugin ?
http://avisynth.nl/index.php/Average
Andouille is offline   Reply With Quote
Old 29th May 2019, 15:31   #11  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Maybe something could be done with Average, by preparing 2 clips with the right opacity. The challenge is that the opacity will vary frame by frame depending on the position of the intermediate frame between the 2 source frames around it. That, I really don't see how to calculate that other that writing a plugin -- which really should be quite simple.
MysteryX is offline   Reply With Quote
Old 29th May 2019, 15:53   #12  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,158
Are you able to compute the required time (0.0->1.0) distance between the two source frames, and is there ever more than a single interp/blend frame to be synthesized [between the two src frames] ?

EDIT: Required time being after you have done any messing due to your 80%:20% type stuff.

EDIT: And I point out this again, but using T=0.0->1.0 rather than percent Ratio, and Arg 0.0->0.5 rather than 0->50 .
Code:
if (Ratio < Arg ||  (100.0-Ratio) < Arg) Then Blend, else Copy.
Something like
Code:
if (T < Arg ||  (1.0-T) < Arg) Then Blend, else Copy.
EDIT: Maybe you can extract some code out of this (BlendDrops_MI) :- http://forum.doom9.org/showthread.ph...86#post1764486

relevant bit supports multiple blends between two src frames, with a few comments added
Code:
    if(Replace==1) {
        StartU = RT_DBaseGetField(DB,n,2)       # Src1 frame
        EndU   = RT_DBaseGetField(DB,n,3)       # Src2 Frame
        Dif    = RT_LumaPixelsDifferent(clp,clp,n=StartU,n2=EndU,Thresh=DifByN,Matrix=Matrix) # scene change between StartU and EndU ?
        Pos    = n-StartU                       # blend index, 1 = first blended frame, EndU-1 is last blended frame index
        frames = EndU-StartU-1                  # Number of frames that will be blended
        if(Dif<Th2) {                           # No scene change
            Weight = 1.0 *  Pos / (Frames+1)
            Last   = Merge(clp.Trim(StartU,-1),clp.Trim(EndU,-1),Weight)
            (Show)?RT_Subtitle("REPLACED: StartU=%d EndU=%d\nPos=%d Frames=%d BlendWeight=%f\nDif Start->End=%f",
                \ StartU,EndU,Pos,Frames,Weight,Dif,align=5):NOP
        } Else {                                # Scene change
            Dif2 = RT_LumaPixelsDifferent(clp,clp,n=StartU,n2=EndU-1,Thresh=DifByN,Matrix=Matrix)
            if(Frames > 1 && Dif2 < Th2) {
                Weight = 1.0 *  Pos / Frames
                Last   = Merge(clp.Trim(StartU,-1),clp.Trim(EndU-1,-1),Weight)
                (Show)?RT_Subtitle("REPLACED: StartU=%d EndU=%d\nPos=%d Frames=%d BlendWeight=%f\nDif Start->End=%f" +
                    \ "\n\a!Because StartU->EndU dif > Th2,\nwe used frame before EndU as terminial Frame\n" +
                    \ "Dif Start->(EndU-1) is %f)"
                    \ ,StartU,EndU,Pos,Frames,Weight,Dif,Dif2,align=5):NOP
            } Else {
                (Show)?RT_Subtitle("SKIP REPLACMENT: StartU=%d EndU=%d\nDif Start->End=%f(Th2)",
                    \ StartU,EndU,Dif,align=5):NOP
            }
        }
    }
EDIT: StartU and EndU are Unique frames, ie non dupes, distinct from their precdessor frame..
EDIT: Above code, input and output frames count same, ie not upping framerate, so Frames would be the number of frames to insert between your two source frames.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 29th May 2019 at 17:31.
StainlessS is offline   Reply With Quote
Old 29th May 2019, 17:19   #13  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Doing it with a GScript might be possible but will be complex and slow; but could serve as proof of concept. Taking the source code of ChangeFps and altering a few lines is probably the way to go. And yes if you go from 24fps to 60fps there is more than 1 interpolated frame between each frames.
MysteryX is offline   Reply With Quote
Old 30th May 2019, 03:51   #14  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
I've got way too much work to do on other stuff to spend time programming Avisynth, but here's what I'm thinking.

ChangeFps has SSIM code to blend 2 frames with a specified ratio, and has the code to calculate that ratio based on the inter-frame position within 2 adjacent frames. All we need to alter that ratio-calculation function. We can use the exact same SSIM code.
MysteryX is offline   Reply With Quote
Old 31st May 2019, 19:11   #15  |  Link
Dreamject
Registered User
 
Join Date: Dec 2018
Posts: 140
Does it have any sense?

Frame A, Frame B =FA,FB
1 blended frames between them mean FA, 50%of FA and 50%of FB, FB
2 mean FA, 100%/(1*3) FA 100%-100%/(1*3) FB, 100%/(2*3) FA 100%- 100%/(2*3)FB, FB
...
x mean i=x-1, then FA,..., 100%(i*(1+x)) FA 100%-100%(i*(1+x))FB...,FB I can make mistake, but...

If you force using 80% of FA it's not good.

Its like FA,80%FA,FB

If you wanna interpolate 24fps to 60 you need 2.5 multipler

Like A B C D (4*2.5)=10
A 80%A B 80%B 80%C C 80%C 80%D D 80%D

Imho its just more laggy

Last edited by Dreamject; 31st May 2019 at 19:22.
Dreamject is offline   Reply With Quote
Old 1st June 2019, 13:11   #16  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,534
Hah! Got it working. Figured out it would be faster to just do it than to talk about it.

Mostly only had to insert this in ConvertFps, but did run into other issues making the job longer.

Code:
if (mix_ratio < half)
    mix_ratio = mix_ratio * ratio / 100;
else
    mix_ratio = one - ((one - mix_ratio) * ratio / 100);
It's looking great. A 50% reduction of blending ratios removes the ghost effect on moving hands.
MysteryX is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:10.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.