SVPflow motion interpolation (Avisynth + Vapoursynth, Linux, Mac?)

[-Vit-]

Originally Posted by chainik_svp

Forward (from frame 0 to frame 1) and backward (from 1 to 0) vectors are finding at the same time

Both those vectors are between the current frame and the next. But the forward vectors returned are from previous frame to current. How do you deal with that in the implementation?

[Atak_Snajpera]

Chainik_svp will you be able to do some magic algorithm to correctly interpolate "moving legs"
http://forum.doom9.org/showthread.ph...06#post1566206

[chainik_svp]

Frame #N in MAnalyse(isb=false) clip contains forward vectors from N-1 to N.
Frame #N in MAnalyse(isb=true) clip contains backward vectors from N+1 to N.

Frame #N in SVAnalyse clip contains forward vectors from frame N to N+1 and backward vectors from N+1 to N.

-Vit-, what's your question?

[-Vit-]

You answered my question .
That means SVAnalyse returns different forward vectors with settings vectors:1 and vectors:3.
I ask because it's important when converting SVAnalyse vectors to MVTools compatible ones. The forward vectors are "shifted" by 1 frame

[chainik_svp]

That means SVAnalyse returns different forward vectors with settings vectors:1 and vectors:3.

Why? o_O With vectors=1 backward vectors will be undefined/zeroes, that's the only difference. At least I've wrote it to work in this way

BTW, smoothfps_params.debug.vectors=true will draw vectors almost like MShow.
If you set 3:1 interpolation then first interpolated frame will show forward vectors and second - backward ones.

-----
Oh, I see. You assume that the description of "vectors" param is right
It's a copy-paste mistake

[-Vit-]

Originally Posted by chainik_svp

Oh, I see. You assume that the description of "vectors" param is right

Ah! Now you see why I asked about implementation, because I was wondering how you honored what the docs said - penalty.prev would be unpleasant to implement with the MVTools approach. The actual way you're doing things is much simpler.

[Zerofool]

Originally Posted by chainik_svp

Zerofool
consider implementing support for YV24 colorspace in a future version

There's no real visual difference in real-time between YV12 and YUY2, but YUY2 processing is 30% slower.
That's the reason why SVPflow supports only YV12 now.
YV24 will be 2 times slower.
So why?

I totally agree, but that's for real-time processing of YV12 sources.
I used MVtools, and I plan to use SVPflow for a different thing - adding motion blur to video games footage (which is RGB), in slow, "offline" manner (so I wouldn't mind if it's slow). When converted to YV12, there's often a visual degradation of the image quality, especially at borders between objects with very different colors (for example, red car on green-ish background). With YV24, this problem is gone, the video looks visually lossless to the source.
When the motion blur is added, the problem with YV12 is less pronounced, but still bugs me, that's why I'd very much like it if SVPflow supported YV24 colorspace (of course, if it's not too much work to implement it).

Originally Posted by gyth

IMO it'd be like subpixel estimation, but with real data instead of interpolated.

Zerofool, you could do something like this. I'm sure there are some gotchas, like chroma placement.

w=width
h=heigth
resize (2 * w, 2 * h)
converttoYV12

#motion stuff

converttoYV24
resize(w, h)

Yeah, my footage has real data (it's RGB).
I've experimented with such approach, for higher precision, but using NNEDI3_rpow2, and I don't think the difference in quality is worth the big loss in speed.
But still, thanks for the suggestion.

[chainik_svp]

-Vit-
All that "temporal" things should be great but they're almost impossible now because they harm multi-threading performance. (This may not be the case here but for consumer software it's very bad)
Every night I think about my own threading inside plugins

Avisynth has different main purpose - non-linear editing. If all we want is just frame-by-frame linear processing then MT caches and "distributor" are not helping
That's why SVP uses a custom Avisynth build that has:
- different distributor that manages threads pool a little more efficient
- tuned caches that hold only up to <some amount> of frames, where <some amount> depends on threads count.

Originally Posted by Zerofool

I plan to use SVPflow for a different thing - adding motion blur to video games footage

Then I think you should do more than posting if you want some features added just for your business o_O

[Atak_Snajpera]

in other words "donate or die"

[Zerofool]

Originally Posted by chainik_svp

Then I think you should do more than posting if you want some features added just for your business o_O

Hey, don't get me wrong, that was just an idea. I guess some developers would be happy to know that some people are using their product in weird, non-standard way, to achieve different things. And expanding the capabilities of their product would only be seen as a positive thing.

It's not business as in Business, I don't get money for that, it's just for personal amateur-grade videos - some of my favorite moments from classic (mostly 5+ years old, rarely newer) games. And hopefully, YV24 support would be useful for other users/gamers as well.

Anyways, good luck to the project.

[chainik_svp]

Originally Posted by Atak_Snajpera

in other words "donate or die"

Why so serious?
Just remember that the one and only goal is to maximize quality and speed for real-time playback. "Removing legs" is OK within this task but adding YV24 is NOT.

Originally Posted by Zerofool

I don't get money for that

Me too
"Just an idea" - ok, we'll append it to the looooong TODO list.

[gyth]

Originally Posted by Zerofool

I've experimented with such approach, for higher precision, but using NNEDI3_rpow2, and I don't think the difference in quality is worth the big loss in speed.

If you want precision, use point resize. (but be on the lookout for chroma shift)

Another possibility is to calculate using YV12 and then apply the motion vectors to the rgb (a separate "YV8" clip for each color).
And with pixel art, using pel=1 should be faster and help keep things pixelated.

[Zerofool]

Originally Posted by chainik_svp

ok, we'll append it to the looooong TODO list.

OK, great. Thank you!

Originally Posted by gyth

If you want precision, use point resize. (but be on the lookout for chroma shift)

Another possibility is to calculate using YV12 and then apply the motion vectors to the rgb (a separate "YV8" clip for each color).
And with pixel art, using pel=1 should be faster and help keep things pixelated.

Thanks for the suggestion, I'll give it a try. Is it OK to ask you for help on how to do that, in a PM, because I don't want to spam this thread?

[Andrey /MAG/]

Originally Posted by Atak_Snajpera

...some magic algorithm to correctly interpolate "moving legs"
http://forum.doom9.org/showthread.ph...06#post1566206

You can try to change some settings in SVAnalyse params:
1. block.w and block.h. The lower value the more precise vector field. Set them to 8.
2. main.search.distance. Use usual positive values or big negative ones. Set it to -10.
3. main.penalty.lambda and main.penalty.plevel.
The more lambda give more coherent movements, but more doubled contours. You need smaller values.
The more plevel give grown of coherence level by level from coarse to finest.
Try this:
main.penalty.lambda=3.0
main.penalty.plevel=1.0

And to change SVSmoothFps params:
4. algo. Set it to 2. Only one source frame will be used. So no double contour.

Fast result animation:

[Terka]

chainik_svp:
thank you!
When aproximately do you think the svtools will be ready for usage?
What advantage do you expect over the mvtools?
Would it be possible to use already calculated vectors for x264 encoding (why should x264 search for vectors, when avisynth already did this)

[Didée]

No. "Vectors ideal for encoding" and "vectors visually reflecting object motion" are two very different ballgames. Also consider the much more complicated situation in x264 - multiple references, multiple adaptive partition sizes, I/P/B-Frames, etc.

In a word, Avisynth's motion vectors are pretty much useless for x264. If at all, they could be used for something like "initial predictors" ... but that levels of ME is pretty fast in x264 anyway. Most probably things would get worse instead of better. In both speed and quality.

[chainik_svp]

Originally Posted by Terka

When aproximately do you think the svtools will be ready for usage?

What do you mean? It's ready and is used in released software called "SVP".

[Terka]

SVPflow only, mvtools has mvdegrain etc.

[chainik_svp]

SVPflow 1.0.2

= support for multiply SVSmoothFps instances needed for proper 3D handling
+ SVConvert function for integration with MVTools-based scripts
+ GPU selection for rendering
= updated SAD/SATD x264 code

How SVConvert should work:

super_params="{pel:1}"
analyse_params="{}"

super = SVSuper(super_params)
vectors = SVAnalyse(super, analyse_params)

forward_mv = SVConvert(vectors, false)
backward_mv = SVConvert(vectors, true)

super_mv = MSuper(pel=1, hpad=0, vpad=0) #padding should be zero here!
MFlowFps(super_mv, backward_mv, forward_mv, num=60, den=1)

[Nevilne]

Awesome stuff, thank you.