GenMotion and DePan plugins for global motion compensation

[Fizick]

I release new version DePan 1.6, August 5, 2005. Here is changelog:
- added YUY2 support (It is the main addition);
- disabled improved=true mode in DePanEstimate. (It seems, it was always broken );
- changed default subpixel=1 (bilinear) for DepanInterleave as quite sufficient for denoising and more fast

[Fizick]

New Version 1.7, September 5, 2005 - DePanStabilize: added parameter initzoom - minimal zoom;
changed limits dxmax, dymax, zoommax, rotmax from hard to soft with larger slope non-linearity.
Changed cache.

[scharfis_brain]

can you add an expanding feature to depanstabilize?

so I can do some kind of panorama out of quick pans.

[Fizick]

I do not understand you.

[scharfis_brain]

When I process VHS, I need to crop away 8px around the image, because there is garbage at this image borders.
So 704x576 becomes 688x560.
This video I feed into depanstabilize.
It is set up to use prev. & next frames to fill the borders.
After this I need to add an 8px border round the image to get 704x576 again.

Is it possible to skip the addborders? So that I get more visible image due to GMC from depanstabilize in shaking scenes?

EDIT: I do not mean a zoom here!

[Fizick]

We can not fill all (and whole) borders so.

[scharfis_brain]

with deshaker I can output a bigger than the input frame without zooming.

Assume an 320x240 input video (Digital photo camera)


the resolution at the right side of the panel is set to 640x480 (doubled height & width). to compensate zooming, the extra zoom factor is set to 0.5 .

This leads to an expanded frame without resizing and without constant black borders (every movement fills them!)
, but for sure not all borders are filled. I am aware of that.

I would be happy to see this featur in depan, too

[Fizick]

So, in Deshaker you get big output frame, with main video in central part only, with big black (partially filled) borders?
So, you want, that output video from depanstabilize will have more width and height size, than input?
it is good for deshaker with many frames used for filling.
Some time ago i think about it, but for more simple boundary proccesing.
Some code re-design will be needed.

May be it is better to add "cropping" option to depan?
and use the "cropping" boundaries as limited mirror positions or solid fill.
Of course, you may use external Avisynth command for these now.
May be i can add some partial fill by previous frame too. But i am not sure.

[videoFred]

I use Depan for frame rate conversion, now.
With very nice results.

AssumeFPS(12.50) > ChangeFPS(25.00) >> Depan Stabilise
Gives very s-m-o-o-t-h panning.

But I do not understand the 'cutof' parameter
Suppose I set it to 2HZ.
2HZ of what?

I also do not understand the 'damping' parameter

But I begin to understand all the other parameters

Fred.

[Fizick]

I use innertial method. consider camera mounted on some inertial platform, with some damped spring (not fixed).
there is some intrinsic frequency for this mechanic system as oscillator.
and threre is its damping (relative factor).
Cutoff frequency is really not strict cutoff, but simply some characteristic frequency value.
Higher frequencies are suppressed. The higher, the more.
(it is simplified algo).

2 Hz is real frequency of shaking.

[loba]

Originally Posted by Fizick

2 Hz is real frequency of shaking.

Does it mean 2 periods per sec?

[Chainmax]

A long time ago, I read a recommendation from scharfis_brain to Boulder of using DePanInterleave in order to improve a script, saying that it helped a lot on video stuff as it usually contains panning/zooming. I have a few very crappy short clips from a digital camera and want to make something discernable out of.

In the previous thread I made about this, the following recommendations were made to me:

i=ConvertToYV12()
mdata=DePanEstimate(i)
r=2 # interleaveradius
DePanInterleave(i,data=mdata,prev=r,next=r)
yourfiltershere()
selectevery(r+r+1,r)
DePanStabilize(last,data=mdata)

Code:
i=ConvertToYV12()
mdata=DePanEstimate(i)
r=2 # interleaveradius
DePanInterleave(i,data=mdata,prev=r,next=r)
yourfiltershere()
i=selectevery(r+r+1,r)
mdata=DePanEstimate(i)
DePanStabilize(i,data=mdata)

None of those options seem to work, in both cases black borders appear and change size in a quite dizzying manner. I can provide one of the clips for you guys to examine if needed.

[johnmeyer]

Are you trying to take 10 fps progressive "video" from a still camera and change its frame rate? On what device do you plan to view it? (NTSC TV, HD TV, computer monitor, etc.).

I have had good luck with Motionperfect as a way to introduce intermediate frames in order to smooth MJPEG video from Canon still cameras. I haven't gotten around to doing it, but I have a 90% complete AVISynth script that accomplishes more or less the same thing using MVTools.

[Chainmax]

The sources are indeed extremely crappy 160x112 videos from a still camera, but the source is 25fps and I don't intend to change the framerate, just deshake the badly done pans. After that, the stream would be cleaned up as much as possible while resizing to 320x256, borders will be added, and it will all be encoded to MPEG2 in order to author a miniDVD.

[Fizick]

Chainmax,
Both your script are generally correct (use first, i.e. second simply slower).
But you must try tune optional parameters.
1. Firstly check, that depanestimate make correct motion estimation. Temporary add "return mdata" line to script after depanestimate,
and add parameter to it:

i=ConvertToYV12()
mdata=depanestimate(info=true, trust=4)
return mdata

Check data, base frame must be changed on scenechanges only.
If not, decrease the "trust" value.
2.Now tune depanstimate (remove "return mdata" of course)
It can not create perfect picture. DEPan does not know, what must be in place of moved borsers (= empty black spaces).
But you may try add some data from previous frames, use mirror, or zoom.
Read DePanStabilize documentation. Then i will aswer your questions.
3. You also may conside apply motion stabilization after resising (320x256) for more smooth results.

[Chainmax]

I see, thanks for the suggestions.

[Chainmax]

I didn't understand how to tweak the trust parameter, but after some experimenting I found that lower values increased the thickness of the black borders, so I raised it until there were no black borders. I also put the DePanStabilize line after the resize as per your recommendation. This is the script I'm currently using:

i=ConvertToYV12()
mdata=DePanEstimate(i,trust=23)
r=2
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
ffdshow("default")
SelectEvery(r+r+1,r)
Lanczos4Resize(320,256)
DePanStabilize(last,data=mdata)
LimitedSharpen(strength=500)
AddGrain(10,0,0)
AddBorders(16,16,16,16)

Thanks for the help .


P.S: how do different values for r affect the result?

[Fizick]

No,no! It is wrong!
1. You really disabled the motion compensation when you rise trust so much.
So, why use DePan?
Decrease trust to default!
2. If you resize clip, its motion is changed, so you MUST use second DePanEstimate.
Or resise at begining of the script (before first estimation and denoise), but it may be not good.
3. Add prev=r,next=r,mirror=15 to DePanStabilize too.
4. r is frame range of motion estimation. It must be used for estimation too.
ffdshow("default") is denoiser? I do not know how good is it.
But try use r=1.

Try this for example:

i=ConvertToYV12()
r=1
mdata=DePanEstimate(i,trust=4,range=r)
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
ffdshow("default")
SelectEvery(r+r+1,r)
Lanczos4Resize(320,256)
i=last
mdata=DePanEstimate(i,trust=4,range=r)
DePanStabilize(i,data=mdata,cutoff=1,prev=1,next=1,mirror=15)
LimitedSharpen(strength=500)
AddGrain(10,0,0)
AddBorders(16,16,16,16)

may be you will need in initzoom

[Fizick]

What video does you try to process (link to short clip)?

[Chainmax]

So, the more trust the more compensation? No wonder the borders disappeared at trust=23 .
ffdshow("default") is a line that allows you to use whatever postprocessing options you select in ffdshow's "video decode configuration" section. In this case, I was using SPP deblocking at strength 100%. It's not good as it loses too much detail, but it's the only deblocker (and I tried lots) that could get rid of the horrible blocking in the clip. I'll try your suggestion and report back. In the meantime, you can download one of the clips here.