GenMotion and DePan plugins for global motion compensation

[Fizick]

GenMotion - General Motion Compensation
C-Plugin for Avisynth 2.5
http://bag.hotmail.ru

This plugin makes general motion compensation (generates a motion),
using ready-made general motion estimation data.
As a such data, the filter uses a log file of Deshaker VirtualDub plugin (by Gunnar Thalin).
This plugin may be used for:
- simple partial motion stabilization (not as good as Deshaker);
- motion compensation for strong temporal denoising and clip (film) restoration.

P.S. Now I know: correct English word is GLOBAL, not GENERAL.

P.P.S. The GenMotion plugin is now obsolete and not supported.

Now I release once more similar plugin!
It has got name DePan - tools for estimation and compensation of global motion (pan).
The DePan plugin replaces my slow experimental GenMotion C-plugin.
DePan is native Avisynth 2.5 plugin.
The DePan works without Deshaker, in one pass, has many options.
See posts at page 3 and below.
The download URL (see depan link)

http://bag.hotmail.ru

[morsa]

Fizick read this post


http://forum.doom9.org/showthread.php?s=&threadid=66682


I guess you are getting to something good!!

[Mango Madness]

though I haven't tried it out yet, the process seems a little cumbersome. Would it be possible to write a tool to automate some of the process?

[Fizick]

May be, it is possible to load Deshaker in avisynth.
But I dont know how.

[Mango Madness]

it's in the included avisynth documentation concerning how to import vdub filters.

[Fizick]

New version 0.5, 28 December 2003 - corrected formula for delta<0 (thanks to Gunnar Thalin), some bugs fixed

[Kintaro]

I never heard of deshaker before, and was pretty amazed when trying it for the first time. Unfortunaly the vdub filter works MUCH better compared to GenMotion, and so I tried importing it. Whenever Im importing it vdub crashes instantly, and Im not good enough to know whats wrong

function VD_DeShaker(clip "clip", int "xy", int "mode", int "block_size", int "differential_search_range",
 \ float "source_pixel_ar", int float "source_pixel_ar_preset", float "destination_pixel_ar",
 \ int "destination_pixel_ar_preset", int "dest_width", int "dest_heigth", int "scale", int "use_pixels",
 \ int "color_mode", int "motionsmooth_horp", int "motionsmooth_verp", int "motionsmooth_rota",
 \ int "motionsmooth_zoom", int "move_wrong_direction", int "video_output", int "edge_compensation",
 \ int "resampling", int "skip_if_blocks_ok", int "initial_search_range", int "discard_match_value",
 \ int "discard_2nd_best_match",
 \ string "logfile", bool "append_log", bool "ignore_image_area_out", int "ignore_out_l", int "ignore_out_r",
 \ int "ignore_out_t", int "ignore_out_b", bool "ignore_image_area_in", int "ignore_in_l", int "ignore_in_r",
 \ int "ignore_in_t", int "ignore_in_b", int "video_type_a", int "video_type_b", float "extra_zoom",
 \ int "maxcorrect_horp", int "maxcorrect_verp", int "maxcorrect_rota", int "maxcorrect_zoom",
 \ bool "use_previous_to_fill", bool "use_future_to_fill", int "previous_frames", int "future_frames",
 \ bool "let_area_follow_motion"){
LoadVirtualDubPlugin(VirtualDub_plugin_directory+"\Deshaker.vdf","_VD_DeShaker",1)
return clip._VD_DeShaker(default(xy,4), default(mode,1), default(block_size,30),
 \ default(differential_search_range,4), default(source_pixel_ar,1), default(source_pixel_ar_preset,0),
 \ default(destination_pixel_ar,1), default(destination_pixel_ar_preset,0), default(dest_width,640),
 \ default(dest_heigth,480), default(scale,1), default(use_pixels,2), default(color_mode,1),
 \ default(motionsmooth_horp,400), default(motionsmooth_verp,400), default(motionsmooth_rota,400),
 \ default(motionsmooth_zoom,1500), default(move_wrong_direction,4), default(video_output,1),
 \ default(edge_compensation,0), default(resampling,2), default(skip_if_blocks_ok,5),
 \ default(initial_search_range,40), default(discard_match_value,300), default(discard_2nd_best_match,4),
 \ default(logfile,"C:\deshaker.log"), default(append_log,false) ?1:0, default(ignore_image_area_out,false) ?1:0,
 \ default(ignore_out_l,0), default(ignore_out_r,0), default(ignore_out_t,0), default(ignore_out_b,0),
 \ default(ignore_image_area_in,false) ?1:0, default(ignore_in_l,0), default(ignore_in_r,0), default(ignore_in_t,0),
 \ default(ignore_in_b,0), default(video_type_a,0), default(video_type_b,0), default(extra_zoom,1),
 \ default(maxcorrect_horp,15), default(maxcorrect_verp,15), default(maxcorrect_rota,5),
 \ default(maxcorrect_zoom,15), default(use_previous_to_fill,false) ?1:0, default(use_future_to_fill,false) ?1:0,
 \ default(previous_frames,30), default(future_frames,30), default(let_area_follow_motion,false) ?1:0)
}

please help me out

[Kintaro]

after sleeping ovet it I just found this evil little thing

\ float "source_pixel_ar", int float "source_pixel_ar_preset",

now its not causing crashes anymore, but Im getting invalid arguments error. any ideas?


edit:
somehow seems to me like this filter is incompatible, even when Im just using the parameter from vdub it says invalid arguments :/

function VD_DeShaker(clip clip){
LoadVirtualDubPlugin(VirtualDub_plugin_directory+"\Deshaker.vdf","_VD_DeShaker")
return clip._VD_DeShaker(4,1,30,4,1,0,1,0,640,480,1,2,1,400,400,400,1500,4,1,1,2,5,40,300,4,
\ "C:\Deshaker.log",0,0,0,0,0,0,0,0,0,0,0,0,0,1,15,15,5,15,0,0,30,30,0)
}

hit me if Im doing something wrong

[Fizick]

New version Genmotion 0.6, 05 April 2004 - add bicubic interpolation (set as default), add grey parameter, remove deflick, correct first and last frame calculation, some reorganization, some speed optimization, use Avisynth C version 0.15, released with source under GNU GPL 2

[SoonUDie]

Fizick, is it possible to turn this into a one-pass plugin? It seems that the deshaker code is closed-source, so you'd have to figure out your own method of getting the motion vectors... but once you do, 1-pass would be relatively easy to do: calculate motion vectors for each cut, then apply the compensation, then move on to the next cut...

Also, I have an idea I've been toying around with for a while - a temporally-based resizer. Perharps you would be interested in coding it? You start out by motion-compensating a scene with sub-pixel accuracy, and output at 200 percent original size. Then, you give unique pixels (the ones obtained through motion compensation) priority over interpolated pixels, because they contain real detail. You can do this because "unique" pixels will not directly overlap from frame to frame; in the best-case scenario of a 1/2 pixel accuracy version, you would be overlaying an image like this:
1 i 2
i i i
3 i 4

Onto an image like this:
i 5 i
6 i 7
i 8 i

to get the final image
1 5 2
6 i 7
3 8 4

Where the numbered pixels are all non-interpolated.

It's not exactly that simple, but I'm sure you get the jist of it.

[SoonUDie]

The 0.6 zip file appears to be corrupted (after re-downloading 3 times...).

[Fizick]

to SoonUdie:
1. temporally-based resizer is some interesting (but not for me right now)
2. Now I try to made one-pass plugin. (I have alfa version).
3. My download quite fine. Anybody else ?
(I can send it by e-mail, if you want. My e-mail is on my WWW).

[SoonUDie]

It turned out that the file I kept downloading was an html file renamed as a zip file. I think it is a problem with the file host.

You may want to change the link to this:
http://www7.newmail.ru/messages/file...enmotion06.zip

[kassandro]

Fizick,
as I can see from the source code of your plugin. The motion information retrieved from the deshaker log file is only 16 bytes per frame (or more precisely per field). Of course, to "deshake" a clip only global motion information is of interest. On the other xvid has a much more sophisticated motion compensation algorithm which is local and not global in nature. If you could transfer this part of xvid to your plugin it could perhaps be improved significantly and may be even faster.

[SoonUDie]

An intersting idea Kassandro, but what would you do on a more local level? Once the camera's been corrected for, what's left?

Perharps, after determining the global motion, the relative motion of macroblocks could be zeroed if under a certain value, potentially increasing compressibility.

... but in any case, I consider full-screen deshaking more important, as you can take care of most other temporal issues with a simple temporalSoften().

[kassandro]

Originally posted by SoonUDie
... but in any case, I consider full-screen deshaking more important, as you can take care of most other temporal issues with a simple temporalSoften().

You certainly did have this thought in mind, when you layed out your temporal resizer. For it you really want to exploit any motion information, to keep normal interpolation at a minimum.

[Fizick]

Kassanro,
I know about the one attempt to use the local motion algorithm from XVID.
Manao make a draft of such plugin.
http://atlas2.tgv.net/~media-video/f...f40bb956da64e2

(French forum)

I try it. The motion vector quality is not very good.

For my old 8 mm kino film, the most important is camera pan.
Of cource, it is not universal.

But good result may be by combining global motion plugin and local motion plugin (DUST).

[Manao]

I'm working on it : the motion estimation is now slightly better than Xvid's one, because I use more predictors than they do. It's much slower though.

I'm currently trying to make the code cleaner, and I don't know yet when I'll be able to release something usable.

[SoonUDie]

One suggestion for improvement: Can you add parameters for max variatians? I've found that Deshaker can be very bad at detecting rotation in motion scenes, so I'd rather just leave it out. I wrote a program to zero the information in the log files, and it worked out well - almost no visual change in low-motion scenes as compared to using rotation, and no problems with artificially introduced rotation. However, it would be much simpler if you could just turn off rotation in the filter.

[Fizick]

O.K., GenMotion version 0.7 is released :

Add parameters dxmax, dymax, rotmax, zoommax.

dxmax - maximum value of x shift in motion data, (default = 100),
dymax - maximum value of y shift in motion data, (default = 50),
rotmax - maximum value of rotation (degree) in motion data, (default = 5),
zoommax - maximum value of zoom in motion data, (default = 1.2),
(if any above max parameter is positive, the motion data value will be limited to it,
if negative, the motion data value will be reseted to null )

P.S. tested very little - try.