Deshaker by Gunnar Thalin

[guth]

Thanks!

Yes, all parameters are in a single string. The reason is that VirtualDub can only handle up to 10 parameters (I think... correct me if I'm wrong).

Smoothness range is -1 to 100000 (where -1 and 0 are special cases). The upper limit is arbitrarily chosen. (If you really want, you should be able to go higher if you do it from a script.)
The math.... is a secret
Well, not really. Without going into specifics I solve four huge equation systems (pan x, pan y, rotation and zoom) where I try to minimize both the motion curvature and the corrections at the same time. The smoothness values are weights that control which of those are more important to minimize. Higher values results in less curvature. I don't think this will help you a lot in choosing good values, but you asked.

[Flexy]

hi guys, i need help there

The problem right now is that this script crashes...what i did (as read elsewhere on a forum) generated jobs and try to load the jobs and run it...but VDubmod crashes. (I have the latest version). Maybe one of you knows more what i do wrong..

I put in some comments in the script for things i have qeustions on Thanks for any help !

guth: Yeah i was the one also emailing you about it. Thanks for reply !



>>> snip <<<

// VirtualDub job list (Sylia script format)
// This is a program generated file -- edit at your own risk.
//
// $numjobs 2
//

// $job "Job 5"
// $input "C:\DATA\test_short.avi"
// $output "C:\DATA\dummy.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DATA\\test_short.avi","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AddComment(0x00000002,"ISFT","VirtualDubMod 1.5.10.1 (build 2366/release)");
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x67706a6d,0,9000,0);
VirtualDub.video.SetCompData(28,"TUpQRxgAAAAAAAAAAgAAAAgAAAACAAAAAQAAAA==");
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");
VirtualDub.video.filters.instance[0].Config("4|1|30|4|1|0|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0");
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,50);
VirtualDub.SaveAVI("C:\\DATA\\dummy.avi");
//
// actually, i do not WANT to save anything here, but i dont know
// how else to create a job for deshaker Pass 1 ??????
//
VirtualDub.Close();

// $endjob
//

// Now PASS 2 <------ !!!!
//--------------------------------------------------
// $job "Job 6"
// $input "C:\DATA\test_short.avi"
// $output "C:\DATA\result.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DATA\\test_short.avi","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AddComment(0x00000002,"ISFT","VirtualDubMod 1.5.10.1 (build 2366/release)");
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x67706a6d,0,9000,0);
VirtualDub.video.SetCompData(28,"TUpQRxgAAAAAAAAAAgAAAAgAAAACAAAAAQAAAA==");
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");
VirtualDub.video.filters.instance[0].Config("4|2|30|4|1|0|1|0|640|480|1|2|1|400|400|400|1500|4|1|2|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0");
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,50);
VirtualDub.SaveAVI("C:\\DATA\\result.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

[Flexy]

It should work. If you really want to know what all the parameters are, let me know. But I recommend making the settings from within VirtualDub.

yeah..the parms are not the problem..i can just save a job/settings and have it in a script. Problem is ratehr i cannot run the script because it crashes. (See other posting)

Unfortunately it's impossible to do both passes internally.

Finally, I would never batch process clips with deshaker. You most probably won't get as good results as you could if you spent some time and finetuned the settings for each clip (and even parts of a clip). Also, personally I would never use the default settings. They are only good for beginners. [/B]

i dont use deshaker for *that* long, so i didnt go in-depth and experimetned with all settings. The defaults are 'good' for my first testing...and i have good results with my vids. 'Tuning'...i can do that later when i have time

An average 'vacation' movie for me consists of about ~50 smaller clips..at elast the last one did and teh next movie might get even bigger. I think i would sit there 'til the year 3000 if i'd analyze each single clip (assuming it's 50 and more)..maybe even each *scene* in each clip....and then do a manual deshake with different params for each clip...that would just be extremely insane work.

I had nice results applying a deshakeover my final result video (which is the 50 clips joined together in Sonic Foundry Vegas)..so...the deshaking looks nice....PROBLEM is the titles....i need to apply the Deshake to the individual clips *before* i render the resulting video in vegas for the titles and everything ok.

[WarpEnterprises]

@guth:
another "technical" question:

How are scenechanges handled?
When do you make the filter snap back to the "0" position?
Is this connected to the value "skip frame" (more precise to the 0,0,1-results generated)?
Is this the same as if a line in the file is manually deleted (as you suggest for human guidance)?

Or summing up: when is in the results (of the motion equations) a "forced" zero crossing?

I may look very curious but although the docu is quite good, some things/settings may be completely clear for you - but not for me, and I can't sleep until I have understood it from the base line

[guth]

@Flexy:
You're right! It totally crashes!
I looked into it now (I never ran jobs before) and it crashes because VirtualDub calls my functions in a slighty unpredictable order (EndProc is called before StartProc and InitProc). Next version will be able to cope with this (I'll send you an unofficial version with a fix by email soon too).

@Warp:
I'm not completely sure I completely understand what you mean, but I think so...
It never snaps back to the 0 position during a single processing. You'll need to split the processing into several parts in order to do that.
Yes, a 0,0,0,1 row is the same as if the row was missing.
The only zero-crossings are the first and last frames that were processed in pass 1 (unless you use adaptive zoom, then there are none).

[Flexy]

Originally posted by guth
[B]@Flexy:
You're right! It totally crashes!
I looked into it now (I never ran jobs before) and it crashes because VirtualDub calls my functions in a slighty unpredictable order (EndProc is called before StartProc and InitProc). Next version will be able to cope with this (I'll send you an unofficial version with a fix by email soon too).

hi gunnar,

thanks !I knew that there was a problem ebcause i read about these crashes before somewhere on another forum where they tried the same and they got these crashes.

Ok..the new version works BETTER and the first couple of runs worked like a charm ! But i just found a minor bug: it crashes when i create a script with the Deshaker option 'no motion vectors shown'...but with the vectors displayed on it seems to work !!! Still testing around


thanks !

[WarpEnterprises]

@guth: that means there is no special treatment of scenechanges.

If I want to treat each scene separated I have to filter it separated.

Wouldn't that be a useful feature to add a scenechange marker (e.g. by manually editing the first frame entry of a scene after the first pass - which will probably already be a 0,0,0,1 row if it is a scenechange) to tell it to _result_ in a 0,0,0,1 ?

@all: it seems the preset of 400 for the smoothing values is pretty low. What are your preferred settings?

[guth]

@Flexy:
My guess is that it crashes because you're using a codec to encode the output of pass 1 that can't handle the extremely small video output (8x8 pixels). It works for me.

@Warp:
Yeah, I guess that could be a useful feature. But it means a lot of work, and I hate programming.
We'll see...

I use all kinds of smoothness values (400-100000). It depends on the clip.

[mrg]

Hi,
it is possibly to use Deshaker for DV without lost the DV Timecode?
Thanks

[WarpEnterprises]

No. Only with direct stream copy, and then you can't use a filter.

[Flexy]

guys, i am in heaven
My next vacation vid will be awesome

I just found a very useful freee program called 'dubman' which aids in creating vdubmod jobs....you can select wildcards for directories etc.

1) save vdubmod processing settings for deshaker pass1
2) save vdubmod processing sttings for deshaker pass2

use these two *.vcf files in 'wildcard' mode in 'dubman' and go thru a whole directory...right now i am deshaking a whole directory in the background...yay

BUT....i have one, related, question...maybe someone can help:

- I am using Deshaker to deshake my clips.
- I am using Dynapel's Motionperfect to convert my 15fps to 24fps. Motionperfect fills in the missing frames...and the results (even if not perfect) are FAR better than what i had with my original 15fps videos !

QUESTION:

What should i do first:

) FIRST convert my clips to a higher fps with MotionPerfect and THEN deshake them ?
) or FIRST deshake them and THEN convert them to higher framerate with MP ?

Logic tells me first to deshake and stabilize the 15fps clips and then let Motionperfect work (which i a doing now) fill in mroe frames

Any tips ?

Also...keep in mind my vids all come in 15fps originally..using MotionPerfect...would you convert them to 24 or should i go NTSC 29,9 fps ????
I figure 24 would be beter since MP would not have to fill in too many 'artificial' frames [artifacts]..but on the other hand if i am planning to make NTSC VCDs...maybe i should just let it generate 29,9 fps instead of only 24 ? (Vide wold be smoother and suited for NTSC VCD..but i'd get more artifacts)

greetings

[mustardman]

Why on earth are you starting with 15fps? Are you using one of those memory stick recorders, or perhaps an old film camera?

Personally, I would convert from 15fps to 30fps, and then use VDub to set the rate to 29.999fps (if you feel it is really necessary).

The reason being (apart from your destination) is that MotionPerfect will then always generate a frame exactly in between the existing frames. I would imagine that doing 15->24 would not work as well. Mind you, having only used MP once (it didn't work with my source - unusual content ) I can't say I have a lot of experience in this area.

[Flexy]

Originally posted by mustardman
Why on earth are you starting with 15fps? Are you using one of those memory stick recorders, or perhaps an old film camera?

Personally, I would convert from 15fps to 30fps, and then use VDub to set the rate to 29.999fps (if you feel it is really necessary).

The reason being (apart from your destination) is that MotionPerfect will then always generate a frame exactly in between the existing frames. I would imagine that doing 15->24 would not work as well. Mind you, having only used MP once (it didn't work with my source - unusual content ) I can't say I have a lot of experience in this area.

hi,

it's not by choice that i us 15fps ! I have a very tiny Canon S230 which does 640x480 vids at 15fps.

HOWEVER - i have *very* good results using deshaker now and MP....not perfect - but far better than ther original vids.
(I guess the 15fps are what you have to live with when you have a cam which would fit into the palm of your hand....still love it because of it's size

Regarding the fps:

I did some testing, but still need more. You're right, MP generates frames in between when i do exacly 30fps (and NOT 29.97 ....however i have to figure out stll whether the results aer better than with 23.976 fps. Reason is a) File size...and b) the fact that twice as much fps via MP means that half of the movie is 'artifcial' while with 24fps only ~60% of frames are added by MP.

[mustardman]

More getting to the issue of wheter MP does calculate a frame exactly in the middle (15 -> 30) or whether it calculates a frame that is "closer" to the preceeding one or the following one. Could give funny looking results? (Again, not used it for that, so I don't know).

If you are going to NTSC video (DVD, videotape, VCD, etc) go to 30fps for the extra little bit of space. Hard drives are pretty cheap these days!

[WarpEnterprises]

MP leaves the original frame only when it is EXACTLY at the same point in time.
ALL other frames are reconstructed using an interpolation algo (based on motion vectors).

So if you have 15fps and go to 25 fps you get:

Source:
1s  -   2s  -   3s  -   4s  -   5s  -   6s  -   7s

Result:

1=1s - 2 - 3 - 4 - 5 - 6=4s - 7 - 8 - 9 - 10 - 11=7s

From 6 resultframes only 1 is an untouched source frame !

[khmann]

yea, so this is pretty ugly and not recently tested, but it used to work for me. When capturing DV, I use a utility that automatically breaks all the scenes into separate files. Then I drop the individual .AVIs onto this batch file and it makes a virtual dub job so that I can do all my motion-analysis in one place.

you shold probably change the paths, and modify the line "echo VirtualDub.video.filters.instance[0].Config" to the set|of|pipe|options apropriate to how you want the pass 1 done.

@echo off
set basename=%0
set path=c:\khmann\bin;%path%
set chain=%temp%\KHJPGTMP.BAT
set chain=d:\dvproc\joblist.jobs
del %chain%

rem cd /d %temp%

echo VDMod auto deshaker.  Drag and drop .JPG files here
echo.

:loop
if !%1!==!! goto done

echo %1

echo // $job "autojob" >>%chain%
echo // $input "%1" >>%chain%
type d:\dvproc\_prcbdy1.txt >> %chain%
echo VirtualDub.Open("%1","",0);  >>%chain%
type d:\dvproc\_prcbdy2.txt >> %chain%
echo VirtualDub.video.filters.instance[0].Config("4|2|24|4|1.21519|4|1.21519|4|720|480|0|1|1|900|600|0|0|1|1|0|2|1|25|350|30|%1.ds|1|0|48|48|24|24|0|0|0|0|0|0|0|1|5|4|5|5|1|1|30|15|1"); >> %chain%
type d:\dvproc\_prcbdy3.txt >> %chain%
echo VirtualDub.SaveAVI("D:\\test.avi"); >> %chain%
echo VirtualDub.Close(); >>%chain%
echo. >>%chain%
echo // $endjob >>%chain%
echo //-------------------------------------------------- >>%chain%

echo. >> %chain%

shift
goto loop

:chain

:done
echo.
@echo on
@if exist %chain% %chain%
@echo off

pause

some other text is included from these files:

_prcbdy1.txt:

// $output "d:\test.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

_prcbdy2.txt:

VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x75796668,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");

_prcbdy3.txt:

VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,57);

not exactly the cleanest solution, but it has served me well.

[Beave]

I have a 720p hdtv source that shakes exactly by one line/pixel up or down. You can easily see this at the bottom edge in the small black part there.
How can I set it up so it only corrects this minimal shaking. Should I maybe rather look into some other filter instead?

[WarpEnterprises]

how often does it shake? every frame?

[Beave]

no, every few frames and sometimes every frame. It's irregular or at least I couldn't find a pattern yet.

[WarpEnterprises]

(1) new version 1.9

(2) still some problems when using with AviSynth - look here