Deshaker by Gunnar Thalin - Page 3

WarpEnterprises · 6th August 2006, 14:03

@guth: the percentage of discarded blocks is a quite important number, yet it can't be seen anywhere. Maybe it could be displayed in the pass1-output along with the other values?

guth · 6th August 2006, 16:57

This is a little complicated.

There are three types of blocks:

A. Ok blocks. Have white motion vectors.
B. Blocks discarded due to bad motion vectors. Have red vectors.
C. Blocks discarded due to bad match. Have no motion vectors.

Which percentage do you want to see? The deep analysis kick-in percentage is compared to the ratio B / (A + B) and the frame discard limit is compared to the ratio A / (A + B + C). I think it makes sense to do that but it's somewhat confusing I guess, especially since it's not documented.

WarpEnterprises · 7th August 2006, 07:07

Quote:

Originally Posted by guth

since it's not documented

... well, now it is

Maybe simply print both?

I used the "skip frame" setting to adjust the sensitivity for big objects moving in the scene. Is this the right setting to tune? (when NOT using deep analysis)

Just another question: if I use scale=quarter does this imply that the found motion is only accurate to +/- 4 pixels (which seems not true) or how else can this value be understood?

guth · 8th August 2006, 18:29

Hmm... I'm not sure I want to print both. We'll see.

The "skip frame" setting is basically just there because if very few motion vectors were good, Deshaker probably got everything wrong so it's probably best to skip the frame completely. There are many ways to deal with big moving objects depending on how big they are, how fast they move and for how long they're in the picture. Skipping the frame completely should only be done as a last resort, or if you're lazy

Quarter scale indeed means that the matching occurs on images that are reduced to 1/4 of the original width and height, but the accuracy is usually better than 4 pixels anyway for two reasons:

1. I sort of use sub pixel accuracy when calculating the best shift for a block. (I do this by fitting a 2nd degree function through the best shift and its two integer neighbors, and their "match probability values", and take the shift where this function has its max.)

2. Any impreciseness in the block vectors will be evened out when I combine them all for the total motion of the frame.

That said, I don't really recommend using quarter scale anyway. Personally I always use full scale.

guth · 12th August 2006, 13:39

New version of Deshaker available for anyone who cares. Get it here.

smok3 · 29th January 2007, 16:41

trying to batch version 2.0 via vdub.exe, but some (most) of the settings for 2nd pass get lost somewhere, should those be in job scripts?

Code:

for example, this seems to be 2nd pass
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|1|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15");

guth · 29th January 2007, 18:15

I'm afraid I don't really understand what you mean. That scripts runs pass 1. Could you give me some more details? (You can email me if you want.)

smok3 · 30th January 2007, 09:00

i did a little batch (which is called from total commander):

Code:

@ECHO OFF
for /f "delims=+++++" %%i in (%1) DO vdub /i "t:\path\vdubdeshake1.script" "%%i" "%%~pi%%~ni_deshaked.avi" ¦ vdub /i "t:\path\vdubdeshake2.script" "%%i" "%%~pi%%~ni_deshaked.avi"

it depends on this 2 vdub scripts:
vdubdeshake1.script

Code:

VirtualDub.Open(VirtualDub.params[0]);
VirtualDub.audio.SetSource(1);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(0);
VirtualDub.video.SetOutputFormat(7);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|1|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15");
VirtualDub.audio.filters.Clear();
VirtualDub.project.ClearTextInfo();
VirtualDub.SaveAVI(VirtualDub.params[1]);
VirtualDub.audio.SetSource(1);

vdubdeshake2.script

Code:

VirtualDub.Open(VirtualDub.params[0]);
VirtualDub.audio.SetSource(1);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(0);
VirtualDub.video.SetOutputFormat(7);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x64737664,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|2|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|3|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|0.7|15|15|5|1|1|1|15|15|0|0|0|0|1|1|0|10|1|15");
VirtualDub.video.filters.Add("sharpen");
VirtualDub.video.filters.instance[1].Config(5);
VirtualDub.audio.filters.Clear();
VirtualDub.project.ClearTextInfo();
VirtualDub.SaveAVI(VirtualDub.params[1]);
VirtualDub.audio.SetSource(1);
VirtualDub.Close();

but i dont get things like:
- use previous and future frames...
- edge compensation mode...
- ect...
(everything seems default)

EDIT: for some unknown reason things are working now...., weird.

------------

other than that, how would i generate a unique name for deshaker log? (like input_filename_deshaker.log) - using this method? EDIT2: solved

guth · 30th January 2007, 20:28

Weird, but good

About unique log files, I don't know. I'm no script guru at all.
But I wouldn't really recommend running deshaker in batch mode at all. The results will rarely be perfect unless you do some supervision.

smok3 · 30th January 2007, 21:14

yes, the idea is to do some short segments using gui, then save those to this 2 scripts and run batch only after that. About unique log name, on 2nd thought it is almost not important since a lot seems to depend on 1st pass anyway, but i will do some renaming in original .bat like;

rename c:\deshaker.log to InputFilepath/InputFilename_deshaker.log
(at the end of the script)

texmex99 · 30th March 2007, 23:08

Just discovered Deshaker and it worked miracles on some handheld ski footage (bit hard to hold the camera still while skiing downhill). I am, however, having problems with another clip that I was hoping to "stabilize". This is a shot from a "fixed" camera (sitting on a deck rail - no tripod available) of the moon rising over the mountains. The clip is 15 minutes long, but will be compressed to about 7 seconds for "effect". Unfortunately, the camera was on the deck, and so was the beer, and whenever someone stepped out on the deck for a beer, the camera shook ever so slightly. I figured Deshaker would have no problems with this clip, but I can't figure out how to adjust the settings to make it work. The frame is mostly black, other than a few lights in fixed positions, car headlights moving through the frame, and then the moon rising. I assume that the easiest way for Deshaker to do its magic is to have it lock on to one of the "fixed" features in the frame, but I can't figure out how to tell it what to lock on to. I've tried isolating several parts of the frame, but it won't "lock on" - I guess it assumes that the black is background and the lights are "moving objects". Anyway, any suggestions from the pros here on how to deshake this scene? I've attached a frame with the "fixed" lights circled - those are the features I need to "lock on" to (i.e. they shouldn't move). Any suggestions would be appreciated! Thanks!

guth · 31st March 2007, 08:00

If you'd like to send me a short problematic sample clip, I'll have a look at it.

texmex99 · 4th April 2007, 11:30

Quote:

Originally Posted by guth

If you'd like to send me a short problematic sample clip, I'll have a look at it.

That would be great - only I'm on vacation at the moment. I'll post a clip as soon as I get back. Thanks so much for this great tool!

wonkey_monkey · 4th April 2007, 11:38

I second that - deshaker is fantastic. I do have one question though...

In the documentation, it says:

Quote:

The motion smoothness calculation is...based on frame count

But for the default of 400 frames, the video doesn't come out as smooth as I would expect. Is it actually measured in frames, or is it a %age (so 400% would be 4 frames)?

David

Terka · 9th April 2007, 10:56

when a motion is fast (+long shutter speed) -the deshaked picture is blurred. is it able to add some motion deblur to this pictures? (like focus magic)

wonkey_monkey · 11th April 2007, 15:35

Guth, are you still working on deshaker? In addition to my previous question, is there any chance that a future version might include perspective correction?

I ask because some stabilised videos suffer from a slight warping if the input is shakey enough (as my time lapse videos can be, and I've seen the same effect on normal video). If deshaker asked for the FOV of the images, it could correct in a more realistic way, rotating the input images in three dimensions rather than just shifting horizontally and vertically.

David

WarpEnterprises · 22nd May 2007, 12:55

I just saw that there is a new Version 2.2.

@guth: I have two questions:
(1) You wrote "Improved handling of new scenes ... allows pass 2 to take appropriate action".
Would you mind giving some details what this action is? Is there a forced "jump" in the resulting deshaking shifts or the like?

(2) regarding the new "Remember discarded areas" option: how is this related to the "blocks OK" percentage?

guth · 26th May 2007, 13:29

Sorry for not replying. I didn't get any notifications of any activity in this thread.

Quote:

Originally Posted by davidhorman

But for the default of 400 frames, the video doesn't come out as smooth as I would expect. Is it actually measured in frames, or is it a %age (so 400% would be 4 frames)?

It's based on frame count. It doesn't mean that it equals the number of frames. What I mean is just that if you have video with more fps (as interlaced video in a way has), you should increase the value to get the same result. If 400 isn't smooth enough for you, just increase it.

Quote:

Originally Posted by Terka

when a motion is fast (+long shutter speed) -the deshaked picture is blurred. is it able to add some motion deblur to this pictures? (like focus magic)

Focus magic looks pretty cool, but it doesn't seem to be able to correct motion blur automatically, you have to manually measure the blur direction and length. That's hard to do automatically. And I'm sure the actual correction isn't easy to do either

. I doubt it will happen.

Quote:

Originally Posted by davidhorman

...is there any chance that a future version might include perspective correction?

Hmmmmm..... I'll have to think about that one.... real hard...

Quote:

Originally Posted by WarpEnterprises

(1) You wrote "Improved handling of new scenes ... allows pass 2 to take appropriate action".
Would you mind giving some details what this action is? Is there a forced "jump" in the resulting deshaking shifts or the like?

It does this:
1. The end of scene 1 and beginning of scene 2 are basically placed at "zero correction". And there's no smoothing over the scene cut, so if scene 1 ends by panning to the right, scene 2 can start by panning to the left, abruptly.
2. It doesn't attempt to use frames from another scene when using previous and future frames to fill in the border.

Quote:

Originally Posted by WarpEnterprises

(2) regarding the new "Remember discarded areas" option: how is this related to the "blocks OK" percentage?

I'm not sure I understand your question. Basically, what I do is use the pattern of good/bad blocks for one frame as a starting point for the next frame. This usually results in that it is able to continue locking to the background when large objects enter the scene.

Terka · 28th May 2007, 12:42

yes to remove the motion blur - the amount of motion is needed to know. and deshaker has this information-direction and amount. thats why im asking about removing motion blur.

guth · 28th May 2007, 17:52

It's not quite that simple, I'm afraid. The motion blur occurs *within* each frame not *between* them. Matching between two motion-blurry frames will be extremely unreliable so you wouldn't be able to get any good information about direction, and specially not about the amount of motion blur, since it can be much different from the amount *between* the frames (the shutter speed doesn't always equal the "frame speed").

30th January 2007, 21:14	#50 \| Link
smok3 brontosaurusrex Join Date: Oct 2001 Posts: 2,392	yes, the idea is to do some short segments using gui, then save those to this 2 scripts and run batch only after that. About unique log name, on 2nd thought it is almost not important since a lot seems to depend on 1st pass anyway, but i will do some renaming in original .bat like; rename c:\deshaker.log to InputFilepath/InputFilename_deshaker.log (at the end of the script) Last edited by smok3; 30th January 2007 at 21:16.

30th March 2007, 23:08	#51 \| Link
texmex99 Registered User Join Date: Jan 2007 Posts: 3	Deshaking a near-black scene Just discovered Deshaker and it worked miracles on some handheld ski footage (bit hard to hold the camera still while skiing downhill). I am, however, having problems with another clip that I was hoping to "stabilize". This is a shot from a "fixed" camera (sitting on a deck rail - no tripod available) of the moon rising over the mountains. The clip is 15 minutes long, but will be compressed to about 7 seconds for "effect". Unfortunately, the camera was on the deck, and so was the beer, and whenever someone stepped out on the deck for a beer, the camera shook ever so slightly. I figured Deshaker would have no problems with this clip, but I can't figure out how to adjust the settings to make it work. The frame is mostly black, other than a few lights in fixed positions, car headlights moving through the frame, and then the moon rising. I assume that the easiest way for Deshaker to do its magic is to have it lock on to one of the "fixed" features in the frame, but I can't figure out how to tell it what to lock on to. I've tried isolating several parts of the frame, but it won't "lock on" - I guess it assumes that the black is background and the lights are "moving objects". Anyway, any suggestions from the pros here on how to deshake this scene? I've attached a frame with the "fixed" lights circled - those are the features I need to "lock on" to (i.e. they shouldn't move). Any suggestions would be appreciated! Thanks! Attached Images

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

6th August 2006, 14:03	#41 \| Link
WarpEnterprises C64 Join Date: Apr 2002 Location: Austria Posts: 830	@guth: the percentage of discarded blocks is a quite important number, yet it can't be seen anywhere. Maybe it could be displayed in the pass1-output along with the other values?

6th August 2006, 16:57	#42 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	This is a little complicated. There are three types of blocks: A. Ok blocks. Have white motion vectors. B. Blocks discarded due to bad motion vectors. Have red vectors. C. Blocks discarded due to bad match. Have no motion vectors. Which percentage do you want to see? The deep analysis kick-in percentage is compared to the ratio B / (A + B) and the frame discard limit is compared to the ratio A / (A + B + C). I think it makes sense to do that but it's somewhat confusing I guess, especially since it's not documented.

8th August 2006, 18:29	#44 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	Hmm... I'm not sure I want to print both. We'll see. The "skip frame" setting is basically just there because if very few motion vectors were good, Deshaker probably got everything wrong so it's probably best to skip the frame completely. There are many ways to deal with big moving objects depending on how big they are, how fast they move and for how long they're in the picture. Skipping the frame completely should only be done as a last resort, or if you're lazy Quarter scale indeed means that the matching occurs on images that are reduced to 1/4 of the original width and height, but the accuracy is usually better than 4 pixels anyway for two reasons: 1. I sort of use sub pixel accuracy when calculating the best shift for a block. (I do this by fitting a 2nd degree function through the best shift and its two integer neighbors, and their "match probability values", and take the shift where this function has its max.) 2. Any impreciseness in the block vectors will be evened out when I combine them all for the total motion of the frame. That said, I don't really recommend using quarter scale anyway. Personally I always use full scale.

12th August 2006, 13:39	#45 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	New version of Deshaker available for anyone who cares. Get it here.

29th January 2007, 18:15	#47 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	I'm afraid I don't really understand what you mean. That scripts runs pass 1. Could you give me some more details? (You can email me if you want.)

30th January 2007, 20:28	#49 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	Weird, but good About unique log files, I don't know. I'm no script guru at all. But I wouldn't really recommend running deshaker in batch mode at all. The results will rarely be perfect unless you do some supervision.

31st March 2007, 08:00	#52 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	If you'd like to send me a short problematic sample clip, I'll have a look at it.

9th April 2007, 10:56	#55 \| Link
Terka Registered User Join Date: Jan 2005 Location: cz Posts: 704	when a motion is fast (+long shutter speed) -the deshaked picture is blurred. is it able to add some motion deblur to this pictures? (like focus magic)

11th April 2007, 15:35	#56 \| Link
wonkey_monkey Formerly davidh***** Join Date: Jan 2004 Posts: 2,652	Guth, are you still working on deshaker? In addition to my previous question, is there any chance that a future version might include perspective correction? I ask because some stabilised videos suffer from a slight warping if the input is shakey enough (as my time lapse videos can be, and I've seen the same effect on normal video). If deshaker asked for the FOV of the images, it could correct in a more realistic way, rotating the input images in three dimensions rather than just shifting horizontally and vertically. David

22nd May 2007, 12:55	#57 \| Link
WarpEnterprises C64 Join Date: Apr 2002 Location: Austria Posts: 830	I just saw that there is a new Version 2.2. @guth: I have two questions: (1) You wrote "Improved handling of new scenes ... allows pass 2 to take appropriate action". Would you mind giving some details what this action is? Is there a forced "jump" in the resulting deshaking shifts or the like? (2) regarding the new "Remember discarded areas" option: how is this related to the "blocks OK" percentage?

28th May 2007, 12:42	#59 \| Link
Terka Registered User Join Date: Jan 2005 Location: cz Posts: 704	yes to remove the motion blur - the amount of motion is needed to know. and deshaker has this information-direction and amount. thats why im asking about removing motion blur.

28th May 2007, 17:52	#60 \| Link
guth Registered User Join Date: Apr 2003 Location: Uppsala, Sweden Posts: 157	It's not quite that simple, I'm afraid. The motion blur occurs within each frame not between them. Matching between two motion-blurry frames will be extremely unreliable so you wouldn't be able to get any good information about direction, and specially not about the amount of motion blur, since it can be much different from the amount between the frames (the shutter speed doesn't always equal the "frame speed").