Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th October 2018, 19:42   #1  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
Completely remove camera motion

Hi

I remember doing this before, but I've lost my scripts and can't seem to figure out what functions to use.

So I have a video recorded with a phone camera, that is supposed to be completely still, but of course there's some small movement. I'd like to make it completely still, not just remove shakiness but simulate a tripod.

I remember that I first picked a reference frame, interleaved that with the whole clip, generated motion data so that every frame's motion is compared to this reference frame. But what to do next, I don't remember. What plugin/function will fully move/zoom/rotate a frame back to where its motion vectors are pointing from?
ajp_anton is offline   Reply With Quote
Old 12th October 2018, 20:52   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Would it be some variation of this from mvtools examples (?),

Code:
To use MDepan with Depan plugin for interlaced source (DepanStabilize function example):

AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc
AssumeTFF().SeparateFields() # set correct fields order
vectors = MSuper().MAnalyse(isb = false)
globalmotion = MDepan(vectors, pixaspect=1.094, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094)
Weave()
but (EDIT: without interlaced stuff &) interleaving reference and then removing afterwards.

(I aint set up for using Avisynth at the moment)

EDIT: Maybe something like
Code:
AVISource("c:\test.avi")
Interleave(Last,RefClip) 
vectors = MSuper().MAnalyse(isb = false)
globalmotion = MDepan(vectors, pixaspect=1.094, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094)
SelectEven
If I got the vector direction wrong then try toggle to isb=true.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 12th October 2018 at 22:30.
StainlessS is offline   Reply With Quote
Old 12th October 2018, 23:16   #3  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Deshaker can do this. I wrote a long tutorial fourteen years ago that, unfortunately, seems to have disappeared from the web. There is a variable that you set to -1 and it will then lock the camera so that it appears to be on a tripod.

You can also do this with Mercalli, but it costs money.
johnmeyer is offline   Reply With Quote
Old 13th October 2018, 00:22   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
I wrote a long tutorial fourteen years ago that, unfortunately, seems to have disappeared from the web.
For those that speak french,
Topic, Discussing Johnmeyer/Gunnar Thalin Guides, in French:- http://letransfert.soforums.com/t215...-de-la-DOC.htm

Topic, Discussing Johnmeyer/Gunnar Thalin Guides, translated from French, into something sensible
:- https://translate.google.com/transla...-de-la-DOC.htm


John Meyer Pdf in French:- http://letransfertpellicule.free.fr/...ex.guideFr.pdf


EDIT: And an actual translation for those that dont speak french (XLate from the PDF):- https://translate.google.com/transla...ex.guideFr.pdf

EDIT: Keep a link to that John, the "missing guide" topic has come up before.

EDIT: Maybe this (near end of pdf)
Quote:
Other uses

If you do not own and can not afford Steadicam equipment, you can use Deshaker
to create a version of "the poor" of Steadicam. The results can often be very good. For example, if
you plan ahead and work a little bit, you can use this technology to "approach" a socket
Dolly. You can walk with the camera using its own stabilizer mechanism, plus a system
cheap stabilizer, use Deshaker software, and end up with something that is VERY
stable. In addition, if your camera technique in hand is good, and if you can maintain stability
wanted, then this software can often give a result at 99% as good as a video taken on a tripod.
This is especially true if you have optical stabilization enabled on your camera. This characteristic
usually eliminates shaking, but the video will still be slightly unstable. Deshaker can often
remove residual drift and wandering.

One possible use, suggested by a user, is to adjust the Horizontal, Vertical and Rotation parameters
from "Motion Smoothness" to -1.
As described in the Deshaker documentation, this makes Deshaker
never moves the camera from its original position, even if it means violently moving the video and destroying
its edges. I used the -1 settings, without finding them useful. However, what I forgot is that the file
Log created by this process is essentially a set of instructions for how to create images-
keys of a video, to exactly match the movement of the clip. So, if you could use "Ignore
Image Area "and also the -1
settings to force Deshaker to follow the movement of a portion of the video,
you could load a different video in Deshaker, and run a "Pass 2" using the Log file
from the first video clip (with "Motion Smoothness" settings to -1) to match that clip with the
movement of the object in the first clip. You would then have to take this out as an AVI file, put it online
in Vegas, then compose it with the first image, thus obtaining a composite track that you would adjust
the movement in the girl track. Again, it's theory, but it's still a theory
fascinating. And again, the main idea is to use the motion vectors stored in a clip for
induce an identical displacement in a second clip, so that both follow each other.
Google Translate is getting a whole helluva lot better.
EDIT: "movement in the girl track", John, what should that have said, should we perhaps read something like
"mate" (or "matched") instead of "girl" ?

EDIT: From Tormento below,
Quote:
Probably it was meant as "subsequent" or "derived" track.
Thanx, that explains it nicely.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 14th October 2018 at 16:54.
StainlessS is offline   Reply With Quote
Old 13th October 2018, 12:28   #5  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by StainlessS View Post
what should that have said, should we perhaps read something like"mate" (or "matched") instead of "girl" ?
"obtenant ainsi une piste composite dont vous ajusteriezle mouvement dans la piste fille"

From my little french knowledge, "fille" should be translated as "daughter", literally. Probably it was meant as "subsequent" or "derived" track.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 13th October 2018, 15:33   #6  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
@StainlessS
DepanStabilize can't do it. It will find some middle ground between the reference and the clip, pulling both closer to each other. And it's not always just in the middle either, it can sometimes actually pull both in the "wrong" direction, as long as the overall resulting movement is smooth (which is what it's designed to do I guess).

@johnmeyer
That seems to be a VirtualDub filter, which I haven't really got working before, so my old solution had to use something else. But if nothing else comes up, I'll have to figure out how to use it.
ajp_anton is offline   Reply With Quote
Old 13th October 2018, 15:42   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
What type of shot is it ? What are the subjects / any foreground movement ? Can you describe it or post a sample ?

A simulated locked off tripod shot should be stabilized based on known stationary objects like a wall, the ground, etc... because if you have foreground subjects, people etc... moving - it will contaminate the motion data and you typically won't get good results with something like mvtools2, deshaker, mercalli etc....For example, you don't want to include the motion vectors from someone nodding their head, or a car driving in the background etc....because that would skew everything and you actually introduce more compensatory camera movement, making it worse than before . If you can crop to a "clean" area, then do the calculations and apply back to original that might work. Or deshaker has the exclude border option. But those only help on avoiding peripheral objects

Usually for those types of situations , different types of motion trackers are used (some are better for certain types of shots) . They are typically found in packages such as after effects, mocha, hitfilm, nuke, shake . Free ones like natron, blender, blackmagic fusion can do this type of thing too. The difference is you can define areas to include/exclude used in motion tracking so it's a lot more accurate, but the downside is it takes some user guidance to define those areas so it's slower, less automated
poisondeathray is offline   Reply With Quote
Old 13th October 2018, 18:08   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
StainlessS, thanks for finding that quote from the old guide which does indeed identify the setting that should be set to -1. I couldn't remember that off the top of my head.

Perhaps I should re-post my tutorial on my own web page, for those who don't speak those other languages.

Then again, maybe it isn't worth it. While Mercalli costs a fair amount of money, and Deshaker is free, the algorithms in Mercalli are far better, especially when trying to achieve a "tripod look." Also, Mercalli also contains a really good algorithm for removing the sometimes grotesque "rolling shutter" artifacts that are so common in cell phone video. If you stabilize using a program that doesn't have this feature, the result is pretty awful.

Last edited by johnmeyer; 13th October 2018 at 18:11. Reason: added last paragraph
johnmeyer is offline   Reply With Quote
Old 14th October 2018, 11:37   #9  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
BTW, depending on camera motion asking for full stabilisation might yield only a tiny part of the picture, if any...
So if camera moved completely out of the original frame, for these source frames one would get an empty frame in return.
(Of course and out of curiosity I tried total stiff stabilisation on Mercalli.
Seeing the resulting frame becoming tinier the more I restricted motion made it clear:
I had ordered content that hadn't been shot, so got presented with the almost empty result.
Went back to the possible, allowing some smoothened motion yielded more pleasing results.)
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."
Emulgator is offline   Reply With Quote
Old 14th October 2018, 11:47   #10  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
@poisondeathray
There is a table and some walls, it's an indoor shot. Two dogs in the foreground, but they are contained in a small area on the image during the whole shot. I know their movements will affect the motion vectors, so I'll probably need some extra processing to get around that, but first I need to get the stabilization to actually work with the motion vectors I have.

@Emulgator
Of course it would restrict the picture. Or well, you could in theory have the full frame, but the video image would move around and not always show itself completely and maybe go outside. But I'd hope the person doing this knows this =). My shot has 90% of the scene always visible, and it's going to get cropped even further anyway.
ajp_anton is offline   Reply With Quote
Old 16th October 2018, 06:13   #11  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by ajp_anton View Post
DepanStabilize can't do it. It will find some middle ground between the reference and the clip, pulling both closer to each other. And it's not always just in the middle either, it can sometimes actually pull both in the "wrong" direction, as long as the overall resulting movement is smooth (which is what it's designed to do I guess).
I conducted some synthetic tests using a still image and translate, rotate, zoom using known values . (Depanstabilize info=true will overlay the x,y, rotation, zoom info, and you can compare with the known values.)

Depanstabilize uses inertia method when set to 0 , but the 1st frame (frame 1) looks like it's solved as good as can be expected by mdepan vectors. After that, it won't work well because of averaging based on frame 0, you can see the values with info=true. So you if you interleaved it so each frame has a preceding reference frame , and chopped it into pairs, and only did the calcs on the pairs, then ditch the 1st frame in the pair, then reassembled it , it would probably give you as good as can be expected with that method. You can change the reference frame (e.g. a tilted one), an it still works - frame 1 becomes similarly tilted and transformed. Maybe a wrapper function could automate it, but it does work as expected

But it's still a bit wobbly, not what I would call "rock stable"; and that's with "ideal" test material (you'd expect a lot worse with real material, especially rolling shutter cell phones, motion blur, compression issues). Depanstabilize info=true will overlay the x,y, rotation info, and you can compare with the known values. Even a slight rotation difference from the actual values like a delta of 0.01 will register as a wobble . You can play with the analyze parameters like blksize etc..., but rotation seems weak with this method, at least with the values I tried.

I do not think method=1 will usable in any combination (even with various combinations interleaving+selecting, of because it averages the frames and the base frame increments (watch the behaviour with info=true)

(Also I noticed slight green hue shift with depanstabilize , it's reduced if you use ConvertBits(16) beforehand , but still there)
poisondeathray is offline   Reply With Quote
Old 16th October 2018, 22:45   #12  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by poisondeathray View Post
I conducted some synthetic tests using a still image and translate, rotate, zoom using known values . (Depanstabilize info=true will overlay the x,y, rotation, zoom info, and you can compare with the known values.)
That sounds something AvisynthOptimizer could do as well, since we can simply compare the stabilized video to the original still image. If you can share your still image, translated video and avs script I could give it a try.
zorr is offline   Reply With Quote
Old 17th October 2018, 00:53   #13  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by zorr View Post
That sounds something AvisynthOptimizer could do as well, since we can simply compare the stabilized video to the original still image. If you can share your still image, translated video and avs script I could give it a try.

Sure, but it's just a simple 5 frame synthetic test - an exercise for me to figure out the behaviour for depanstabilize.

But you wouldn't have the original, original reference image used to create the video in real life ; you'd only have the "shaky" video and you'd pick a reference frame to stabilize against. It was actually a larger image scaled down to 1920x1080 and then the transformations were applied. The reason it's done this way, is so the "BG canvas" is larger than the target field of view of 1280x720, so there are no black borders when producing the "shaky" video . Since we are trying to emulate a real case, you should use frame 0 as the reference. (If you want I can upload the other images, but I don't think they should be used; or it's easy enough to make your own "shaky" videos using known values)

The video uses lagarith ~5MB
https://www.mediafire.com/file/upiwb..._LAGS.avi/file

Actual transform values used:

Code:
Frame number, x,y,rotation,zoom%

0
1 +5,+5,+1,100%
2 -5,-5,-1,100%
3 +5,+5,+1,101%
4 +2,-2,-0.5,99%
my ugly depanstabilize testing script
Code:
orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16)
a=orig.ConvertToYV12(matrix="rec709").Trim(0,-1)
b=orig.ConvertToYV12(matrix="rec709").Trim(1,-1)
c=orig.ConvertToYV12(matrix="rec709").Trim(2,-1)
d=orig.ConvertToYV12(matrix="rec709").Trim(3,-1)
e=orig.ConvertToYV12(matrix="rec709").Trim(4,-1)

a+b
vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0)
globalmotion = MDepan(vectors, range=1, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true)
trim(1,-1)
a1=last

a+c
vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0)
globalmotion = MDepan(vectors, range=1, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true)
trim(1,-1)
a2=last

a+d
vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0)
globalmotion = MDepan(vectors, range=1, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true)
trim(1,-1)
a3=last

a+e
vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0)
globalmotion = MDepan(vectors, range=1, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true)
trim(1,-1)
a4=last


a+a1+a2+a3+a4
#histogram("levels")
If you toggle the info=true, values are a bit off, but the killer is the rotation wobble . If the vectors or global motion using mdepan settings (or depanestimate or something else) could be tweaked so there is less rotation error maybe using avisynth optimizer, then it might be usable in that test case

FYI, a handful other stabilizers can solve it pretty much perfectly and automatically - but only because it's a simple test.

But in real life, there are the other problems mentioned above, also things like moving objects (like dogs, people) . There are different types of stabilizers and trackers used in different situations. There are "ideal" tools for certain situations, and it depends on the specific shot characteristics
poisondeathray is offline   Reply With Quote
Old 17th October 2018, 13:23   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Dont know if of any use (as for PDR script above, untested, I'm not set up for Avisynth use)

Code:
Function Func(clip c,
    \ Int "Search",int "blkSize",Bool "isb",int "dct",
    \ Int "Range",Int "thSCD1",
    \ int "cutOff",int "dxMax",Int "dyMax",Int "method",Float "zoomMax",Bool "info"
    \ ) {
    n=current_Frame
    if(n != 0 ) { # Req Avs+
        ###
        Search=Default(Search,4)        blksize=Default(Blksize,8)      isb=Default(isb,false)
        dct=Default(dct,0)              range=Default(Range,1)          thSCD1=Default(thSCD1,400)
        cutoff=Default(cutoff,1)        dxmax=Default(dxmax,160)        dymax=Default(dymax,90)
        method=Default(method,0)        zoomMax=Default(zoomMax,1.1)    info=Default(info,true)
        ###
        Last = Trim(0,-1) + Trim(n,-1)
        vectors = Last.MSuper.MAnalyse(search=search, blksize=blksize, isb=ISB, dct=DCT)
        globalmotion = Last.MDepan(vectors, range=range, thSCD1=thSCD1)
        Last.DepanStabilize(data=globalmotion, cutoff=cutoff, dxmax=dxmax,dymax=dymax, method=method, zoommax=zoommax, info=info)
        Last = Trim(1,-1)
    }
    return last
}

ARGS="Search,blkSize,isb,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info"
SSS="Func(Last,"+ARGS+")"

orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16)
conv=orig.ConvertToYV12(matrix="rec709")
Last=conv

### Set non default args
# ...
# EDIT: For defaults, set eg non defined Search, "Search=Undefined"
###

Last.ScriptClip(SSS,args=ARGS,Local=True) # Req: Grunt for Local & Args
EDIT: Or probably faster (and more suitable to Zorr, Avisynth Optimizer)
Code:
SSS="""
    n=current_Frame
    if(n != 0 ) {  # Req Avs+
        Last = Trim(0,-1) + Trim(n,-1)
        vectors = Last.MSuper.MAnalyse(search=search, blksize=blksize, isb=False, dct=DCT)
        globalmotion = Last.MDepan(vectors, range=range, thSCD1=thSCD1)
        Last.DepanStabilize(data=globalmotion, cutoff=cutoff, dxmax=dxmax,dymax=dymax, method=method, zoommax=zoommax, info=info)
        Last = Trim(1,-1)
    }
    return last
"""

#ARGS="Search,blkSize,isb,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info"
ARGS="Search,blkSize,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info"

orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16)
conv=orig.ConvertToYV12(matrix="rec709")
Last=conv

### Set all args (where none allowed to default)
Search = 4
blksize= 8
#isb    = false
dct    = 0
range  = 1
thSCD1 = 400
cutoff = 1
dxmax  = 160
dymax  = 90
method = 0
zoomMax= 1.1
info   = true
###

Last.ScriptClip(SSS,args=ARGS,Local=True) # Req: Grunt for Local & Args
EDIT: isb should probably be removed as option, and always set to false. [algo will not work as is, if true]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th October 2018 at 14:16.
StainlessS is offline   Reply With Quote
Old 3rd December 2020, 15:49   #15  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Since this thread is linked to from other threads, here is a link to my Deshaker guide, as originally written (in English), without multiple translations:

John Meyer Deshaker Guide
johnmeyer is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.