GenMotion and DePan plugins for global motion compensation

[Fizick]

I release version 1.0 of DePan.
Since version 1.0, DePan can also use more fast FFTW library version 3 (http://www.fftw.org)
as Windows binary DLL (compiled with gcc under MinGW by Alessio Massaro),
which support for threads and have AMD K7 (3dNow!) support in addition to SSE/SSE2.
It may be downloaded from ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip
For fftw using, you must put FFTW3.DLL file from this package to some directory in path (for example, C:\WINNT).

Please post test (speed) report.

[morsa]

WOW!!
A lot faster now on this Athlon XP machine I'm using to test it!!!
As always Fizick, THANK YOU!!

(Taste the Russian advanced knowledge )

[Fizick]

I get some e-mail from Andreas J. and decide post it here for discussion:

I trying to use DePanEstimate for scene detection & Extraction of JPEG-FIles.
As far as I have tested, It will do what I want.
I would like to do detect the scenes an extract JPEG thumbnails in one step.
My problem is, that I havent found a way to formulate the if-clause because I Havent Found a way to get the "Trust" Value or the dx-value for each frame
As I Understand your Plugin, it is currently not possible to get these values during runtime.
Is it possible to extend DePanEstimate to do this?

My Ideas for this are:
1) getting the "trust" - value as a variable during the processing of the each frames
in AviSynth a condition like
DePanEstimate.Trust < 3.5 ? ExtractKeyframe(currentFrame):nop()
(Okay, I need todo it on every frame with EvaluateFrame() or ScriptClip() but my Scripting Experiance
ist only only 10 Hours with Avisynth ;-):
2) by adding a function "DePanIsKeyFrame" with nearly the same parameters as DePanEstimate
(as far as I understand DePanEstimate SceneChange will come if myTrust-value > Trust(CurrentFrame)
3) another possibility could ba a function (DePanExtractKeyFrames (trust = 3.5, log = "c:\myfilename, KeyFramesEvery=1500, SkipMultiKeyframes=25…)
The log will consist of lines where myTrust <= Trust (Frame) where
"KeyframeEvery" will force-add keyframes(in this case every minute 25fps*60= 1500 and
"SkipMultiKeyframes" will not allow new keyframes for x frames if another (not forced) keyframe was detected before
(it should prevent to get 25 keyframes a second…. )
The log should also contain the trust-value and a info if KeyframeEvery-conditions was positive
4) my last idea is: a extlog="c:\myFilename.log" the result is a the same as the "log=" -LogFile
plus 2 additdional values: the trust value and "Scenechange"-Info.
So it would be goot to parse the logfile with PHP.
But a second step is needed to extract the thumbnails.

A Perfect solution would be a combination out of 1) & 3) & 4)

Sorry for so many ideas, but your tool has so much power….
Do you think you will find / a way to make scene-detection & frame extraction easier?



Here is my current script:
MayBe it ist much easier as I thought…..

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEGDecoder.dll") 
global ThumbNailPath = "D:\_Szenenerkennung\videos\Bilder\" 
global LogFilename   = "D:\_Szenenerkennung\MyLog.txt" 
myMPEGSource = MPEGSource("D:\_Szenenerkennung\videos\25.mpg") 
myVideo = ReduceBy2(myMPEGSource) 
myVideo = ConvertToYV12(myVideo) 


function SaveFrame (myClip, myFrameNumber) 
{ 
###Uncomment to display frame number 
 myClip = Subtitle( myClip, string(myFrameNumber)) 

### Export JPEG Image 
  ImageWriter(myClip, file = ThumbNailPath, start = myFrameNumber, end = myFrameNumber, type = "jpeg",info=true) 
### WriteLogFile 
  WriteFile(LogFilename,  """ "CurrentFrame: " """ ,"current_frame") 
return (last) 
} 

#myVideo = ScriptClip(myVideo,"depanestimate(myVideo,trust=4,log=""" "scenes.log" """,info=true,show=true)") 
myVideo = depanestimate(myVideo,trust=4,log= "scenes.log",info=true,show=true) 



###Export KeyFrame 
### a Key Frame yould only be exported when myTrust <= (DePanEstimate.Trust(frame)) 
myRGBVideo = ConvertToRGB24(myVideo) 
  ScriptClip(myRGBVideo, "SaveFrame (myRGBVideo,current_frame)")

It would make me very happy to get an Idea how this problem could be solved..

[Fizick]

My first answer to Andreas is:

I thinked about implementing scenedetection function some time ago.
For example, I can easy to realize a function DePanScenes(clip, inputlog),
which uses for input either DePanestimate motion data clip, or log file.
DePanScenes output will be clip with all pixels luma (or croma) = 0 for all ordinary frames, and = 255 for scenechanges.
This clip can be analyzed by AverageLuma() for example, and results used in ScriptClip.

But DePanEstimate make some calculation at constructor stage (sinus table for fft etc), and frame processing will be slow.

[Manao]

Thinking of this, I could also quickly make a filter from the ME engine in MVTools, which decides whether a frame is a scenechange or not. I don't know however if the scenechange I get from vectors would be more reliable than those you're getting with your filter.

Anyway, if you want to test the basic scenechange detection in mvtools, you can use the following trick :

mv = source.mvanalyse(isb = false)

mask= source.mvmask(mv, ml = -1, gamma = 100)

white = source.blankclip(color = $ffffff)

black = source.blankclip(color = $000000)

scmask = conditionalfilter(mask, white, black, "AverageLuma()", ">", "0.2")

return scmask

( It's a hack, if there is no motion at all, it will be considered as a scenechange, but anyway, it gives a result )

[Fizick]

New version 1.1, September 16, 2004 - Added experimental DepanScenes function.

[Ark]

I found this script on Depan help file regarding FPS conversion from 16,6 fps to 25 fps:

Avisource("kino.avi")
loadplugin("depan.dll")
i=ConvertToYV12()

data=DePanEstimate(i, range=1, trust=5)
f1_3=DePan(i,data,offset=1./3)
b1_3=DePan(i,data,offset=-1./3)
interleave(f1_3,i,b1_3)
selectevery(6,0,1,2)

But I've some movies at 15 FPS that I want to convert to 25 FPS, how do I have to change values to achieve that?

(the above script result in 22.5 FPS...)

[johnmeyer]

When trying to load Depan 1.1 or Depan 1.0, AVISynth 2.5 returns the error message:

Unable to load "depan.dll"

If I copy the version 0.91 depan.dll to the same directory, the dll loads and the script runs just fine. I have no problems loading other dlls, so I assume there is some bug in these two latest versions.

I am running XP Pro (SP1). I am running AVISynth 2.54.

Edit: I upgraded to the latest (AVISynth 2.55) and still couldn't load the two latest versions. I was hoping to use this plugin to generate slow motion from standard NTSC DV video. Can this be done using the DePanEstimate function?

[Fizick]

Ark,
Here is a possible function for framerate converting (progressive) with factor=5/3, for example from 15 fps to 25 fps :

function fps3to5(clipYV12)
{
# change FPS from 3 to 5 (or 15 to 25, or 18 to 30 and so on), i.e. with factor=5/3
# uses global motion compensation
# input must be YV12 progressive (or separated fields probably ?)
data = DePanEstimate(clipYV12)
t3_5 = DePan(clipYV12, data, offset=-2./5)
t6_5 = DePan(clipYV12, data, offset=1./5).trim(2,0)
t9_5 = DePan(clipYV12, data, offset=-1./5).trim(1,0)
t12_5 = DePan(clipYV12, data, offset=2./5).trim(3,0)
Interleave(clipYV12, t3_5, t6_5, t9_5, t12_5)
SelectEvery(15,0,1,2,3,4)
}

AviSource("e:\video.avi")
LoadPlugin("depan.dll")
ConvertToYV12()
fps3to5()

[Fizick]

johnmeyer,
I have similar reports from some users.
Probably you must have FFTW3.DLL
I will look later.
What is your CPU ?

[johnmeyer]

Probably you must have FFTW3.DLL

I'll look into it tomorrow. I have a Pentium P4 2.8 GHz.

[Fizick]

Version 1.1.1, October 07, 2004 - bug fixed with compensation near right bottom and top corners2,
change from FFTW_MEASURE to FFTW_ESTIMATE for more short init, without speed change (for power-2 windows),
compiled without /G7 flag (as before v.1.0), FPS script functions added to doc.

(not public)

Version 1.1.2, October 09, 2004 - delayed loading of fftw3.dll (now optional). Compiled with VC6.

[Ark]

Originally posted by Fizick
Ark,
Here is a possible function for framerate converting (progressive) with factor=5/3, for example from 15 fps to 25 fps :

function fps3to5(clipYV12)
{
# change FPS from 3 to 5 (or 15 to 25, or 18 to 30 and so on), i.e. with factor=5/3
# uses global motion compensation
# input must be YV12 progressive (or separated fields probably ?)
data = DePanEstimate(clipYV12)
t3_5 = DePan(clipYV12, data, offset=-2./5)
t6_5 = DePan(clipYV12, data, offset=1./5).trim(2,0)
t9_5 = DePan(clipYV12, data, offset=-1./5).trim(1,0)
t12_5 = DePan(clipYV12, data, offset=2./5).trim(3,0)
Interleave(clipYV12, t3_5, t6_5, t9_5, t12_5)
SelectEvery(15,0,1,2,3,4)
}

AviSource("e:\video.avi")
LoadPlugin("depan.dll")
ConvertToYV12()
fps3to5()

Thanks a lot Fizick, I'll try thah script tomorrow!

[johnmeyer]

fftw3.dll fixes the problem. Thanks!

[scharfis_brain]

Bug in depanstabilize found:
it produces dark-green frames (I think Y=U=V=0 )sometimes.
also it makes one or two lines at the bottom looking dark green.

the error is reproduceable, but I didn't tried it with another script for now.

[Fizick]

[scharfis_brain]

you won't like it:

setmemorymax(512)
loadplugin("d:\x\masktools.dll")
loadplugin("d:\x\depan.dll")
loadplugin("d:\x\removedirt.dll")

function repairfieldswap(clip c)
{d=c.swapfields().crop(0,0,0,-1)
d=stackvertical(d.crop(0,0,0,1),d)
e=c.converttoyv12()

global edges=e.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
global swapedges=e.swapfields().yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)

ConditionalFilter(e, c, d, "float(averageluma(edges)-averageluma(swapedges))", "lessthan", "0.0")
}
a=avisource("capture (2004-11-14 at 14-34-24).avi")
b=avisource("capture-55630.avi").converttoyuy2().levels(16,1,235,0,255)
c=avisource("capture-81443.avi")
d=avisource("unblended.avi").changefps(25).killaudio().addborders(2,0,6,8)

a1=a.trim(0,55630)
a2=a.trim(55638,81442)
a3=a.trim(81447,0)
a1+b+a2+c+a3
trim(580,100296)
assumetff()
doubleweave().selectodd()
audiox=last
last.killaudio()
last.trim(0,2205)+d+last.trim(3074,0)
audiodub(last,audiox)
r=2
c=2
mergechroma(crop(0,c,0,0).addborders(0,0,0,c))

repairfieldswap()


crop(2,0,-6,-8)

converttoyv12()
x=depanestimate(last,pixaspect=1.094,trust=0.0,range=1)
depaninterleave(last,data=x,next=r,prev=r,mirror=15)
removedirt()
selectevery(r+r+1,r)

de0=last
de1=de0.depanstabilize(data=x,dxmax=12,dymax=12,zoommax=1.0,rotmax=0.0,pixaspect=1.094,mirror=15) #next=12,prev=12)
de1.trim(0,190)+de0.trim(191,540)+de1.trim(541,0)

[Fizick]

Confirmed
The bug was becouse of infinite calculated shifts for improperly Depanestimate() parameters setting.
Bug fixed in new version 1.1.3. Now base frame reseted.
Thanks.
Try!

[scharfis_brain]

Wow, you're damn fast in fixing bugs!

but can you explain this:
"The bug was becouse of infinite calculated shifts for improperly Depanestimate() parameters setting."

a little bit more detailed?

what are infinte shifts?
why improper settings?

[Fizick]

1. improper settings - some scenechange was not found,
and resulted in very big false pan by Depanestimate.

2. Depanstabilize try calculate shift for these false data,
and internal get divergence of caclulation (infinite shift).