Repair bad PAL video conversion

[lbrider]

Hi, I've been in a bit of trouble lately. I recently purchased the Italian Sailor Moon DVDs, since they are far better quality than the Japanese, though there is a problem with them. There is slow motion during pan shots. I guess its a result of the Italians converting the anime from NTSC to PAL. Is there anyway to fix this problem using avisynth filters? If anyone could help me out I would greatly appreciate it. I cut some clips from the original .vob files using MPEG2Cut-1.15

clip 1 - http://www.sendspace.com/file/9tmcjw

clip 2 - http://www.sendspace.com/file/t0ux8k

[johnmeyer]

It looks like a frame has been dropped, which creates a jump forward in the motion, followed by a duplicate frame to bring everything back in sync. There are several threads on this subject. Here is one I started a couple of years ago:

Automatically fix dups followed (eventually) by drops

I didn't explain the problem very well, but the solutions offered by various people, including those provided in the links listed in my first post, can get the job done.

[johnmeyer]

I had a few seconds, so I found the old script from two years ago, looked at the metrics and set the threshold to 0.8, and then generated the following result:

Fixed Video

I left the "showdot" parameter turned on so you can see where the script synthesized new frames at the point where a frame was probably dropped. It is not perfect, but it is sure a lot better than what you started with. Obviously you will set the showdot parameter to false when you run the script yourself. Here is the script. It is only very slightly modified (for 24 fps instead of 30 fps) from what was posted two years ago:

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
#
# Create interpolated frames at 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video
# However, at each point where motion-estimated frame was inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8
showdot = true # true for troubleshooting; otherwise, false

global source=AVISource( "E:\fs.avi" ).converttoyv12()

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2) 
bvec  = MAnalyse(super, overlap=4, isb = true, search=4, dct=5) 
fvec  = MAnalyse(super, overlap=4, isb = false, search=4, dct=5) 
double = source.MFlowFps(super, bvec, fvec, num=48, den=1, blend=false) #Change num to double the framerate of the progressive source

#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics in order to determine proper JumpThresh
#test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)

interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate
RequestLinear

#Increase cycleR (and make sure to make cycle=2*cycleR) if residual duplicated frames still exist
final=tdecimate(display=false,mode=1,cycleR=12,cycle=24)  # Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)

return final
#return test


#----------------
#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}



#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}

[lbrider]

I tried your script, at default settings, to test it myself and I got some weird shaking...

http://www.sendspace.com/file/w99mew

[johnmeyer]

I think you got the field order reversed somewhere. Use AssumeBFF or AssumeTFF to fix.

[lbrider]

This is the original clip: http://www.sendspace.com/file/yoummp

And I get that weird shaking after applying your script. I tried to use assumebff and then assumetff and it didn't help at all.

[johnmeyer]

Originally Posted by lbrider

This is the original clip: http://www.sendspace.com/file/yoummp

And I get that weird shaking after applying your script. I tried to use assumebff and then assumetff and it didn't help at all.

I used the clip in your original post. I frame served it out of Sony Vegas into the script, opened that script in VirtualDub, and rendered out. I posted the exact script I used. I don't know what else to tell you. Maybe someone else has some idea.

[StainlessS]

Perhaps try AssumeTFF/AssumeBFF before JohnMeyer script rather than after ?
(Well after loading source clip)

[johnmeyer]

Originally Posted by StainlessS

Perhaps try AssumeTFF/AssumeBFF before JohnMeyer script rather than after ?
(Well after loading source clip)

For sure. If you do it after the script, it will do absolutely nothing.

[lbrider]

Originally Posted by johnmeyer

I used the clip in your original post. I frame served it out of Sony Vegas into the script, opened that script in VirtualDub, and rendered out. I posted the exact script I used. I don't know what else to tell you. Maybe someone else has some idea.

Ah, I found the problem. Some setting in VirtualDub was changing the framerate to 25fps when the original framerate is 24fps. Now that problem has been fixed but I noticed this:




@Johnmeyer: It's also in the fixed clip you you rendered for me.

[johnmeyer]

Originally Posted by lbrider

Ah, I found the problem. Some setting in VirtualDub was changing the framerate to 25fps when the original framerate is 24fps. Now that problem has been fixed but I noticed this: ... @Johnmeyer: It's also in the fixed clip you you rendered for me.

Yes, this is what sometimes happens with motion estimation algorithms: they fail when pixels in the video move in "unpredictable" directions from frame to frame. However, when motion is coherent (like the waves crashing to the shore), they do a pretty good job.

The two MAnalyze lines in the script are where you can tweak to try to get better results. Be aware that you can sometimes get one frame to look better, only to find a problem pop up on another frame. The parameters which are likely to make a difference are "blksize" (which I did not explicitly use in this script). You often get fewer artifacts when creating intermediate frames if you use a blksize of 16 instead of 8. You can try using "overlap" of 0 or 2. The various "search" modes may give you better results. Finally, in looking at the script, I have no idea why I used dct=5, and if I had to do it over again, I set it to the default of 0, which would make the script faster. Using dct makes everything slower.

Others can perhaps offer additional hints on how to avoid artifacts like the one you posted. However, at the risk of repeating myself, motion estimation is not perfect, and this includes the programs for which you pay hundreds of dollars, like Twixtor.

[johnmeyer]

Suggested starting point for your experiments:

bvec  = MAnalyse(super,blksize=16, overlap=2, isb = true,  search=3, searchparam=3,dct=0) 
fvec  = MAnalyse(super,blksize=16, overlap=2, isb = false,  search=3, searchparam=3,dct=0)

In addition, you may need to do more work on the anime portions of the video prior to feeding it to this script. I have no experience whatsoever in restoring anime, so I don't know what to recommend. However, as I think you already know, the problem to be solved is that anime usually repeats frames in a very irregular pattern, and this can make it very difficult to apply AVISynth scripts that expect some sort of regular pattern in the motion from frame-to-frame.