The power of Avisynth: restoring old 8mm films.

[johnmeyer]

First, the slides. The best way to scan them is a Nikon Coolscan scanner. They don't make them anymore, but you can get them on eBay. You can sell it when you are finished in order to recoup the $1,000+ cost.

However, slide scanning is slow. It took me 3+ years to get through my dad's 10,000 slides.

I recently needed to scan 3,000 slides for a client whose dad is dying and wanted to see the slides one last time. So, I converted a Carousel projector into a very high-quality, extremely fast scanner. The results, when using my old Nikon D-70 as the capture device are probably 85-90% as good as the quality of my Nikon scanner, albeit without any automatic dirt or dust removal, although that only works well on Ektachrome and doesn't really do much with Kodachrome. The Nikon Coolscan takes about 1.5-2.0 minutes per scan. My Carousel/Digital Camera lashup can capture about 15-20 slides per minute. So, it is almost 40 times faster.

I can provide a few construction details on how to do this conversion in a way that gives you good results. You cannot simply point the projector at a screen and then snap the result with a digital camera. The result is absolutely awful. Instead, you point the camera directly at the Carousel lens, at point blank range. You have to use the correct lenses in both the projector and camera; dim the bulb; and add an additional diffuser in he projector's light path. It is actually quite easy to do, but figuring out which parts to use took me quite a bit of time and effort.

As for your other question, VideoFred put all the control variables for his script at the beginning. You just simply change them and then look at the result.

I created my own version of his script, which I optimized for speed, but with far less sharpening. My work involves mostly really old film and I find that my version's lower levels of sharpening and grain reduction give results which I find more pleasing when dealing with typical amateur film.

Here are the variables from my script. The comments are intended to tell you what they do. So, just get the script working and then start changing these variables. VideoFred's scripts work the same way.

#GENERAL PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
result="resultS1"                                                #specify the wanted output here 
trim_begin=0  play_speed=15                                     #trim frames and play speed (PAL: 16.6666 or 18.75)

#COLOR AND LEVELS PARAMATERS
#----------------------------------------------------------------------------------------------------------------------------
saturation=1.0                                                  #for all outputs
gamma=1.0                                                       #for all outputs 
blue= -0  red=-0                                                #manual color adjustment, when returning result3 & result4. Values can be positive or negative
black_level=0  white_level=255 output_black=0  output_white=255 #manual levels, when returning result2 & result4
 

#SIZE, CROP AND BORDERS PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
CLeft=16  CTop=16  CRight=16  CBottom=16                         #crop values after Depan and before final resizing 
W=720  H=480                                                     #final size after cropping 
bord_left=0  bord_top=0  bord_right=0  bord_bot=0                #720p= borders 150
in_bord_left=0  in_bord_top=0  in_bord_right=0  in_bord_bot=0   #Borders around input that must be removed


#STABILISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
maxstabH=20                                                         #maximum values for the stabiliser (in pixels) 20 is a good start value 
maxstabV=20
est_left=40  est_top=40  est_right=40  est_bottom=40  est_cont=1.6  #crop and contast values for special Estimate clip


#DENOISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
denoising_strength=600                     #denoising level of first denoiser: MVDegrain() 
block_size= 16                             #block size of MVDegrain
block_size_v= 16
block_over= 8                              #block overlapping of MVDegrainMulti()
dirt_strength=23                           #sets amount of dirt removal (big spots)
#dirt_strength=13                           #sets amount of dirt removal (big spots)

#FOUR STEP SHARPENING PARAMETERS
#--------------------------------------------------------------------------------------------------------------------------------
PRE_sharp_ness= 120   PRE_radi_us= 3            #presharpening (UnsharpMask) just after first denoising
LSF_sharp_ness=160 LSF_radi_us=2  LSF_sub=1.5   #second sharpening parameters (LimitedSharpenFaster) sub=subsampling

#Alternative lower sharpening values
#PRE_sharp_ness= 120   PRE_radi_us= 3                      #presharpening (UnsharpMask) just after first denoising
#LSF_sharp_ness=120 LSF_radi_us=2  LSF_sub=1.5 #second sharpening parameters (LimitedSharpenFaster) sub=subsampling


Sharp_Strength = 0.4


#AUTO LEVELS PARAMETER
#--------------------------------------------------------------------------------------------------------------------------------
X=4   #X is a special parameter for reducing the autolevels effect on the whites
X2=2  #X2 is a special parameter for reducing the autolevels effect on the blacks 


# END VARIABLES, BEGIN SCRIPT

[goodiesguy]

What parts of VideoFred's script do I remove to stop the colour correction and auto gain/brightness?

I prefer to do this manually and my few attempts to stop the script from doing this have broken it.

[johnmeyer]

Originally Posted by goodiesguy

What parts of VideoFred's script do I remove to stop the colour correction and auto gain/brightness?

I prefer to do this manually and my few attempts to stop the script from doing this have broken it.

You simply change the "result" parameter.

[sys32768]

Was curious whether anyone had already done the work to modify Fred's original script here to use GAMMAC?

As an AVIsynth newbie, I find myself tripping up quite often and then starting over so I thought I'd ask.

[johnmeyer]

I created my own branch of Fred's script and posted it many years ago. However, like Fred, I haven't tried to post all my changes and updates because I don't have time to support it when people ask questions.

With that as preface, here is what I am using. It is not really fit for publication (i.e., not many comments on the new stuff), and there are probably bugs all over the place. However, it works for me.

It will take two posts to get it all.

/*
Film restoration script by videoFred.
Denoising, resizing, stabilising, sharpening, auto-levels and auto-white balance.
 
Modified by John Meyer, October 7, 2012
Modified by John Meyer, July 7, 2015 (remove errors)
Modified by John Meyer, September 13, 2017
 
Changes made by Meyer (June 8, 2010):
    1. Replaced the MVDegrainMulti function, which was part of the original MVTools
       with the equivalent function using the newer and faster MVTools2.
    2. Added RemoveDirt function prior to Degrain in order to eliminate large dust spots. This
       substantially improves the dirt removal capabilities of this script. The user will need to 
       download this plugin at:
           http://www.removedirt.de.tf/
    3. Eliminated a great deal of sharpening. The original script did sharpening in at least four places.
       With grainy film stock, this sometimes created objectionable grain. Also, the limitedSharpenFaster
       function, while excellent for video, is uncessarily slow, and isn't really needed for this low 
       resolution source. Too much sharpening can make the film look too much like video.
    4. Added multithreading. I was able to roughly triple the script performance. It may be possible to
       substantially increase this, perhaps as much as 12x instead of 3x. However, the autolevels function
       would have to be replaced.
    5. Fixed several things I found. The result3 option didn't have the manual color correction code, so I 
       added that back in. All the numbered "stab" variables (stab1, stab2, etc.) aren't needed except 
       for providing a test function for stabilization. I eliminated all of this to streamline the script.
    6. Deflicker seemed redundant, given all the averaging that takes place with MDegrain, and also the averaging
       that is done when the autolevels outputs are selected. 
    7. I reduced the number of frames used for averaging autolevels from the default (which is 5) to 2.
       I probably should add a variable in the header so the user can change this. Something else to do
       in the future ...
    8. I added yet another set of crop parameters. I did this because both my capture and my output are 
       done using NTSC DV AVI which is 720x480. However, 8mm film is almost exactly square, so the captured
       720x480 video has black bars on the side. These need to be cropped off prior to doing motion 
       stabilization, but then added back prior to the final output, which must still be 720x480 with the
       black bars on the side.
    9. I reduced the default depan settings to 20, which is what was recommended in the original script. I
       also reduced the post-depan cropping. I did this so I could keep as much of the original frame
       as possible.
   10. I removed the second denoising and sharpening function. It just seemed to be too much, and made the 
       result too artificial
   11. I removed the MVFLowFPS interpolation. I did this during one of dozens and dozens of attempts
       to improve the speed of the script. I should probably add this back, but if I do so, I also 
       need to make it work correctly for interlaced output. If the goal is to show this on an NTSC or PAL
       television set, then it is not correct to convert from the film fps to 25 fps progressive (PAL) or
       29.97 fps progressive (NTSC). Instead, this should be done as follows (example given is NTSC):
 
          MFlowFPS(source,super,backward_vec, forward_vec, num=60000, den=1001,ml=200)
          SeparateFields()
          SelectEvery(4, 0, 3)
          Weave()
 
       This yields interlaced 29.97, which has twice the temporal resolution as 29.97 progressive, and will
       therefore look correct on a TV set. I have done a lot of this as part of my Kinescope to video
       conversion scripts. For those scripts, the goal is to make the filmed version of a TV show look
       like it was actually videotaped. However, for something that originated on film, this "does violence"
       to the original feel of the media. It is true that it can make horizontal pans less "juddery," but
       it won't feel like film anymore. Also, this technique does break down, espcially with fast motion
       in the foreground.
   12. Removed the unecessary "coloryuv(off_U=blue,off_V=red)" statement from the denoising section.
   13. Added killaudio() statement to prevent lockups when using SetMTMode().
 
Changes made on October 7, 2012
 
    1. Modified RemoveDirtMC function to do estimation in two steps. This improves performance and quality.
    2. Modified MDegrain to do estimation in two steps. This provides roughly the same quality with blocksize=8
       as what I was getting with blocksize=4, but with virtually no performance penalty
    3. Updated to use special build of MVTools2 that works with SVP.
    4. Moved Autolevels inside special multi-threading section in order to improve performance with autolevels.
    5. Removed all trim commands because I do all editing outside the script (in Vegas).
    6. Included two alternative ways to do frame interpolation: SVP and InterFrame. The InterFrame script is probably better
       but I included the SVP code because it may provide a starting point for more customized approaches. Neither is enabled by default.
 
Changes made on September 13, 2017
 
    1. Replaced old useless color correction with GamMac.
    2. Removed never-used manual controls.
    3. Moved deflicker to end of chain because GamMac introduces flicker.
    4. Moved RemoveDirtMC to happen AFTER denoising.
*/
#====================================================================================


#VIDEO FILE
#----------------------------------------------------------------------------------------------------------------------------
#Change the following line to point to your video file
film="e:\fs.avi"


#GENERAL PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
result="result2"                                 #specify the wanted output here 
play_speed=17.98201798                            #play speed (8mm=16; Super8=18; 16mm sound=24)


#COLOR AND LEVELS PARAMATERS
#----------------------------------------------------------------------------------------------------------------------------
saturation = 1.0                                  #for all outputs
gamma      = 1.0                                  #for all outputs 

#GamMac Parameters
LockChan   = 1                                     #(0=red channel)
LockVal    = 128.0                                 #default 128 -- Used when LockChan = -1 (for flicker)
Scale      = 2                                     #Fred recommended 2 instead of 1
RedMul     = 1.0
GrnMul     = 1.0
BluMul     = 1.0
Th         = 0.1
GMx        = 0
GMy        = 0
GMw        = 0
GMh        = 0
LOTH = 0.2
HITH = 0.2
OMIN =   0                                         #limiting the output a little bit makes it a little 'softer' to look at
OMAX = 255
Al2  =  20
autolev_bord1 = 50
borderV=10 borderH=10    

#SIZE, CROP AND BORDERS PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
CLeft=8  CTop=8  CRight=8  CBottom=8                             #crop values after Depan and before final resizing 
W=720  H=480                                                     #final size after cropping 
bord_left=0        bord_top=0     bord_right=0     bord_bot=0    #720p= borders 150
in_bord_left=0  in_bord_top=0  in_bord_right=0  in_bord_bot=0    #Borders around input that must be removed

#STABILISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
maxstabH=10                                                      #maximum values for the stabiliser (in pixels) 20 is a good start value 
maxstabV=10
est_left=40  est_top=40  est_right=40  est_bottom=40             #crop and contast values for special Estimate clip
est_cont=1.1                                                     #Too large a value defeats stabilization


#DENOISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
#denoising_strength=600                    #denoising level of first denoiser: MVDegrain() 
#block_size=   16                           #block size of MVDegrain
#block_size_v= 16
#block_over=    8                           #block overlapping of MVDegrainMulti()

#Alternate denoising using smaller block size
#denoising_strength= 600                   #denoising level of first denoiser: MDegrain() 
#block_size=   8                           #block size of MVDegrain
#block_size_v= 8
#block_over=   4                           #block overlapping of MVDegrainMulti()

#Alternate denoising using smalles block size
denoising_strength= 300                   #denoising level of first denoiser: MDegrain() 
block_size=   4                           #block size of MVDegrain
block_size_v= 4
block_over=   2                           #block overlapping of MVDegrainMulti()

dirt_strength=18                           #sets amount of dirt removal (big spots)


#FOUR STEP SHARPENING PARAMETERS
#--------------------------------------------------------------------------------------------------------------------------------
#120 is pretty aggressive; 85 seems more subtle
#PRE_sharp_ness= 120   PRE_radi_us= 3            #presharpening (UnsharpMask) just after first denoising
 PRE_sharp_ness= 85    PRE_radi_us= 3            #presharpening (UnsharpMask) just after first denoising

Sharp_Strength = 0.05                             #0.05 is often better (avoid too much sharpening)


#AUTO LEVELS PARAMETER
#--------------------------------------------------------------------------------------------------------------------------------
X=4   #X  parameter for reducing the autolevels effect on the whites
X2=4  #X2 parameter for reducing the autolevels effect on the blacks 


#NUMBER OF THREADS
#--------------------------------------------------------------------------------------------------------------------------------
threads=3


# END VARIABLES, BEGIN SCRIPT
#=================================================================================================================================

[johnmeyer]

[edit]Fixed errors in Mrecalculate lines
Script continued ...

SetMemoryMax(768)

AddAutoloadDir("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins",toFront=true)

/*
Here is a list of plugins used
Loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools\mvtools2.dll")
LoadPlugin ("mvtools2.dll")        #Version 2.5.11.9    2/24/2012
LoadPlugin("autolevels.dll")       #Version 0.6.0.0     1/09/2011
LoadPlugin("Deflicker.dll")        #Version 0.4.0.0     8/16/2004
Loadplugin("Depan.dll")            #Version 1.10.0.0    4/09/2007
LoadPlugin("DepanEstimate.dll")    #Version 1.9.2.0     3/25/2007
LoadPlugin("fft3dfilter.dll")      #Version 2.1.1.0     2/20/2007
Loadplugin("mt_masktools.dll")     #Version 2.0.23.0    3/14/2008
loadplugin("RemoveDirtSSE2.dll")   #Version 0.9         5/05/2005    
Loadplugin("RemoveGrainSSE2.dll")  #Version 0.9         5/01/2005
Loadplugin("removegrain.dll")     #Version 0.9         5/01/2005
Loadplugin("warpsharp.dll")        #                    4/05/2010
*/

source1= Avisource(film).killaudio().assumefps(play_speed).converttoYV12()
cropped_source=source1.crop(in_bord_left,in_bord_top,-in_bord_right,-in_bord_bot)               #remove any black borders on input video
#cropped_source=filldrops(cropped_source)     #Use this when removing bad frames that have been removed by duplicating previous frame


#STABILIZING
#....................................................................................................................................................................
stab_reference= cropped_source.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()
mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
#stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15).deflicker()
stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15)

#DENOISING
#...................................................................................................................................................................
input_to_removedirt=stab.crop(CLeft,CTop,-CRight,-CBottom)
stabcrop=RemoveDirtMC(input_to_removedirt,dirt_strength,false)

  prefiltered = RemoveGrain(stabcrop,2)
  superfilt =   MSuper(prefiltered, hpad=32, vpad=32,pel=2)
  super=        MSuper(stabcrop, hpad=32, vpad=32,pel=2)

  halfblksize= (block_size>4)   ? block_size/2 : 4
  halfoverlap= (block_over>2)   ? block_over/2 : 2

  bvec1 =  MAnalyse(superfilt, isb = true,  delta = 1, blksize=block_size, overlap=block_over,dct=0)
  bvec1 =  MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100) 

  fvec1 =  MAnalyse(superfilt, isb = false, delta = 1, blksize=block_size, overlap=block_over,dct=0)
  fvec1 =  MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  bvec2 =  MAnalyse(superfilt, isb = true,  delta = 2, blksize=block_size, overlap=block_over,dct=0)
  bvec2 =  MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  fvec2 =  MAnalyse(superfilt, isb = false, delta = 2, blksize=block_size, overlap=block_over,dct=0)
  fvec2 =  MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

  denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)


#SHARPENING
#...................................................................................................................................................................
sharp1=denoised.sharpen(Sharp_Strength)	
PreBorderFrame = sharp1.Lanczos4Resize(W - bord_left - in_bord_left - bord_right - in_bord_right, H - bord_top - in_bord_top - bord_bot - in_bord_bot)

#FRAME INTERPOLATION (optional)
#...................................................................................................................................................................
#Everything in this section is commented out because I usually choose NOT to do frame interpolation because 
#it causes too many unexpected artifacts on certain frames. It also makes the film look like video which
#aesthetically is not always pleasing.

/*
prefiltered = RemoveGrain(PreBorderFrame,2)
super = MSuper(PreBorderFrame,hpad=16, vpad=16, levels=1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad=16, vpad=16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize=16,overlap=8,search=3,dct=0)
forward = MAnalyse(superfilt, isb = false, blksize=16,overlap=8,search=3,dct=0)
forward_re = MRecalculate(super, forward, blksize=8, thSAD=100)
backward_re = MRecalculate(super, backward, blksize=8, thSAD=100)
PreBorderFrame = MFlowFps(PreBorderFrame,super, backward_re, forward_re, num=30000, den=1001,ml=200,mask=2)
*/

#Alternative interpolation
/*
superfps= MSuper(PreBorderFrame,pel=2)
backward_vec2 = MAnalyse(superfps, isb = true,blksize=16)
forward_vec2  = MAnalyse(superfps, isb = false,blksize=16)
PreBorderFrame = MFlowFps(PreBorderFrame,superfps, backward_vec2, forward_vec2, num=60000, den=1001, ml=200).SeparateFields().SelectEvery(4, 0, 3).Weave()
*/

#Yet another way to do interpolation
#The following alternative can provide synthesized frames for hi-def material that causes MSuper to choke
#////// Begin alternative to MFlowFPS ////
#SmoothFPS2(PreBorderFrame,threads)
#////// End alternative to MFlowFPS ////

#A final way to do interpolation, although Interframe is just a front-end to SmoothFPS, so I don't think the results are any different.
#The following is another alternative that can provide synthesized frames for hi-def material that causes MSuper to choke. Probably better than the one above.
#////// Begin alternative to MFlowFPS ////
#PreBorderFrame=InterFrame(NewNum=30000,NewDen=1001,PreBorderFrame,GPU=true,Cores=threads)
#////// End alternative to MFlowFPS ////

#RESULT1: AUTOLEVELS,AUTOWHITE
#......................................................................................................................................................................

Baseclip = PreBorderFrame.crop(borderV,borderH,-borderV,-borderH,align=true).bicubicresize(W,H)
blank_black = Blankclip(baseclip, width=autolev_bord1,height=autolev_bord1)
blank_white=  Blankclip(baseclip, width=autolev_bord1,height=autolev_bord1, color=$FFFFFF)
Average= baseclip.FredAverage().invert()
over1 = overlay(baseclip,blank_black, x=40,y=300) 
over2 = overlay (over1,blank_white, x=160, y=300) \
        .bicubicresize(width(baseclip)-(al2)*2,height(baseclip)-(al2)*2)
Detect = (al2 >1) ? overlay (Average, over2,x=al2,y=al2) \
         .converttoRGB24(matrix="rec709") : over2.converttoRGB24(matrix="rec709")

result1= PreBorderFrame.ConvertToRGB24.GamMac(verbosity=4,DC=Detect,Show=False, \
         LockChan=LockChan, Th=Th, LockVal=LockVal, Scale=Scale, RedMul=RedMul,\
         GrnMul=GrnMul, BluMul=BluMul, loTh=LOTH,hiTh=HITH,oMin=OMIN,oMax=OMAX,\
         x=GMx,y=GMy,w=GMw,h=GMh).converttoYV12().deflicker().addborders(X,0,0,0,$FFFFFF) \
         .addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0) \
         .addborders(bord_left+in_bord_left, bord_top+in_bord_top, \
         bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT2: AUTOWHITE
#......................................................................................................................................................................
result2= PreBorderFrame.ConvertToRGB24.GamMac(DC=Detect,Show=False, Dither=TRUE,verbosity=5,LockChan=LockChan, \
         Th=Th, LockVal=LockVal, Scale=Scale, RedMul=RedMul, GrnMul=GrnMul, BluMul=BluMul, \
         loTh=LOTH,hiTh=HITH,oMin=OMIN,oMax=OMAX,x=GMx,y=GMy,w=GMw,h=GMh).converttoYV12() \
         .deflicker().addborders(bord_left+in_bord_left, bord_top+in_bord_top, \
         bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT3: AUTOLEVELS
#.....................................................................................................................................................................
result3= PreBorderFrame.addborders(X,0,0,0,$FFFFFF).addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT4: NO CORRECTIONS
#.....................................................................................................................................................................
result4= PreBorderFrame.addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#PARAMETERS FOR THE COMPARISONS
#.....................................................................................................................................................................
W2= W+bord_left+bord_right
H2= H+bord_top+bord_bot
source4=Lanczos4Resize(source1,W2,H2)


#COMPARISONS: ORIGINAL VS RESULTS
#......................................................................................................................................................................
resultS1= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result1,"autolevels, GamMac",size=28,align=2))
resultS2= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result2,"GamMac",size=28,align=2))
resultS3= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result3,"autolevels",size=28,align=2))
resultS4= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result4,"no level or color corrections",size=28,align=2))

Eval(result)
#return Detect
# Enable MT!
Prefetch(threads)


# END SCRIPT, BEGIN FUNCTIONS

#=================================================================================================================================
#REMOVE DIRT FUNCTION
#......................................................................................................................................................................
function RemoveDirt(clip input, int "limit", bool "_grey")
{
  clensed=input.Clense(grey=_grey, cache=4)
  alt=input.RemoveGrain(2)
  return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,dmode=2,debug=false,noise=limit,noisy=4, grey=_grey)

  # Alternative settings
  # return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=true)
  # return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,tolerance= 12,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=false)
}

function RemoveDirtMC(clip,int "limit", bool "_grey")
{
  _grey=default(_grey, false)
  limit = default(limit,6)

# Alternative way to do pre-filtering.
#  prefiltered = fft3dfilter(clip,sigma=1,sigma2=2,sigma3=3,sigma4=5,bw=64,bh=64)
  
  prefiltered = RemoveGrain(clip,2)
  superfilt = MSuper(prefiltered, hpad=32, vpad=32,pel=2)

  super=MSuper(clip, hpad=32, vpad=32,pel=2)

  bvec = MAnalyse(superfilt,isb=true,  blksize=16, overlap=2,delta=1, truemotion=true)
  fvec = MAnalyse(superfilt,isb=false, blksize=16, overlap=2,delta=1, truemotion=true)

# Increase thSAD if moving objects are being removed
  bvec_re = Mrecalculate(super,bvec,blksize=8, overlap=0,thSAD=100)
  fvec_re = Mrecalculate(super,fvec,blksize=8, overlap=0,thSAD=100)

  backw = MFlow(clip,super,bvec_re)
  forw  = MFlow(clip,super,fvec_re)

  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

[sys32768]

Thanks, John. I finally got yours working on my ProRes .MOV files that are 1920 x 1080 but am unclear on size, crop and borders.

It appears that my .MOV files have a 150px left border and 150px right border. I assume for every filter to work correctly, those must be cropped.

If I set those to 150 on in_bord_, the script crashes with an out-of-bounds memory access error on RemoveDirtSSE2. If I set them to 120 it doesn't crash.

I'm also unclear on what my final W= and H= should be in light of the other cropping/border options.

[videoFred]

Originally Posted by sys32768

If I set those to 150 on in_bord_, the script crashes with an out-of-bounds memory access error on RemoveDirtSSE2. If I set them to 120 it doesn't crash.

I think this is a mod8 issue. 1920-150-150= 1620/8= 202.5.
I you crop 152 on both sides, it"s 1920-152-152= 1616/8= 202.

I'm also unclear on what my final W= and H= should be in light of the other cropping/border options.

It depends on the aspect ratio and the final result you want.
Do you want HD again? Then you must add borders again.

And what is your source? Transfered 8mm film or something else?
Because 8mm should be 4:3 and 1616x1080 is not 4:3.

1440x1080 is 4:3.


Fred.

[johnmeyer]

Fred,

Isn't 8mm (without the material between sprocket holes) closer to 1:1? Super 8 is most definitely 4:3, but I think 8mm is pretty close to square.

Also, to add to what Fred said about cropping: he is right that it is a mod issue, but I can't remember if it is mod4 or mod8. However, you'll be safe either way if you follow his suggestion and make the crop a multiple of 8. Also, I've never had good luck using a different crop size for the left and right so I always make the black borders equal on either side in my NLE before feeding it to the script so I can use the same crop size for both. Since you'll want it centered on the screen, this actually makes sense to do even if there wasn't a problem when the sizes are different on the left and the right.

[sys32768]

Originally Posted by videoFred

I think this is a mod8 issue. 1920-150-150= 1620/8= 202.5.
I you crop 152 on both sides, it"s 1920-152-152= 1616/8= 202.

It depends on the aspect ratio and the final result you want.
Do you want HD again? Then you must add borders again.

And what is your source? Transfered 8mm film or something else?
Because 8mm should be 4:3 and 1616x1080 is not 4:3.

1440x1080 is 4:3.

You are correct about 152...math is my weakest subject. Thank you!

My sources are HD 1920 x 1080 16FPS ProRes .MOV files from 8mm reels, transferred by a FlashScan system. They all have the black borders which if cropped makes the full frame about 1560 x 1080. I should look more closely to ensure each file has the same border width.

I was hoping to do everything with these scripts but some post-production editing in Vegas Pro is necessary. I also tend to have more luck with filters like unsharp mask in Vegas Pro because the feedback is so quick, allowing me to fine tune much easier.

I've attached an unprocessed HD frame capture roughly cropped.

John: I put trim beginning back into your script, and am going to attempt to add trim end back. It helps me a lot because several of my reels are more than 30 minutes and often contain half footage I don't want, so it speeds up my processing a great deal to have things split out from the get-go.

[johnmeyer]

Originally Posted by sys32768

I was hoping to do everything with these scripts but some post-production editing in Vegas Pro is necessary. I also tend to have more luck with filters like unsharp mask in Vegas Pro because the feedback is so quick, allowing me to fine tune much easier.

I totally agree: do as much as you can in Vegas.

If you need any help with Vegas workflow, let me know. For instance, I mentioned above that you have to be careful when rendering to MPEG-2 for DVD because you don't want to screw up the pulldown (the fields that are added to go from 15, 16, or 24 fps to 29.97 fps). The key setting in Vegas which will avoid problems is to "disable resample" for every event. I wrote a Vegas script that does this which I'd be happy to post. I have dozens of other Vegas scripts which automate many other aspects of editing film. For instance, one of my first Vegas scripts, from fifteen years ago, is one that will trim an event from the current cursor position to the beginning of the frame. It really helps speed up editing out all those "flash frames" that you get at the beginning of each scene, when the camera's shutter was getting up to speed and therefore overexposing the first 1-2 frames.

I also have several AVISynth scene detection scripts, some of which just look for the flash frames.

[sys32768]

Thanks! I would be interested in scripts/techniques for rendering 16fps 8mm at "true speed". My 16fps 8mm sources seem to be somewhat dreamy slow motion. I am unsure if this is the case with all 8mm or something my family did when filming.

Interpolated to 30fps they look much smoother but they are still slow. Being able to see my departed family members moving at natural speed would be a delight.

[videoFred]

Originally Posted by sys32768

Thanks! I would be interested in scripts/techniques for rendering 16fps 8mm at "true speed". My 16fps 8mm sources seem to be somewhat dreamy slow motion. I am unsure if this is the case with all 8mm or something my family did when filming.

What is your source? Super-8 or Regular-8? Super-8 was standard 18fps, Regular-8 was standard 16fps. But some cameras could be set to higher frame rates like 24 or even 32fps.

Anyhow, if your digital transfer is frame accurate (one film frame = one digital frame, no duplicates) then you can easily change the play speed with assumefps().

Fred.

[videoFred]

Originally Posted by johnmeyer

Fred,

Isn't 8mm (without the material between sprocket holes) closer to 1:1? Super 8 is most definitely 4:3, but I think 8mm is pretty close to square.

Hi John

The film frame of standard (regular) 8mm is 4,50 x 3,30mm. This is almost 4:3. But camera and projector gates are not always the same of cource.

Fred.

[sys32768]

Originally Posted by videoFred

What is your source? Super-8 or Regular-8? Super-8 was standard 18fps, Regular-8 was standard 16fps. But some cameras could be set to higher frame rates like 24 or even 32fps.

Anyhow, if your digital transfer is frame accurate (one film frame = one digital frame, no duplicates) then you can easily change the play speed with assumefps().

Fred.

Hi Fred, these would all be 16fps scans of Regular-8. I was hoping to interpolate them and "normalize" their speed.

[johnmeyer]

Originally Posted by sys32768

Thanks! I would be interested in scripts/techniques for rendering 16fps 8mm at "true speed". My 16fps 8mm sources seem to be somewhat dreamy slow motion. I am unsure if this is the case with all 8mm or something my family did when filming.

That is odd. This can be caused by several things.

1. The original scanning process duplicated frames. This seems unlikely because you are editing them in Vegas and you would notice duplicate frames.

2. The speed header is wrong. I assume you have looked at the media in Vegas' Project Media view where you can see what fps the video is set for. Also, when viewing in Vegas, make sure to match the project properties to the source (there is a little "match" icon in the upper right corner of the Project Properties menu).

3. The people who operated the camera depressed the button too far and filmed at a high rate of speed.

If the problem is #3, then you are actually in luck because if you do things right and simply change the playback speed (don't interpolate frames), you will end up with a beautifully smooth result that looks more like video than film.

For fixing #3, you do NOT want to be doing anything with interpolation because you already have enough frames. The problem, in fact, is just the opposite for which interpolation is usually used. All you want to do is change the fps from 16 to 32, or whatever number gives you playback that seems like the right speed. If you want the playback speed to match a standard, then you change the playback rate to 29.97. In either case, all it takes is a single AssumeFPS(29.97) at the end of the script. What you want is something that does the exact same thing as changing the speed control on a variable speed film projector: you crank up the speed until the motion looks normal.

Remember that these days TV sets can play any speed, so there is no reason to conform the video to the old 24, 25, or 29.95 North American or European standards.

In Vegas, you can experiment to find the correct playback speed by using the J-K-L keys which let you scrub back and forth. Most Vegas users learn how to use these fairly early on, but what most users don't know is that there is a second mode to these keys. If you press and hold the "K" key and then press either the "L" (forward) or "J" (backward) keys, the playback will start at 0.25 of normal speed and then, each time you press the "L" key again (assuming you are playing in the forward direction), the speed will increase by another 0.25. You can release holding the "K" key and Vegas will stay in this playback mode until you press "K" again, at which point it will revert to its regular playback mode. Once in this special mode, if you press the "J" key, instead of immediately playing in the reverse direction, it merely slows down the speed to the next slowest setting.

So use this mode and observe the motion until you feel you have achieved the right speed. I have found that in these situations, if you have to err, make the speed a little too slow, rather than too fast. Also, keep an eye out for a clock with a second hand somewhere in the scene. I've been lucky enough to find a clock many times (I do a lot of sporting events, and old stadium clocks had a sweep second hand). You simply count the number of frames as the second hand crosses adjacent marks on the face, and that will give you a very accurate reading of how many frames per second the camera was taking when the scene was filmed. You can also use physics: if someone drops something from a kitchen counter, those are standard 36 inches height in this country. At 16 fps, that object will take almost exactly 7 frames to hit the floor.

BTW you scan look very good, although there is some red in the upper left corner that may just be a developing artifact in the film, but also may be chromatic aberration in the transfer setup. Also, you might want to use the Color Curves feature in Vegas. I've posted about this many times before. Here is what my preset looks like:

Vegas Color Curves For Film

[sys32768]

Thanks that's a great help. I think 16fps looks unnatural, period, so I may be fooled by that alone.

The curves are definitely helping. I thought GamMac would cure everything.

Can I turn off sharpening on your script (and Fred's) by zeroing out the parameters? I'm preferring to do the sharpening in Vegas post. Also, my footage isn't very noisy so I would likely want to turn that down too.

[johnmeyer]

Fred uses a LOT more sharpening than I do. That was one of the changes I made when I adapted his work a decade ago. Just set the two remaining sharpening parameters to zero in my script.

[sys32768]

I was going to invest in https://www.svp-team.com/wiki/Manual:SVPflow to double my 8mm 16fps to 32fps with interpolation. Is there any reason I'd want that to be 30fps instead of 32fps?

[wonkey_monkey]

Is there any reason I'd want that to be 30fps instead of 32fps?

It would look smoother on displays. 32fps would result in a 2/2/2/2/2/2/1 pattern (or something like that) when viewed on a 60fps display. Maybe noticeable, maybe not. If it was me, I'd go to 32fps by interpolation so every other frame is a "real" frame, then I'd assumefps(30) to avoid that little judder. I wouldn't interpolate directly to 30fps because then the majority of frames would be interpolations.