View Single Post
Old 12th October 2018, 18:25   #71  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 85
Two problems, one is that after I modified the script to disable YV24 conversion, limit the sharp parameter and introduce the negative badRange (didn't work with -50..50 values so I added a boolean flag according to which I do or do not multiply the range by -1), the script started crashing, complaining that "MflowFps can't work in reentrant multithreading" even though I didn't touch anything MT-related. While I do have MT versions of Avisynth 2.6 and Avisynth+ installed I do not call setmtmode or prefetch in either but it crashes in both randomly, I'm lucky if I manage to get one run finished.

I used to get this error before AvsOptimizer sometimes too, usually when switching between multiple heavy scripts in Avsp editor, but it was a rare occurrence. Before I edited the script I managed to run it overnight totalling in over a thousand iterations without a problem. If it didn't work right away I'd consider that launching two upsampling operations in succession like we do here might've been the culprit.

Here's my script as of now:

Quote:
TEST_FRAMES = 10
MIDDLE_FRAME = 600

# original framerate
FPS_NUM = 30
FPS_DEN = 1

# source clip
Asrc=FFmpegSource2("c:\Hibikin - Watashtachi wa Zutto... Deshou (AVS test video 60fps 720p 10bit CRF0).mkv",threads=1 ).assumefps(60)
Asrc=Asrc.trim(0,60*30-9)+Asrc.trim(60*40+60*60+9,0)
asrc.selecteven
AssumeFPS(FPS_NUM, FPS_DEN)

#return last

# needed for some parameter combinations
#ConvertToYV24()

orig = last

mt=true #happens without this parameter as well since mt is true by default, just introduced it for later use


super_pel = 2 # optimize super_pel = _n_ | 1,2,4 | super_pel
super_sharp = 2 # optimize super_sharp = _n_ | 0..2 ; max:super_pel 1 == 0 2 ? | super_sharp
super_rfilter = 4 # optimize super_rfilter = _n_ | 0..4 | super_rfilter
super_render = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, orig, mt=mt )

blockSize = 12 # optimize blockSize = _n_ | 4,6,8,12,16,24,32,48,64 ; min:divide 0 > 8 2 ? ; filter:overlap 2 * x <= | blockSize
searchAlgo = 3 # optimize searchAlgo = _n_ | 0..5 D | searchAlgo
searchRange = 4 # optimize searchRange = _n_ | 1..30 | searchRange
searchRangeFinest = 4 # optimize searchRangeFinest = _n_ | 1..40 | searchRangeFinest
lambda = 16000 # optimize lambda = _n_ | 0..20000 | lambda
lsad=400 # optimize lsad=_n_ | 8..20000 | lsad
pnew=0 # optimize pnew=_n_ | 0..256 | pnew
plevel=0 # optimize plevel=_n_ | 0..2 | plevel
overlap=4 # optimize overlap=_n_ | 0,4,8,12,16,20,24,28,32 ; max:blockSize 2 / | overlap
divide=2 # optimize divide=_n_ | 0..2 ; max:blockSize 8 >= 2 0 ? overlap 4 % 0 == 2 0 ? min | divide
globalMotion = true # optimize globalMotion = _n_ | false,true | globalMotion
badSAD = 2000 # optimize badSAD = _n_ | 4..10000 | badSAD
badRange = 24 # optimize badRange = _n_ | 2..50 | badRange
negBadRange = false # optimize negBadRange = _n_ | false,true | negBadRange
meander = true # optimize meander = _n_ | false,true | meander
temporal = false # optimize temporal = _n_ | false,true | temporal
trymany = false # optimize trymany = _n_ | false,true | trymany
dct = 0 # optimize dct = _n_ | 0,2,3,4,5,6,7,8,9,10 D | dct

delta = 1
useChroma = true

resultBadRange = (negBadRange ? -1 : 1)*badRange
bv = MAnalyse(super_render, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap ,
\ divide=divide, badSAD=badSAD, badrange=resultBadRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct, mt=mt)
fv = MAnalyse(super_render, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap ,
\ divide=divide, badSAD=badSAD, badrange=resultBadRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct, mt=mt)



threshold = 10000
maskScale = 100 # optimize maskScale = _n_ | 1..300 | maskScale
mask_fps = 2 # optimize mask_fps = _n_ | 0..2 | mask_fps
inter = orig.MFlowFPS(super_render, bv, fv, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold )

# return this to look at the clip with doubled framerate
#return inter

fps_only = inter.SelectOdd()

# second pass
super_render2 = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, fps_only )
bv2 = MAnalyse(super_render2, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap,
\ divide=divide, badSAD=badSAD, badrange=resultBadRange, meander=meander, temporal=temporal, trymany=trymany, mt=mt)
fv2 = MAnalyse(super_render2, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap,
\ divide=divide, badSAD=badSAD, badrange=resultBadRange, meander=meander, temporal=temporal, trymany=trymany, mt=mt)
inter2 = fps_only.MFlowFPS(super_render2, bv2, fv2, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold )
fps_only2 = inter2.SelectOdd()


delimiter = "; "

inter_yv12 = fps_only2#.ConvertToYV12()
orig_yv12 = orig#.ConvertToYV12()

# for comparison original must be forwarded one frame
orig_yv12 = trim(orig_yv12,1,0)

inter_yv12 = inter_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
orig_yv12 = orig_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = inter_yv12

global total = 0.0
global ssim_total = 0.0
global avstimer = 0.0
frame_count = FrameCount()
FrameEvaluate(last, """
global ssim = SSIM_FRAME(orig_yv12, inter_yv12)
global ssim_total = ssim_total + (ssim == 1.0 ? 0.0 : ssim)
""", args="orig_yv12, inter_yv12, delta, frame_count")

# NOTE: AvsTimer call should be before the WriteFile call
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (ssim, time)
resultFile = "c:\avsoptim\results\perFrameResults.txt" # output out1="ssim: MAX(float)" out2="time: MIN(time) ms" file="c:\avsoptim\results\perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "ssim", "delimiter", "avstimer")
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "ssim_total", append=true)

return last
The second thing is that limiting iterations by time is rather loose, apparently because it does not include the time spent on generating populations, so where I set it to 3 runs with 3 hours it went for 3:05, 3:20 and over 4 hours on last run. Not very convenient when you expect it to be done in the morning.

Also, for some reason sensitivity estimation is always marked as N/A in the log, even if I try to pass -sensitivity true. It would be interesting to see which parameters are more important.
Seedmanc is offline   Reply With Quote