Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th January 2021, 12:23   #61  |  Link
Frank62
Registered User
 
Join Date: Mar 2017
Location: Germany
Posts: 138
Sorry, bad idea. Should rather be the opposite. Doubles are good for removedirting. Motion-compensated frames are of course less good.
So forget my proposal...
Frank62 is offline   Reply With Quote
Old 27th January 2021, 13:40   #62  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
Maybe just try the temporal median filter that spotless uses.
Code:
RadT = ??? [int, try bout 2]
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT)  # Temporal median blur only [not spatial]
If it is so static, it might work well.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th January 2021 at 13:51.
StainlessS is offline   Reply With Quote
Old 27th January 2021, 16:00   #63  |  Link
Frank62
Registered User
 
Join Date: Mar 2017
Location: Germany
Posts: 138
What exactly makes it more static than f. e. RemoveDirt?
To me it seems somehow more stepless, but how does it do this? "Blur" implicates some temporal interpolating, but it seems to be more than this.
Frank62 is offline   Reply With Quote
Old 27th January 2021, 16:29   #64  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
Dont know but,
Here it is on Wiki:- http://avisynth.nl/index.php/MedianBlur2
And link (from wiki page) to a paper on it:- http://nomis80.org/ctmf.html
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 27th January 2021, 18:40   #65  |  Link
Frank62
Registered User
 
Join Date: Mar 2017
Location: Germany
Posts: 138
Thanks, but it says nothing about the temporal part of filtering.
Frank62 is offline   Reply With Quote
Old 14th February 2021, 20:28   #66  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 54
Another variant of the script with saving fast moving objects.

Quote:
# MvTools2, RemoveDirt, RgTools, DeSpot
Function SpotLess2(clip c, int "RadT", int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2", int "pel", bool "chroma", int "BlkSz", Int "Olap", bool "tm", Bool "glob") {
RadT = Default(RadT, 1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD, 10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2, ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel, 2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma, true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz, 8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, 2) # Default half of BlkSz.
Tm = Default(tm, true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob, true) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Spot = Default(Spot, 32) # Spot size
DeGrain = Default(DeGrain, false) # MDeGrain in MvTools

pad = max(BlkSz, 8)
sup = c.ColorYUV(levels="PC->TV").MSuper(hpad=pad, vpad=pad, pel=pel, sharp=2)
sup_rend = MSuper(c, hpad=pad, vpad=pad, pel=pel, sharp=2, levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = MAnalyse(sup, multi=true, delta=RadT, blksize=BlkSz*2, blksizeV=BlkSz*2, overlap=OLap*2, chroma=Chroma, truemotion=Tm, global=Glob)#, search=5, dct=7, divide=0, plevel=0)

# c0 = DeGrain ? MDeGrainN(c1, sup_rend, MultiVec, RadT, mt=true, plane=4, limit=255.0, thSad=400) : c # gave a bad result

c1 = MCompensate(c, sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=130)
c1 = c1.ForwardClense() #faster than MedianBlurTemporal(radiusY=0, radiusU=0, radiusV=0, temporalradius=RadT) # Temporal median blur only [not spatial]

c1 = Repair(SelectEvery(c1, RadT*2+1, RadT), c.RemoveDust().Sharpen(0.1), mode=17) # Return middle frame

bvec1 = MAnalyse(sup, isb=true, blksize=BlkSz, blksizeV=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0, truemotion=true) # backward vectors
fvec1 = MAnalyse(sup, isb=false, blksize=BlkSz, blksizeV=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0, truemotion=true) # forward vectors
backw1 = MFlow(c1, sup_rend, bvec1)
forw1 = MFlow(c1, sup_rend, fvec1)
clipa = interleave(backw1, c1, forw1)

filtered = DeSpot(clipa, p1=6, p2=6, pwidth=Spot, pheight=Spot, mthres=16, mwidth=Spot/2, mheight=Spot/2, blur=1, motpn=true, seg=2, color=false, ranked=true, show=0)
restore = clipa.RemoveDust().Sharpen(0.1)

clipb = RestoreMotionBlocks(filtered, restore, alternative=RemoveGrain(clipa,2), pthreshold=10, cthreshold=10, gmthreshold=160, dist=3, dmode=2, noise=3, noisy=9, grey=false, debug=false, show=false)

SelectEvery(clipb, 3, 1)

DeGrain ? MDeGrain1(last, sup_rend, bvec1, fvec1, plane=4, limit=255.0, thSad=400) : last
}


function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep=Repair(clensed, input, mode=mode)

RemoveGrain(rep)
}
Arx1meD is offline   Reply With Quote
Old 15th February 2021, 20:33   #67  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
Arx1meD,
Perhaps you should create your own thread for your script function, also maybe give it another name other than Spotless2(),
as it does not function much at all like the Spotless filter.
I only tried this to see if it threw an error or not due to you having 2 [EDIT: formal] args named RadT,

Code:
AviSourcE("D:\parade.avi")
Spotless2()
However, caused access violation on line 33, ie the DeSpot line.

Suggest you create new thread, and correct/rename your script filter,
and post an example sample clip together with a working client script.
EDIT:
Code:
Function SpotLess2(clip c, int "RadT", int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2",
    \  int "pel", bool "chroma", int "BlkSz", Int "Olap", bool "tm", Bool "glob") {
also, it is not good practice to destroy compatibility by inserting new formal arguments before existing arguments ie your
Code:
 int "Spot", bool "DeGrain"
EDIT: Version Despot I was using was v3.6.3.0, there is a v3.6.3.1 here:- http://www.avisynth.nl/index.php/DeSpot
V3.6.3.1 does not cause access violation.

Strangely, it does not throw an error due to 2 different instances of the formal parameter "RadT", maybe a bit more checking needed in Avs+.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th February 2021 at 23:27.
StainlessS is offline   Reply With Quote
Old 16th February 2021, 07:45   #68  |  Link
Arx1meD
Registered User
 
Arx1meD's Avatar
 
Join Date: Feb 2021
Posts: 54
Thank you StainlessS. I will fix the errors and create another thread.
I am using Avs+. When testing the script, I got no errors.

Last edited by Arx1meD; 16th February 2021 at 07:49.
Arx1meD is offline   Reply With Quote
Old 2nd April 2021, 00:45   #69  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
New Test [1.07beta] version of SpotLess, comments ?

Changed bBlur default from 0.0 [OFF] to 0.6, slower but should be considerably better on MC analysis.
Would it be preferable to remain at 0.0 faster.

Also, added DC [prefiltered detection clip], where supplied, bBlur prefiltering ignored and uses dc clip instead.

Code:
/*
    SpotLess v1.07Beta. temporal spot/noise remover, by StainlessS @ Doom9.   https://forum.doom9.org/showthread.php?t=181777
    Original idea from Didée post :-                                      https://forum.doom9.org/showthread.php?p=1402690#post1402690
    Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
          With appropriate plugins, will be AVS+ colorspace and x64 compatible.
          Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

    NOT FOR cartoon/anime, live video only, sorry.

    v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
    v1.02, Add some stuff.
    v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
    v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
    v1.05, Additional checks on args.
    v1.06, Glob Default true, Almost always better.
    v1.07Beta, bBlur Default changed from 0.0 OFF, to 0.6, slower but better for mc analysis. Maybe try 1.0 for HD to overcome HD 'gritiness'.
               Added dc arg, detect clip.

    SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
            \ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.6, clip "dc"=Undefined)

    RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
        RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
        Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

    ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
        ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
        ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
        In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
        greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
        in the result MCompensated frame].
        Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
        block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
        or some other mixture.
          A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
        block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
        after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
        threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
        Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
          It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
        no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
        middle pixel value.
        I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
            SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
        And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
        as we still had the temporal median filter to fall back on and pick the middle pixel value.

        From mvtools2 docs:
          ThSAD is SAD threshold for safe (dummy) compensation.
              If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

    ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
        ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
        acquiring a SAD threshold linearly interpolated between the two.
        From mvtools2 docs:
          ThSAD2:
              Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
              The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
              and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
              risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
        EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
        however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
        of having too high a SAD threshold.

    Pel,     Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

    Chroma,  Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

    BlkSz,   Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

    OLap,    Default half BlkSz, Block overlap.

    Tm,      TrueMotion Default True. Some folk swear truemotion=false is better.

    Glob,    Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
             From MvTools2 docs for MAnalyse,
               global
                 Estimate global motion (at every level) and use it as an additional predictor.
                 Only pan shift is estimated (no zoom and rotation).
                 Use false to disable, use true to enable.

    bBlur,   Default 0.6. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip
             for better motion analysis. Maybe try 1.0 for HD clip to counteract HD 'grittiness'. bBlur ignored if dc clip specified,
             If providing bBlur, then ALWAYS specify as named argument ie bBlur=bBlur, we will likely insert any additional MvTools args
             before bBlur to keep them together.

    dc,      Default UnDefined. If dc Specified, then must be same size and colorspace as source clip.
             If detection clip specified then bBlur prefilter arg is ignored, and analysis is performed on
             dc.MSuper() clip [ instead of c.Blur(bBlur).MSuper() ]. Allows to provide your own prefiltered clip.
             If providing dc clip, then ALWAYS specify as named argument ie dc=DC, we will likely insert any additional MvTools args
             before bBlur to keep them together.

*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",
    \ Float "bBlur", clip "dc" ) {
    myName   = "SpotLess: "
    RadT     = Default(RadT,1)        # Temporal radius. (MCompensate arg)
    ThSAD    = Default(ThSAD,10000)   # SAD threshold at radius 1 (Default Nearly OFF).
    ThSAD2   = Default(ThSAD2,ThSAD)  # SAD threshold at radius RadT.
    Pel      = Default(pel,2)         # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
    Chroma   = Default(chroma,True)   # MAnalyse chroma arg. If set to true, use chroma in block matching.
    BlkSz    = Default(BlkSz,8)       # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps  better, esp for HD clips. Maybe also better where BIG noise.
    OLap     = Default(OLap, BlkSz/2) # Default half of BlkSz.
    Tm       = Default(tm,True)       # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
    Glob     = Default(glob,True)     # Default True, Allow set MAnalyse(global) independently of TrueMotion.
    bBlur    = Default(bblur,0.6)     # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
    HasDC    = dc.Defined             # bblur ignored if HasDC, ie user already provided prefiltered clip.
    Assert(1 <= RadT,myName + " 1 <= RadT")
    Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
    Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
    pad = max(BlkSz,8)
    sup = (HasDC ? dc : bBlur>0.0  ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
    # Only 1 Level required where not MAnalyse-ing.
    sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
    MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
    c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
    MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT)  # Temporal median blur only [not spatial]
    SelectEvery(RadT*2+1,RadT)                                             # Return middle frame
}
I'm quite happy with it as is, but anybody anything to say before I post as new non beta update.

Here some Local de-flicker whotsit, can be used for DC clip where local flicker.
Code:
Function DeFlickLocal(clip c) {
/*
    Local DeFlicker. Didee, https://forum.doom9.org/showthread.php?p=1599682#post1599682
    Calms local flickering, where some small object in frame flickers, rather than whole frame flickering.
    [Didee:]
        A small clip is created. The small clip is temporal-softened.
        The difference achieved by temporalsoften is taken from the small clip, then re-scaled, and applied to the original clip.
        Effectively, this is (very similar to) a spatial lowpass on the difference of a normal temporalsoften.
        In particular with MVTools-Denoising etc., then the result makes a good source for motion vector search. No need for sluggish DCT=1.
*/

    c
    w=Width/8*2  h=Height/8*2                                #  Can be altered, but ~25% of original resolution seems reasonable
    sm  = c.bicubicresize(w,h, b=1.0/3, c=1.0/3)             # b,c as defaults, b=blurring, c=ringing
    smm = sm.temporalsoften (1,32,255,24,2).merge(sm,0.25)
    smm = smm.temporalsoften(2,12,255,20,2)
    c.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(c.width,c.height,b=0,c=0),U=3,V=3) # blurring=0.0, ringing=0.0
}
Also, GMJCZP posted a script function Small_Deflicker():- https://forum.doom9.org/showthread.p...60#post1812060
which is based on DeFlickLocal(), which is an implementation of a Didee script as a function.
NOTE Didee comments.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 2nd April 2021 at 01:55.
StainlessS is offline   Reply With Quote
Old 2nd April 2021, 03:50   #70  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 516
Hi TinMan, I wish you good luck with your script. The biggest challenge you are going to have is with anime, specifically with scenes with rain, because if the spot detection method is incorrect you will notice how the raindrops disappear before your eyes, KillerSpots was precisely tested exhaustively under these conditions
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 2nd April 2021, 04:13   #71  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
Hi GMJCZP, [I think I messaged you when I implemented Spotless, to let you know I extended your/didee script.]
I'm personally not bothered about anime, and said 'not for anime' in 1st or 2nd post, as uses RadT greater than 1, there is little
chance that any anime rain would survive RadT>=2.
If you aint tried it [SpotLess], give it a go, is almost magic with RadT>1, removes double spots, and even had it remove 4 consecutive BIG BIG BIG spots.
Glad to see you back, keep safe.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 2nd May 2021, 12:56   #72  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,820
SpotLess is perfect for my VHS sources. Thank you!
Sharc is offline   Reply With Quote
Reply

Tags
denoise, despot, prefilter

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.