RemoveDirt crashes depending on mod

[Selur]

I noticed something odd when I using (this avi)
MPEG-4 Visual, color space: YUV420P8, bit depth: 8, resolution: 1500x1080, fps: 24, color matrix: 709, yuv luminance scale: limited, scanorder: progressive

and the following script:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/RemoveDirt/RemoveDirtVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import removeDirt
# current color space: YUV420P8, bit depth: 8, resolution: 1500x1080, fps: 24, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading C:\Users\Selur\Desktop\RemoveDirtCras\RemoveDirtConfusion.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/clips/RemoveDirtCrash/RemoveDirtConfusion.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
# Setting detected color matrix (709).
clip = core.std.SetFrameProps(clip, _Matrix=1)
# Setting color transfer info (709), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=1)
# Setting color primaries info (), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 24
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive

clip = removeDirt.RemoveDirt(input=clip, limit=10)

# set output frame rate to 24fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
# Output
clip.set_output()

Processing the script directly crashes when rendering the second frame.

when add to mod8:

clip = core.std.AddBorders(clip=clip, left=2, right=2, top=2, bottom=2) # add borders to archive mod 8 (vsRemoveDirt) - 1504x1084
# denoising using VsRemoveDirt
clip = removeDirt.RemoveDirt(input=clip, limit=10)
clip = core.std.CropRel(clip=clip, left=2, right=2, top=2, bottom=2) # removing borders (vsRemoveDirt) -  1500x1080

it works fine.
using:

clip = core.std.AddBorders(clip=clip, left=2, right=2, top=0, bottom=0) # add borders to archive mod 8 (vsRemoveDirt) - 1504x1080
# denoising using VsRemoveDirt
clip = removeDirt.RemoveDirt(input=clip, limit=10)
clip = core.std.CropRel(clip=clip, left=2, right=2, top=0, bottom=0) # removing borders (vsRemoveDirt) -  1500x1080

so seems like mod16 width is the problem.

when add to mod16:

clip = core.std.AddBorders(clip=clip, left=4, right=4, top=4, bottom=4) # add borders to archive mod 16 (vsRemoveDirt) - 1508x1088
# denoising using VsRemoveDirt
clip = removeDirt.RemoveDirt(input=clip, limit=10)
clip = core.std.CropRel(clip=clip, left=4, right=4, top=4, bottom=4) # removing borders (vsRemoveDirt) -  1500x1080

it crashes.

=> Would be nice to know if others can reproduce this too.

Cu Selur

[_Al_]

Is there RemoveDirtVS.dll available for download somewhere?
I think it is needed to run core.rdvs in script.

[Selur]

I uploaded the two RemoveDirt libaries I got (RemoveDirtVS.dll and vsremoveDirt.dll) to my GoogleDrive.
RemoveDirtVS.dll was from (https://forum.doom9.org/showthread.php?t=169771 when the download worked and no clue where the other was from, but both have this issue.
If someone has a newer RemoveDirt build which might fix this, please let me know.

Cu Selur

[_Al_]

same results as you described, original video crushed, it worked with first crop(mod8), not the second(mod16), used this script:

import vapoursynth as vs
from vapoursynth import core

def RemoveDirt(input, repmode=16, remgrainmode=17, limit=10):
  cleansed = core.rgvs.Clense(input)
  sbegin = core.rgvs.ForwardClense(input)
  send = core.rgvs.BackwardClense(input)
  scenechange = core.rdvs.SCSelect(input, sbegin, send, cleansed)
  alt = core.rgvs.Repair(scenechange, input, mode=[repmode,repmode,1])
  restore = core.rgvs.Repair(cleansed, input, mode=[repmode,repmode,1])
  corrected = core.rdvs.RestoreMotionBlocks(cleansed, restore, neighbour=input, alternative=alt, gmthreshold=70, dist=1, dmode=2, noise=limit, noisy=12)
  return core.rgvs.RemoveGrain(corrected, mode=[remgrainmode,remgrainmode,1])

clip = core.ffms2.Source("RemoveDirtConfusion.avi")
clip = RemoveDirt(input=clip, limit=10)
clip.set_output()

[Selur]

Thanks for testing, now I at least know that it is not some strange thing due to my system.

[yuygfgg]

I've converted all SSE code to scalar, and the 1508x1088 input clip works for me.

Also fixed an issue related to dFactor.

Please test to see if it resolves your issue.

My fork: https://github.com/yuygfgg/removedirtvs. Binaries can be found in Actions.

[Selur]

Nice Will test and report back.

Cu Selur

[Selur]

Ran it through ~20 test clips and had no crash, so it looks fine so far.
Will report back in case I encounter problems.
Thanks.

Cu Selur

[Selur]

@yuygfgg: planning a release any time soon?

[yuygfgg]

Originally Posted by Selur

@yuygfgg: planning a release any time soon?

https://github.com/yuygfgg/removedirtvs/releases/tag/R1

[Z2697]

Of course you need a release, actions artifacts arn't "perpetual"

[Selur]

Thanks!
I usually try to build the last tagged release, and actions tend to vanish after a while.

Cu Selur

[yuygfgg]

I just found that pinterf's fork also supports vapoursynth. If it works, it would be a better replacement of mine.

https://github.com/pinterf/RemoveDirt

[Selur]

Thanks for the info.

[Z2697]

The SSE code might just be too old, the scalar version or more precisely the optimization done by compiler is slightly faster.
Maybe you can do it on the pfmod fork, or port the pfmod to your fork.

[Selur]

yes, a pull request to pinterf would be cool.