Nasty noise on a bad DVD transfer

[samf88]

Hi, hope someone out there can help me out.

I've ripped a DVD which looks shockingly bad and I was hoping there was something I could do to clean it a little. I fear it's too far gone and I'd end up decimating what little detail there is, but maybe there's a magic filter chain out there that's perfect for making this situation slightly more bearable? I think this will be one of those shows that was edited on video so a lot of the damage may be permanent.

I've been dabbling with MDegrain3 which does clear up small parts of the picture on default settings, but everything else (such as TBilateral, VagueDenoiser, RemoveGrain) has to be ramped up to insane settings to even touch the sides of this awful noise, probably because each speck is massive. Does it have a name? I thought originally it was chroma noise, but now I'm not so sure.

This is a particularly bad frame, just using MPEG2Source and QTGMC's Very Slow preset.


Here's a video excerpt cut from the VOB, incase it's easier to identify frame-by-frame: https://drive.google.com/open?id=1Sr...fF7Rj8qShxhu3h

If anyone can shed some light - or even recommend a filter/chain that may help in some way - I'd really appreciate it. This show has a very special place in my heart and I wish it was treated better for its (now out of print) DVD release.

I use AVISynth+ 64 bit, if that matters.

[tebasuna51]

For what do you use QTGMC?
It is not interlaced, and you can use the QTGMC denoisers directly to see what is your prefered.

[samf88]

Hi - thanks for replying!

QTGMC is being used to deinterlace the TFF footage which contains visible combing. I did try with EZDenoise early on in my quest but unfortunately it didn't really affect this type of noise on sensible values, and of course high values just turn it to a blur. More advanced settings - such as switching to dfttest - also don't seem to touch it.

I'm guessing I would need to combine a filter with some kind of edge mask to try and retain some detail, but I highly doubt I'll be able to get good results from this.

[hello_hello]

Just throwing some scripts at it, I preferred the TemporalDegrains.

TemporalDegrain removes more noise than TemporalDegrain2, but it blurs more. You'd probably have to encode a bit to judge how it looks when the pictures are moving. MAA is from the AnimeIVTC script, or there's MAA2. It's one of real.finder's functions, as is DeHalo_alpha_mt
https://forum.doom9.org/showthread.php?t=174121

Some SMDegrain experts might come along with some clever denoising settings using it instead.

MAA()
TemporalDeGrain()
DeHalo_alpha_mt()

Source


TemporalDegrain


TemporalDegrain2

[VoodooFX]

What about something like:

MCDegrainSharp( frames=1, bblur=0.2, csharp=0.0 )
QTGMC( Preset="Medium", InputType=1, Sharpness=0.6 )
Santiag(1,1, nns=4)

QTGMC "Medium" wouldn't blur fast motion areas much like other presets, but sometimes in can create some stabilization issues, when "Slower" stabilization is more precise/better.

Originally Posted by hello_hello

I preferred the TemporalDegrains.

Don't you get some strange "effects-bugs" with TemporalDegrain2? Every time I tried it I noticed something strange going on besides noise removal.

[samf88]

Originally Posted by VoodooFX

What about something like:

MCDegrainSharp( frames=1, bblur=0.2, csharp=0.0 )
QTGMC( Preset="Medium", InputType=1, Sharpness=0.6 )
Santiag(1,1, nns=4)

QTGMC "Medium" wouldn't blur fast motion areas much like other presets, but sometimes in can create some stabilization issues, when "Slower" stabilization is more precise/better.

Many thanks for your contribution.

I'm seeing aliasing on the black/white stripey top with InputType=1, but InputType=3 seems to solve that. http://www.framecompare.com/image-co...rison/999F1NNU

I think I might have to treat the end credits differently to the rest of the script though. There does seem to be 50fps worth of unique frames there in the background graphic, although given that it's only end credits I might just leave it. http://www.framecompare.com/image-co...rison/999FMNNU InputType=1 does seem to look best for the text, but InputType=3 looks best for the graphics. (Link if anyone wants to investigate: https://drive.google.com/file/d/1ohK...ew?usp=sharing)

[hello_hello]

Originally Posted by VoodooFX

Don't you get some strange "effects-bugs" with TemporalDegrain2? Every time I tried it I noticed something strange going on besides noise removal.

I haven't used it much, but I didn't see anything nasty. The main problem I have with it, is I don't seem to be able to adjust how much noise it removes to any great extent. When there's just the right amount of noise it works, but considering the denoising logic came from QTGMC you'd think it'd have something like the EzDenoise option where you can simply dial in how much noise removal you want.

One day I might pull QTGMC apart to see if I can do better. Probably not, but maybe one day I'll try.

[samf88]

Thanks! That's wonderful, gives me something to build on. I really appreciate your help, thank you. :-)

If anyone else has any alternative suggestions I'd still be happy to hear them.

[hello_hello]

I had a silly idea. I can't say I've done it before myself (it'd be dog slow), and I don't know if there'd be a better way to go about it, but what if you only denoised the chroma, then denoised the chroma and luma? You could probably get away with less noise removal the second time around because it'd look more like grain. It's probably mainly an improvement for flat areas of color, such as the sky, and not so much everywhere else.

A = MAA()
MergeChroma(A, A.TemporalDegrain())
TemporalDegrain2()
DeHalo_alpha_mt()



You could do something like this, just to double denoise where it's needed.

A = MAA()
B = A.TemporalDeGrain2()
C = MergeChroma(A, A.TemporalDegrain()).TemporalDegrain2()

B.Trim(0,478) ++ C.Trim(479,683) ++ B.Trim(684,0)
DeHalo_alpha_mt()

Except the logic for running two instances of TemporalDeGrain2 in a script doesn't seem right. To make that work you need to change line 156 in the script to this:

ReplaceGlobals = true

[Mounir]

That should be more than enough to me:

SetMemoryMax(1200)
SetMTMode(3,4)
MPEG2Source("tgft.d2v", cpu=0, info=3)
# Step 1: Remove chroma noise:
assumetff()
ConvertToYV16(interlaced=true)
orig=last
ev=orig.assumetff().separatefields().selecteven()
od=orig.assumetff().separatefields().selectodd()
ev
ue_chroma = UToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
ve_chroma = VToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(ue_chroma, ve_chroma)
MergeLuma(ev)
ev_filtered=last
od
uo_chroma = UToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
vo_chroma = VToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(uo_chroma, vo_chroma)
MergeLuma(od)
od_filtered=last
interleave(ev_filtered,od_filtered)
assumefieldbased().assumetff().weave()
# Step2: color correction
ConvertToRGB32(interlaced=true)
########
LoadVirtualDubPlugin("C:\Program Files (x86)\VirtualDub\plugins\CMYK_rj.vdf", "CMYK", 0)
CMYK(255,0,266,10,257,-24,255,0,0,42) # Modifying Yellow high tones (-24%) and magenta lows (4%)

https://www.hostpic.org/view.php?fil...0130260109.jpg

[Boulder]

QTGMC(preset="very slow", inputtype=2) should already help nicely as it will press down the sharp noise and remove some aliasing.
It's hard to remove all the noise without removing all the detail. If you like, you could hit it with something TemporalDegrain2 and apply some light artificial noise with GrainFactory to fake detail.

[samf88]

Originally Posted by hello_hello

I had a silly idea. I can't say I've done it before myself (it'd be dog slow), and I don't know if there'd be a better way to go about it, but what if you only denoised the chroma, then denoised the chroma and luma? You could probably get away with less noise removal the second time around because it'd look more like grain. It's probably mainly an improvement for flat areas of color, such as the sky, and not so much everywhere else.

Thank you so much, I do really like those results. Thanks so much for your help

Originally Posted by Mounir

That should be more than enough to me:

Thanks so much for that. I've had to modify it slightly for 64 bit, but I do like the results. Still trying to find a 64 bit version of the VDub filter, but I tried it manually in VDub itself to see the results. Thanks for taking the time to work on that, I really appreciate it.

Originally Posted by Boulder

QTGMC(preset="very slow", inputtype=2) should already help nicely as it will press down the sharp noise and remove some aliasing.
It's hard to remove all the noise without removing all the detail. If you like, you could hit it with something TemporalDegrain2 and apply some light artificial noise with GrainFactory to fake detail.

Thanks - definitely thinking I will do pre-treatment with QTGMC before anything else. I will look in to GrainFactory - that's a good idea to reintroduce some "grain" to disguise other things.

Much appreciated everyone!

EDIT: Here's what I've been able to achieve so far, using the help provided here: http://www.framecompare.com/screensh...rison/99J2JNNU. I'm much happier with this than what I was getting a few days ago.

[Katie Boundary]

I swear by TNLmeans: http://avisynth.nl/index.php/TNLMeans

There's also a KNLmeans but I think it requires an NVIDIA GPU or something.

[samf88]

Thanks! I actually use KNLMeansCL quite often but on its own it wasn't touching the main problem. My current filter chain (with results shown in the link I edited in my post above) does include it as part of a denoise mask.

[hello_hello]

Originally Posted by samf88

Thanks! I actually use KNLMeansCL quite often but on its own it wasn't touching the main problem. My current filter chain (with results shown in the link I edited in my post above) does include it as part of a denoise mask.

So don't keep us in suspense. What's the script you're using?

[samf88]

Below is what I've got at the moment. Hopefully it all makes sense and I haven't made any silly mistakes. I'm using AvsPmod to preview/make changes which seems to crash every time I change a parameter, so it wasn't an easy task to adjust things. I haven't yet found an alternative filter to adjust the CMYK levels as suggested by Mounir. I'm also not confident in my grain settings and keep adjusting them. It's possible I've used the masks wrong too - I never was confident using those. I may have switched colour space more than necessary too. I've also taken out all the SetFilterMTMode/Prefetch stuff for the sake of posting here.

MPEG2Source("movie1.d2v", cpu=0)

# Mounir's Chroma Noise Removal
assumetff()
ConvertToYV16(interlaced=true)
orig=last
ev=orig.assumetff().separatefields().selecteven()
od=orig.assumetff().separatefields().selectodd()
ev
ue_chroma = UToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
ve_chroma = VToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(ue_chroma, ve_chroma)
MergeLuma(ev)
ev_filtered=last
od
uo_chroma = UToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
vo_chroma = VToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(uo_chroma, vo_chroma)
MergeLuma(od)
od_filtered=last
interleave(ev_filtered,od_filtered)
assumefieldbased().assumetff().weave()

## Mounir's Colour Correction
#ConvertToRGB32(interlaced=true)
#LoadVirtualDubPlugin("VirtualDub\x86\plugins32\CMYK_rj.VDF", "CMYK", 0)
#CMYK(255,0,266,10,257,-24,255,0,0,42) # Modifying Yellow high tones (-24%) and magenta lows (4%)

ConvertToYV12()

LoadPlugin("DeGrainMedian64\DeGrainMedian.dll")
Import("kirsch.avsi")
aux=DeGrainMedian(mode=0, limity=7, interlaced=true)
denoised = KNLMeansCL(d=4,a=4,s=6,h=2.0,channels="Y",rclip=aux)
edgemask1 = ConvertToRGB32(interlaced=true).Kirsch().ConvertToYV12(interlaced=true,matrix="PC.601")
edgemask2 = edgemask1.mt_binarize(120).Blur(1.0).Blur(1.0)
mt_merge(last,denoised,edgemask2)

# Deinterlace
QTGMC(preset="Very Slow", FPSDivisor=2)

# End filtering
DeHalo_alpha_mt()
Grainfactory3mod(g1str=2,g2str=2,g3str=2)

# Shifting image up 1px and cropping (I should probably do the shift and crop all in RGB32 colour space actually)
ConvertToRGB32(matrix="Rec601", interlaced=false)
Crop(0,1,0,0)
AddBorders(0,0,0,1)
ConvertToYV12()
Crop(10, 2, -18, -2)
FillMargins(0,1,0,1) # I like to keep to mod4 resolutions, so this just makes the extra 2px less obvious

[Boulder]

Your QTGMC call is incorrect. The sample was not interlaced like tebasuna51 mentioned. Set inputtype=2 instead of FPSDivisor if you want to see how it works on deinterlaced (as that's what your sample looks like) content.

[samf88]

Hi,

I've adjusted it now, thanks.
http://www.framecompare.com/image-co...rison/77DPWNNX

[Boulder]

Originally Posted by samf88

Hi,

I've adjusted it now, thanks.
http://www.framecompare.com/image-co...rison/77DPWNNX

You could also try inputtype=3, I couldn't tell which one looks better but type 2 seems worse in that sample screenshot.

[samf88]

Just gave it a go with inputtype=3 (http://www.framecompare.com/image-co...rison/99J21NNU). Judging by the pattern on the red blouse alone I'd say it looks best treating it as interlaced (inputtype=0 and discarding the dupe frames using FPSDivisor). The stripey top and pants definitely look better with inputtype=3 though.