req: advice on current denoise methods

[FredThompson]

It's been years since I did much of anything with AviSynth. I'm now starting a huge project of converting silver DVDs, TiVo captures, DV, and other old sources into x264 mkv. Source is usually NTSC but there is some PAL.

This is the script I used many, many years ago for general cleaning and prep:

AVISource("EM Mechanical Service Camera 2.avi")
LoadPlugin("DeGrainMedian.dll")
LoadPlugin("EEDI2.dll")
LoadPlugin("medianblur.dll")
LoadPlugin("mt_masktools.dll")
LoadPlugin("MVTools.dll")
LoadPlugin("ReduceFlicker.dll")
LoadPlugin("RemoveGrainS.dll")
LoadPlugin("RepairS.dll")
LoadPlugin("SSEToolsS.dll")
Import("DeHalo_alpha.avs")
Import("LRemoveDust.avs")
Import("MCBob.avs")
ConvertToYV12(Interlaced=True)
SeparateFields()
DeGrainMedian(limitY=2,limitUV=3,mode=3,interlaced=true)
evn=SelectEven().DeHalo_Alpha().LRemoveDust_YV12(10,1)
odd=SelectOdd().Dehalo_alpha().LRemoveDust_YV12(10,1)
Interleave(evn,odd)
weave()
GetParity() ? nop() : DoubleWeave().SelectOdd()

# Deinterlace

#MCBob().SelectEven()

Some of these functions go back to 2009, possibly earlier. It fit my needs then and would probably work ok now but updating to more capable methods is a goal.

I've started poking around various threads which discuss noise removal to get myself familiar with current methods.

The recent thread about "best of three" source frames looks interesting for TiVo TV captures I've made for which the top 2 files (by size) are kept. I'd been looking at various video quality metric methods, including Netflix's published stuff, in hopes of finding a way to create a summary number to state the quality of a source so newer captures could be compared to the number generated from a pre-encoded source just in case a "better" version is captured later. In the interest of time, I'll probably go with a "best of two" frames approach. Want to include some form of glitch detection as well (blockiness, partial frame, etc. - all that wonderful stuff that happens with OTA broadcasts...)

So, those being my current thoughts and research, would people who have been doing similar encodes recently please share helpful ideas about how to improve this script?

[Gser]

QTGMC is the newest version of MCBob. It is quite effective at deinterlacing and denoising
http://avisynth.nl/index.php/QTGMC

[johnmeyer]

There is no "one size fits all" script for improving video. Every video source is quite different, and the problems are not universal. Applying functions that are not needed for one particular source will do nothing but degrade it.

In addition, it looks to me like your script has a few flaws that would cause it to do wrong things, although others should look and see if I am wrong.

1. You separate fields, and then do a degrain process on that result. I don't know if that function is temporal, but if it is, it will be operating by comparing each field to its neighbor, and since you now have even fields next to odd fields, and since they are from different spatial positions, the result will be wrong. I would instead move two copies of that function down to the "evn" and "odd" lines.

2. DegrainMedia has interlaced=true, but the video it is operating on is no longer interlaced.

3. You are running Dehalo, but are all your sources exhibiting halos? If not, this will probably introduce them.

4. I actually ran your script to check things out, and the Doubleweave line doesn't do anything, at least not with BFF video. I suspect it might cause problems.

The actual noise reduction was not very good because it killed a lot of details on some random NTSC VHS capture I had lying around. And, that result was without applying your LRemoveDust plugin which I don't have and wasn't about to try to find and install, just to test this out. That will only fuzz it up some more.

I use MDegrain function found in MVTools2. I change the settings for each and every capture to "tune" it to the particular problems of that source.

[FredThompson]

Thanks for the feedback, guys.

@Gser,

I'm looking at QTGMC now. If a source was originally interlaced, I leave it interlaced. However, converting interlaced source to progressive at double the frame rate has been useful.

@johnmeyer,

You are correct, this is a baseline which I adapted as needed. I should have stated that. Thanks.

1 and 2) Good points. My intent was to create separate streams for the different fields and process each individually. Thanks for catching my mistakes.

3) Yes, this is a skeleton. If there was no halo, that line was marked as a comment.

4) Hmmm...it was supposed to pass TFF and convert BFF to TFF. I'll check it.

This was used only on digital sources. You are correct that analog, especially VHS, needs specific correction such as chroma shift.

I'll look at MDegrain.

[hello_hello]

My logic regarding de-interlacing is nothing does it better than QTGMC, so you might as well bake in the best quality de-interlacing instead of using lesser quality de-interlacing on playback, plus interlaced encoding increases the bitrate by around 15% for a given CRF value compared to progressive de-interlaced to double frame rate by QTGMC (if I remember correctly), but that's just me.....

Aside from using QTGMC for de-interlacing, try it as a noise filter for progressive video, or just to clean it up a bit. I use QTGMC for noise filtering more than I use it for de-interlacing.

QTGMC(InputType=1, Preset="Medium", EzDenoise=1.5)
Gradfun3()

For fun, when I first started playing with QTGMC for stabilising and denoising progressive video, I tried torture testing it, along with a few other Avisynth scripts as a comparison. Not that it's always the perfect solution, or a totally fair comparison as I only used the defaults for the other noise filters, but if you want to check the results. comparison encodes 2.zip (76MB)

[burfadel]

Fft3dfilter is a brilliant denoiser, and has excellent performance compared to other methods. You need to download the latest fftw and put it in your system32 folder. Only the fftw.dll file is needed. Download the appropriate 32/64 version here: http://www.fftw.org/install/windows.html

Latest version of fft3dfilter here: https://github.com/pinterf/fft3dfilter/releases

Example of decent setting:
fft3dfilter(sigma=2.5, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=0.4, plane=4, dehalo=1.6, wintype=1, smax=23)

Documentation is here: https://avisynth.org.ru/fft3dfilter/fft3dfilter.html
Only settings you'd really want to change is sigma (strength), sharpen and the sharpen limiter smax, and dehalo strength. Plane=4 means chroma is processed as well.

[ChaosKing]

There is also vapoursynth with 2 very nice filters:
- BM3D – Denoising filter using the BM3D algorithm
- KNLMeans – OpenCL Non-local means denoising

http://www.vapoursynth.com/doc/pluginlist.html

[cork_OS]

Originally Posted by hello_hello

Aside from using QTGMC for de-interlacing, try it as a noise filter for progressive video, or just to clean it up a bit. I use QTGMC for noise filtering more than I use it for de-interlacing.

QTGMC with InputType=1/2 is also great for antialiasing.

Originally Posted by ChaosKing

There is also vapoursynth with 2 very nice filters:
- BM3D – Denoising filter using the BM3D algorithm
- KNLMeans – OpenCL Non-local means denoising

KNLMeansCL is available for AviSynth.

[johnmeyer]

re: fft3dfilter

It is a pretty nifty filter for some things, and is rather unique in using an FFT transform to take the spatial information and put it into the "frequency" domain. However, it does not use information from adjacent frames and in many ways is similar to a regular spatial denoiser. It therefore is prone to losing details. It is a very good filter to use as a prefilter for temporal denoising. If you do use it by itself, I highly recommend to NOT use just the sigma value, but instead read the documentation and learn about Sigma1, Sigma2, etc. You will get far more control over what noise gets reduced, and what details can get preserved.

Personally, I usually come back to MDegrain2. However, if you are willing to use Vapoursynth, I have seen results from denoisers built only for that environment that look pretty darned good. Learning Vapoursynth is still on my to-do list for this year.

p.s. I just noticed the reference in the previous post to KNLMeansCL. I've played around with that, and it looks quite good. You should look at that as well.

[manono]

Originally Posted by johnmeyer

However, it does not use information from adjacent frames...

Ah, but it does. From the included doc:

So, it is a spatial-temporal (3D) filter, and works as follows:

Subsequent to the 'as follows' is more text about the 3-D and temporal qualities it has. However, you're right in that it does spatial cleaning and is prone to losing detail. If better compression is the goal - like for YouTube uploads, for example - then it can't be beat as far as I'm concerned. I use it often but you have to keep the Sigma settings under control.

[johnmeyer]

Originally Posted by manono

Ah, but it does [use adjacent frames]

I just went and read the doc, and you absolutely right.

However ...

... I am still not clear what it actually does with the information from neighboring frames. What's more, I looked for, but could not find, any setting for a scene change threshold. Most temporal plugins include this because otherwise you get weirdness (technical term) at scene transitions. The lack of that parameter, especially for a plugin that has been frequently revised over many years, makes me think that the adjacent frames may not contribute that much to the result.

But it is definitely doing something with adjacent frames, depending on the bt variable.

Speaking of the bt variable, note that bt=0 uses "all previous frames (switch Kalman filter mode)." This sounds to me more like an averaging algorithm, rather than the difference mode algorithms used by many temporal denoisers. In those, you motion estimate to get the adjacent frame to line up with the current frame, take a difference, and then use some algorithm to go after things that only appear in the difference.

But, I'm only guessing. I'm sure this has been discussed before in this forum, and the answer is known.

I know there are a few people here who don't like fft3dfilter, but I think it is a very good denoiser for certain situations. It just wouldn't be my first choice for denoising NTSC VHS video.

[feisty2]

Originally Posted by johnmeyer

I know there are a few people here who don't like fft3dfilter, but I think it is a very good denoiser for certain situations. It just wouldn't be my first choice for denoising NTSC VHS video.

cuz fft3d, along with dfttest are toxic against high frequency structures, if your video is pretty high quality (lots of high frequency delicate details), you'll see all kinds of shit with these 2 frequency domain denoisers

[manono]

Funny, I like DFTTest as well. But my sources aren't that high quality. VHS tapes and crummy field-blended garbage DVDs for the most part.

Yes, I doubt I'd use FFT3DFilter in a decent hi-def source.

[FredThompson]

Well, y'all just drifted into another area I'm exploring. Will be doing some VHS and S-VHS capture using a pro editing deck and an osprey card. Also have quite a bit of 60 fps progressive 1080p camcorder source. FWIW, the NTSC DV will be encoded at 4:2:2. Looking at results of various DV decoders now. Used Cedocida many years ago. Saw the infamous HD chroma bleed tonight. Forgot how horrible that could be...

If something is interlaced, I keep it that way. As with most

[burfadel]

Originally Posted by manono

Yes, I doubt I'd use FFT3DFilter in a decent hi-def source.

Very true, when I said fft3dfilter was brilliant I didn't mean it should be used on something that is already of excellent quality to begin with. If you can make out high frequency, fine detail, amongst any noise, is any denoiser really necessary? It seems the only denoisers that may be advantageous there are too slow considering the outcome. Not the filters fault, just most aren't specifically designed for fast noise removal in a mostly clean, high-def source.

Going back to fft3dfilter, yes you can set different settings etc depending on the source. It is spatio-temporal with the bt=5 setting.

An idea for the ideal noise filter is you could make a difference between the source and the fft3dfilter content that would just show the stuff removed containing both detail and noise. From here, you could use a very simplified version of smdegrain or whatever to remove the temporal noise changes, supposedly leaving those high frequency fine details intact including any sharpening etc by fft3dfilter. Scenechanges, if they aren't handled by fft3dfilter, will show as a massive change, but this isn't an issue and should be ignored by the simplified smdegrain filter, still only removing the temporal noise. What you have left is the fft3dfiltered content and the simple smdegrain (or whatever) filtered difference clip. This filtered difference clip should then be put back on to the fft3dfiltered content.

Fft3dflter will do the heavy lifting and the simplified smdegrain filter (or alternative) just has to work on a very basic picture containing just the changes between the filtered and source clips. Noise can be very easily isolated from this basic clip, and by adding what's left back to the fft3dfiltered content you have the ideal denoise situation. This difference noise filter should be very fast and as simple as practical without affecting results. The key thing with this principle is speed, things like KnlmeansCL is much slower than pinterf's recent fft3dfilter, as is smdegrain etc. Also parallell encoding with KnlmeansCL can be unreliable and stall, whereas fft3dfilter you can run multiple encodes in parallel. I have found fft3dfilter to also be very fast. The noise filtering difference if done right should only have a very small affect on the speed.

With fft3dfilter, there's an option interlaced=1 which separates fields before processing. If you are using nnedi3 etc for deinterlacing before anything else you could separatefields first, selectodd or even fields (whichever is of nicer quality), process that field, which is great since it's only have the resolution so is fast! and then use nnedi3(dh=true) to restore original dimensions. This should also give nicer quality than using nnedi3 or another deinterlacer first. There's several possibilities for a filter wrapper when you consider all of this, but like I said, the focus needs to be on speed as much as it is on quality, especially if the speed drop for a HQ source is from 5 fps encoding to .86 fps encoding with you having to take a screenshot of a particular frame, zooming it, and squinting to say 'oh look, there's a grain of a different luma there'. If you really want to keep that kind of functionality, you simply need to make a placebo preset.

[feisty2]

Originally Posted by burfadel

Very true, when I said fft3dfilter was brilliant I didn't mean it should be used on something that is already of excellent quality to begin with. If you can make out high frequency, fine detail, amongst any noise, is any denoiser really necessary? It seems the only denoisers that may be advantageous there are too slow considering the outcome. Not the filters fault, just most aren't specifically designed for fast noise removal in a mostly clean, high-def source.

Going back to fft3dfilter, yes you can set different settings etc depending on the source. It is spatio-temporal with the bt=5 setting.

An idea for the ideal noise filter is you could make a difference between the source and the fft3dfilter content that would just show the stuff removed containing both detail and noise. From here, you could use a very simplified version of smdegrain or whatever to remove the temporal noise changes, supposedly leaving those high frequency fine details intact including any sharpening etc by fft3dfilter. Scenechanges, if they aren't handled by fft3dfilter, will show as a massive change, but this isn't an issue and should be ignored by the simplified smdegrain filter, still only removing the temporal noise. What you have left is the fft3dfiltered content and the simple smdegrain (or whatever) filtered difference clip. This filtered difference clip should then be put back on to the fft3dfiltered content.

Fft3dflter will do the heavy lifting and the simplified smdegrain filter (or alternative) just has to work on a very basic picture containing just the changes between the filtered and source clips. Noise can be very easily isolated from this basic clip, and by adding what's left back to the fft3dfiltered content you have the ideal denoise situation. This difference noise filter should be very fast and as simple as practical without affecting results. The key thing with this principle is speed, things like KnlmeansCL is much slower than pinterf's recent fft3dfilter, as is smdegrain etc. Also parallell encoding with KnlmeansCL can be unreliable and stall, whereas fft3dfilter you can run multiple encodes in parallel. I have found fft3dfilter to also be very fast. The noise filtering difference if done right should only have a very small affect on the speed.

With fft3dfilter, there's an option interlaced=1 which separates fields before processing. If you are using nnedi3 etc for deinterlacing before anything else you could separatefields first, selectodd or even fields (whichever is of nicer quality), process that field, which is great since it's only have the resolution so is fast! and then use nnedi3(dh=true) to restore original dimensions. This should also give nicer quality than using nnedi3 or another deinterlacer first. There's several possibilities for a filter wrapper when you consider all of this, but like I said, the focus needs to be on speed as much as it is on quality, especially if the speed drop for a HQ source is from 5 fps encoding to .86 fps encoding with you having to take a screenshot of a particular frame, zooming it, and squinting to say 'oh look, there's a grain of a different luma there'. If you really want to keep that kind of functionality, you simply need to make a placebo preset.

man, you need a better video card, GTX1080 maybe...
and removegrain is even faster than fft3d, use that instead, I don't really get this weird OCD about speed, when we talk about denoisers, we talk about the denoising quality, not the processing speed, cuz it could vary significantly among different implementations of the same algorithm...

noisy = last
clean = last.removegrain(11)
clean.repair(noisy, 1)

IT'S FAAASSST, bro!!

[burfadel]

I have a RX 480. Although not as fast as a GTX 1080 obviously, it's more than capable of running KNLMeansCL. The above example I gave processes luma and chroma, if you want to do just luma you drop the plane=4 option. KNLMeansCL operates just on luma channel as well ordinarily, to work on chroma you need to use channels="YUV", but first have the script converted to yv24. As I said in regards to KNLMeansCL, it doesn't really like running multi instances in parallel. If you have a high thread count CPU such as an 8C/16T, to saturate the CPU you really need to be doing parallel encoding. This is useful if you have multiple jobs, or one job on a UHD clip where you can split the clip, process in parallel, and append. I've seen before where people use very slow scripts that might produce good results but then they don't back it up with the highest quality settings in the encoder, which you need to do to make use of the quality. Another issue with KNLMeansCL is that it's hugely inefficient with discrete cards, since the data has to be constantly transferred to the GPU, processed, and transferred back. It much better suits where you have an APU where the inefficiency is greatly reduced.

Regardless of what some people think, speed is importantant.

A very basic example of the concept is:

orig=last
filt=fft3dfilter(wintype=1,bt=5,bw=32,bh=32,ow=16,oh=16,sharpen=0.5,ncpu=4,plane=4,dehalo=2,sigma=3,smax=23)
diff=mt_makediff(orig,filt)
filtdiff=repair(fluxsmootht(diff),diff,mode=12)
mt_adddiff(filtdiff,filt)

To add normal, non-telecined deinterlacing to it and be faster:

separatefields().selecteven()
orig=last
filt=fft3dfilter(wintype=1,bt=5,bw=32,bh=32,ow=16,oh=16,sharpen=0.5,ncpu=4,plane=4,dehalo=2,sigma=3,smax=23)
diff=mt_makediff(orig,filt)
filtdiff=repair(fluxsmootht(diff),diff,mode=12)
mt_adddiff(filtdiff,filt)
nnedi3(dh=true)

Of course, prefetch=4 in nnedi3 if you have set up prefetch in the script. Prefetch(4) seems to be a decent option, it seems to have diminishing returns if you set it too high.

[feisty2]

Originally Posted by burfadel

Regardless of what some people think, speed is importantant.

then you should really use removegrain

Originally Posted by burfadel

A very basic example of the concept is:

orig=last
filt1=fft3dfilter(wintype=1,bt=5,bw=32,bh=32,ow=16,oh=16,sharpen=0.5,ncpu=4,plane=4,dehalo=2,sigma=3,smax=23)
diff=mt_makediff(orig,filt1)
filtdiff=repair(fluxsmootht(diff),diff,mode=12)
mt_adddiff(filtdiff,filt1)

To add normal, non-telecined deinterlacing to it and be faster:

separatefields().selecteven()
orig=last
filt1=fft3dfilter(wintype=1,bt=5,bw=32,bh=32,ow=16,oh=16,sharpen=0.5,ncpu=4,plane=4,dehalo=2,sigma=3,smax=23)
diff=mt_makediff(orig,filt1)
filtdiff=repair(fluxsmootht(diff),diff,mode=12)
mt_adddiff(filtdiff,filt1)
nnedi3(dh=true)

if you're determined to use a frequency denoiser, dfttest or fft3d, the best you can do is avoid high frequency filtering completely, dfttest got "sstring" for that kinda stuff, something like sstring = "0.0:8.0 0.5:4.0 0.75:0.0 1.0:0.0" might be nice for frequency filters, then you use some other algorithms like NLMeans that are good for high frequency components, then you merge the filtered low frequency components and high frequency components back together, it's better off than mess the high frequencies up then try to refine them

[burfadel]

Using removegrain wouldn't make sense becuase it's spatial. KNLMeansCL and fft3dfilter (with bt=5) are spatio-temporal. The example I gave is both fast and effective. The fluxsmootht is there as a concept thing, it can be replaced with something more effective. Doing something like the example I gave is perfectly fine. In fact, most fancy scripts and even filters do a similar thing. If you got rid of any script or plugin that doesn't use masktools, repair, or soothe function you really will have a very bare script.

The concept you gave is still doing a similar thing, you are using two different filters and combining the results. The example I gave does basically the same thing, just using a different method (difference filtering). You could even replace fft3dfilter with dfttest, or even knlmeanscl! However since you are only working on the temporal noise component in adding it back in, these will probably be left effective.

[feisty2]

that's not basically the same thing...
they might sound similar but the results are very different, both methods are used in Oyster (something I developed to remove compression artifacts), but for different purposes.. (hint: frequency separation could be mostly perfect, refining could hardly be)