Introducing Zopti (ex AvisynthOptimizer)

[Dogway]

Originally Posted by zorr

When doing denoising you need to make a clip with added noise, then remove the noise and finally compare the denoised clip with the original. See here for an example. I have thought about other ways to do it but this is currently the most reliable way.

I see, that makes more sense although it kind of becomes a synthetic test since normally we don't have a ground truth to compare to. I'll be doing grain modeling with GrainFactory instead since it resembles more film grain.

I thought on using the dynamic mode because if I only use iterations I felt it was being stucked into some ranges.
Leaving it at low iterations and 5 run cycles it would get out of said ranges but now it would totally ignore past cycle logs which is undesired. I will try to do some other tests with low cycles and high iterations.

[Dogway]

Following your indications and still get the least-change variables.

Best result:

smdegrain(tr=1,thSAD=100,contrasharp=0,prefilter=0,blksize=32,overlap=12,divide=0)

Not sure if I'm doing something wrong. If you need my scripts please tell me and I upload them.

[zorr]

Originally Posted by Dogway

Not sure if I'm doing something wrong. If you need my scripts please tell me and I upload them.

Yes that's a good idea. I will take a look.

[EDIT] I have a suspicion of what the problem is. If your video is very grainy then the degraining process cannot remove a lot of noise in order to look like the original. This add noise -> remove noise -> compare only works well when the original video doesn't have much (or any) noise. You could try finding a part of the video where there is very little noise or perhaps even use a completely different video (with very little noise but similar content and movement) as the source in the optimization.

[Dogway]

Here's the link with the 3 clips. A slightly modified version of smdegrain to test contrasharp and divide. Prefilter has a few dependencies so you might want to comment it.

[zorr]

Originally Posted by Dogway

Here's the link with the 3 clips.

Thanks. I don't know why but my 7zip says it doesn't support the compression method. I downloaded the latest version but apparently it needs a reboot in order to install and I'm in the middle of optimization process so I don't want to do it now. Can you provide the files in another zip format?

[StainlessS]

Zorr, My version 7zip from April 2018 opens Dogway zip no problem, yours must be reasonably old version.

Latest update 2 days ago, 2018 Dec 30:- https://www.7-zip.org/download.html

No reboot required here (x64 version).
I think about 12 -> 18 months ago, some fancy new algo was added to 7zip.

For 7zip using zip archive, should perhaps set to Normal compression,
and whatever you wish for 7z archives.

[zorr]

Originally Posted by Dogway

Here's the link with the 3 clips. A slightly modified version of smdegrain to test contrasharp and divide.

Is your modified version uploaded somewhere? Looks like the latest released SMDegrain version (v3.1.2.100s) doesn't have the divide argument. And it would be a good idea to use the exact same function anyway so we can compare results.

The source video doesn't have any grain so it should work well...

I'm going to have to delay the testing a bit because I also need to update MVTools2 and the optimization task I'm running uses an older version. I don't want to do anything radical while it's running... should be done tomorrow.

[zorr]

Originally Posted by ChaosKing

I would be nice if we would have also halo and "other crap" simulators to easily test (optimize) other kinds of filters.

The halo removal is something I have thought about. Using a synthetic test is probably not useful there because the effect is unique in every video so it needs customized parameters.

The way I would do it: take a single frame (or even a small part of the frame) where the halo effect is clearly visible and use Photoshop or whatever to remove it manually (using the clone stamp tool for example). Then use a halo removal filter in AviSynth and optimize its parameters so that it replicates the result. The same parameters should then work well for the whole video.

[Dogway]

Here it is, in the rush I forgot to include it. This is not a release or anything, I modified it a bit to test divide and integer type data for contrasharpening (the generic "val" data type wasn't suited for the variable). I skipped prefilter=3 because I can't make dfttest work on my end. The denoised version was denoised with: smdegrain(tr=3,thSAD=250,contrasharp=40,prefilter=2,blksize=16,overlap=8,divide=0,refinemotion=true,lsb=true). Which is (maybe?) a good starting point for grainy 1080p, in lack of better denoisers for avs+ like BM3D.

Whenever you have time is ok, I use the mvtools2 from pinterf github.

[zorr]

Originally Posted by Dogway

I see, that makes more sense although it kind of becomes a synthetic test since normally we don't have a ground truth to compare to.

I agree that it's a synthetic test. But there's something you can do to improve SMDegrain that doesn't rely on such a synthetic test. Since SMDegrain relies on MVTools you can optimize its arguments using motion compensation (MAnalyze + MCompensate). If the motion compensation is high quality then SMDegrain should benefit from that also.

Does anyone know whether MDeGrain uses pixel-based motion compensation (like MFlowInter) or the block based (like MCompensate)? I think it's the latter but I'm not sure.

Another open question: if you find good parameters for MCompensate (or MFlowInter) do they work equally well with MDeGrain? Even if they're not similar there's the MAnalyze (and MRecalculate) part which should work in identical way for all use cases.

[ChaosKing]

Originally Posted by zorr

The way I would do it: take a single frame (or even a small part of the frame) where the halo effect is clearly visible and use Photoshop or whatever to remove it manually (using the clone stamp tool for example). Then use a halo removal filter in AviSynth and optimize its parameters so that it replicates the result. The same parameters should then work well for the whole video.

Ahhh why didn't I think of what! And testing one-two frame(s) should be enough

[Dogway]

Yes I dabbled with MCompensate back then, we used it for all kinds of filtering I guess I wanted to keep things simple and fast at that time but I had some beta version with it included. I want to run some tests with avsoptimizer and see where it takes me.
I think it uses MCompensate internally, you can see the vectors with MMask. In my opinion MFlowInter artifacts are less forgivable but I have been long out of the loop so things might have changed.

[Boulder]

Regarding denoising, do you have any plans for the Vapoursynth support that was briefly discussed when the optimization method was published?

[zorr]

Originally Posted by Dogway

This is not a release or anything, I modified it a bit to test divide and integer type data for contrasharpening (the generic "val" data type wasn't suited for the variable).
...
Whenever you have time is ok, I use the mvtools2 from pinterf github.

All right, figured out the problem. In the script you're comparing the denoised clip to the noisy clip when you should compare it to the original (without added noise). It was trying hard to come up with parameters that don't remove any noise.

So just add a line to load the original clip:
orig=ffvideosource("source.avi")

and then calculate SSIM between the original and denoised. Remove
global ssim = SSIM_FRAME(source, denoised)
and replace with
global ssim = SSIM_FRAME(orig, denoised)

I'm running a small test with such modifications and seems like it prefers tr=4 (as it should).

[zorr]

Originally Posted by StainlessS

Zorr, My version 7zip from April 2018 opens Dogway zip no problem, yours must be reasonably old version.

Still couldn't open the 7zip after the update. Perhaps the package was corrupted during download. The .zip opened without problems.

[zorr]

Originally Posted by Boulder

Regarding denoising, do you have any plans for the Vapoursynth support that was briefly discussed when the optimization method was published?

I haven't looked at Vapoursynth yet. I know almost nothing about it so it's hard to say how much effort it would take. Can you recommend a tutorial / site to get one started with VapourSynth?

[ChaosKing]

The official site and for a quickstart my portable pack https://forum.doom9.org/showthread.php?t=175529 <- there are some bat ecnode and VS editor examples
It's very similar to avisynth.

[Dogway]

yes, that makes more sense. I guess I got confused since adding synthetic grain seems counterintuitive... off to some tests.

[StainlessS]

EDIT:

Can you recommend a tutorial / site

Perhaps a succinct Python3 intro would be easiest way into VS,
here "Learn X in Y Minutes" for Python3:- https://learnxinyminutes.com/docs/python3/
[already linked on Vapoursynth home site]

EDIT: Python 2 is somewhat different to Python 3, so give tutorials on that a wide berth.

[ChaosKing]

Here are some basics:

from vapoursynth import core
import somescript as xy


clip = core.d2v.Source(r'E:\test.d2v') # r means raw string, so u can copy paste a windows path. Alternatively use (without r) 'E:\\test.d2v'
video = clip

# call a function from our imported script
clip = xy.Filter(clip)
clip = somescript.Filter(clip) #alternative

# call a plugin function. All plugins have a namespace.
clip = core.dfttest.DFTTest(clip, sigma=5)
clip = clip.dfttest.DFTTest(sigma=5) #alternative

#compare
clip = core.std.StackHorizontal([video.text.Text("original"), clip])		
clip.set_output()

EDIT
Oh and it would be also very easy to run Avisynth plugins/scripts within VS

clip = core.d2v.Source(r'E:\test.d2v') 
clip_distorted = core.avsw.Eval('FineDehalo(darkstr =0)', clips=[clip], clip_names=["last"], slave_log=r"D:\debug.txt",  avisynth =r"D:\AVSPLUS_x86\Avisynth.dll")

vmaf.VMAF(clip_reference, clip_distorted) # https://github.com/HomeOfVapourSynthEvolution/VapourSynth-VMAF