Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
28th August 2017, 19:58 | #81 | Link |
Registered User
Join Date: Mar 2002
Location: Krautland
Posts: 903
|
@ burfadel:
1.:I really like your script! 2.:Bad news. Your first version (1.1) of the function is working flawless on my side. The new versions (1.2+1.3) getting stucked at line 54 (outbits). ------------------------------------ Avisynth 2.60 MT All necessary plugins uptodate. YV12 8bit video. ----------------------------------- Can you give me a hint for troubleshooting? |
28th August 2017, 20:07 | #82 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
His script is designed for AVS+ only.
If you want to use with Avisynth 2.6, you need to remove ConvertBits, and you will only process in 8-bit, which will considerably degrade the results because of applying a series of actions each having a subtle effect. The rounding errors will add up. |
29th August 2017, 03:19 | #83 | Link | |
Registered User
Join Date: Sep 2006
Posts: 1,657
|
Quote:
|
|
29th August 2017, 03:51 | #85 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Burdafel, you did the bitrate conversion wrong. FF3DFilter and SMDegrain were unnecessarily processing in HBD resulting in poor performance. Here I fixed your script.
Testing back my first video, thSAD is good at 450 (difference between 450 and 550 is extremely minimal). Here I've set enh=13 and rn=10, but it will take more tests on various sources. But so far, I wouldn't go higher on low-noise source, and sources with high noise can't take any more sharpening. The only case I can see where I'd set those higher are for videos that are a little blurry. Code:
# mClean spatio/temporal denoiser # Version: 1.3 (27 August 2017) # By burfadel # +++ Description +++ # This script is intended to remove noise whilst retaining as much detail as possible. # Typical spatial filters work by removing large variations in the image on a small scale, reducing noise but also making the image less # sharp or temporally stable. # mClean works primarily in the temporal domain, although there is some spatial limiting. # Chroma is processed via a different method to luma for optimal results. # Input must be # +++ Artifacts +++ # Spatial picture artifacts may remain as removing them is a fine balance between removing the unwanted artifact whilst not removing detail # Additional dering/dehalo/deblock filters may be required, but should ONLY be uses if required due the detail lose/artifact removal balance # A deblocking filter such as Deblock_QED can be applied after, as MClean currently does not deblock; this may be provided as an option later # +++ Sharpening +++ # Additional sharpening filters may not be required, mClean does some light detail enhancement. Any additional sharpening filters may require # a little less strength. Alternatively use a higher 'enh' setting. Range of normal sharpening is 0-100. There are two extra options, 101 and # 102. These are for 'overboost' sharpening, suitable only for high quality, high resolution sources. Overboost sharpening requires the modplus # plugin, this is only required if overboost is used. # +++ ReNoise +++ # ReNoise adds back some of the removed luma noise. Re-adding original noise would be counterproductive, therefore ReNoise modifies this noise # both spatially and temporally. The result of this modification is the noise becomes much nicer and it's impact on compressibility is greatly # reduced. It is not applied on areas where the sharpening occurs as that would be counterproductive. Settings range from 1 to 20, default # value is 12. The strength of renoise is affected by the the amount of original noise removed and how this noise varies between frames. # +++ Outbits +++ # Specifies the bits per component (bpc) for the output for processing by additional filters. It will also be the bpc that mClean will process. # By default, mClean processes as 12 bits if the input is 8 bit, and converts back to 8 bit. If the input is 10 bits or higher no conversion is # done unless outbits is specified and is different to the input bpc. If you output at a higher bpc keep in mind that there may be limitations # to what subsequent filters and the encoder may support. # +++ Required plugins +++ # Latest RGTools, MVTools2, Masktools2, FFT3DFilter # Latest Modplus, only required if using sharpening overboost (enh settings 101 and 102) # Requires latest fftw.dll to be installed as instructed on the website - http://www.fftw.org function mClean(clip c, int "thSAD", int "blksize", int "blksizeV", int "overlap", int "overlapV", int "enh", int "rn", int "outbits", int "cpu") { defH = Max (C.Height, C.Width/4*3) # Resolution calculation for auto blksize settings thSAD = Default (thSAD, 450) # Denoising threshold blksize = Default (blksize, defH<360 ? 8 : defH<750 ? 12 : defH<1200 ? 16 : defH<1600 ? 24 : 32) # Horizontal block size for MDegrain2 blksizeV = Default (blksizeV, blksize) # Vertical block size for MDegrain2, default same as horizontal overlap = Default (overlap, blksize>4?(blksize/4+1)/2*2:0) # Horizontal block overlap overlapV = Default (overlapV, blksize>4?(blksizeV/4+1)/2*2:0) # Vertical block overlap enh = Default (enh, 13) # Detail enhancement (detail orientated sharpen) strength rn = Default (rn, 10) # ReNoise strength from 0 (disabled) to 20 outbits = Default (outbits, c.BitsPerComponent) # Output bits, default input depth calcbits = c.BitsPerComponent == 8 ? 12 : c.BitsPerComponent calcbits = outbits > calcbits ? calcbits : outbits cpu = Default (cpu, 4) # Threads for fft3dfilter Assert(isYUV(c)==true, """mClean: Supports only YUV formats (YV12, YV16, YV24)""") Assert(isYUY2(c)==false, """mClean: Supports only YUV formats (YV12, YV16, YV24)""") Assert(isYV411(c)==false, """mClean: Supports only YUV formats (YV12, YV16, YV24)""") Assert(enh>=0 && enh<=102, """mClean: "enh" ranges from 0 to 102""") Assert(rn>=0 && rn<=20, """mClean: "rn" ranges from 0 to 20""") Assert(outbits>=8 && outbits<=16, """mClean: "outbits" ranges from 8 to 16""") # Spatio/temporal chroma noise filter filt_chroma = fft3dfilter (c, bw=blksize*2, bh=blksizeV*2, ow=overlap*2, oh=overlapV*2, sharpen=0.12, bt=3, ncpu=cpu, dehalo=0.3, sigma=2.35, plane=3) # Temporal luma noise filter super = c.MSuper (chroma=false,hpad=16, vpad=16) bvec2 = MAnalyse (super, chroma=false, isb = true, delta = 2, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=5) bvec1 = MAnalyse (super, chroma=false, isb = true, delta = 1, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=3) fvec1 = MAnalyse (super, chroma=false, isb = false, delta = 1, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=3) fvec2 = MAnalyse (super, chroma=false, isb = false, delta = 2, blksize=blksize, blksizeV=blksizeV, overlap=overlap, overlapV=overlapV, search=5, searchparam=5) clean = c.MDegrain2 (super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, plane = 0) clean = calcbits != clean.BitsPerComponent ? clean.ConvertBits(calcbits) : clean c = calcbits != c.BitsPerComponent ? c.ConvertBits(calcbits) : c # Masks for spatial noise reduction and noise independent detail enhancement noised = mt_makediff (clean, c, u=1, v=1) noise = mt_binarize (clense(mt_makediff(mt_binarize(noised, u=1, v=1), mt_edge(sharpen(clean, 0.85), "prewitt", u=1, v=1), u=1, v=1), grey=true), u=1, v=1) # Spatial luma denoising clean2 = mt_merge (clean, removegrain(clean, 18, modeU=-1, modeV=-1), noise, u=1, v=1) # Unsharp filter for spatial detail enhancement clsharp = (enh>0<=100) ? mt_adddiff (mt_makediff(clean, blur(clean, 0.80*(enh/100), 0.50*(enh/100)), u=1, v=1), clean2, u=1, v=1) : clean clsharp = (enh>=101<=102) ? mt_adddiff (mt_makediff(clean, gblur(clean, enh-100), u=1, v=1), clean2, u=1, v=1) : clsharp # If selected, combining ReNoise renoise = (rn==0) ? nop : tweak(temporalsoften (noised, 4, 128, 0, scenechange=0, mode=2), cont=1.010+(0.020*(rn/20)), bright=1.01+(0.04*(rn/20))) clean2 = (rn>0<=20) ? mergeluma (clean2, mt_adddiff(clean2, renoise, u=1, v=1), 0.3+(rn*0.035)) : clean2 # Combining spatial detail enhancement with spatial noise reduction using prepared mask filt_luma = mt_merge (clean2, clsharp, mt_invert(mt_convolution(noise, u=1, v=1), u=1, v=1), u=1, v=1) # Converting bits per channel filt_luma = outbits < filt_luma.BitsPerComponent ? ConvertBits(filt_luma, outbits, 1) : filt_luma # Combining result of luma and chroma cleaning return mergechroma (filt_luma, filt_chroma) } |
29th August 2017, 03:57 | #87 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Quote:
The added grain won't make it to the output to benefit the encoder. This explains why I saw improvement in quality but no difference in file size.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 29th August 2017 at 05:48. |
|
29th August 2017, 04:08 | #88 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Quote:
Oddly enough, first screenshot has 0 difference between the 2 settings. 2nd screenshot however has considerable difference and doesn't look good with enh=20 I was about to upload 3 screenshots but the 3rd has files missing after upload, perhaps because postimage detected some images as being exactly the same. |
|
29th August 2017, 04:26 | #89 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Here are some tests on another source that definitely has more noise
KARA - Dazzling Red - Original - RemoveGrain(21) - KnlMeansCL(D=2, A=2, h=1.8, channels="YUV") # raised from 1.4 to 1.8 here - mClean(450, enh=12, rn=10) - mClean(450, enh=20, rn=12) Here we see more clearly the characteristics of each method. RemoveGrain does little, KNLMeans blurs out details, and mClean does a pretty good job in both cases.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 29th August 2017 at 04:29. |
29th August 2017, 06:54 | #90 | Link | ||
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,079
|
Quote:
If you do get better results with mClean instead of RemoveGrain, then what is the major difference between the two? Quote:
Cheers manolito |
||
29th August 2017, 07:37 | #91 | Link | ||
Registered User
Join Date: Sep 2006
Posts: 1,657
|
Quote:
First of all, your video has no grain, so the smdegrain inside mclean is basically doing nothing except creating artifacts on motion objects by its nature, you can see the artifacts in the fog which doesn't exist in the original. Then what you have left in the script is a simple removegrain(18). So right now you're basically comparing a removegrain(21) against KNLMeansCL and a removegrain(18), and you're saying removegrain(18) wins, I am speechless. Quote:
I have a feeling that you're just trolling the OP with all these nonsense. |
||
29th August 2017, 08:31 | #92 | Link | |
Registered User
Join Date: Aug 2006
Posts: 2,229
|
Quote:
Do you think there should be an auto adjustment for default enh and rn values based on resolution like there is for blocksize? Hopefully people won't have too many low quality 720P and 1080P videos. |
|
29th August 2017, 17:30 | #93 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
There's a big difference between a denoiser and a degrainer. I'm testing this as a denoiser. Noise is generally encoding artifacts. Grain is something else.
Quote:
Here's my feedback/observations so far 1. I like the results of this script 2. When it comes to using it as a prefilter for FRC, it does bring some benefits: better motion vectors 3. The re-added grain to help the encoder isn't one of those benefits When it comes to FRC with preset="slower", the performance of mClean isn't really an issue, but perhaps not all parts of it are necessary in that case, it has to be tested. Does renoise contribute to better motion vectors at all? My suspicion is that motion vectors are better defined for having more clearly-defined edges. The main issue is that it's hard to do each test because it can't yet be run in the same script. |
|
29th August 2017, 17:52 | #95 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Does it really make a difference on textures or are my eyes bad? Had to test for sure. I took an image of a full head and compared RemoveGrain(21) with mClean as a FRC prefilter. Then I put it in Photoshop, do a diff, use magic wand to select all with tolerence set to 0, 1 and 2.
There *IS* a difference. Not sure I can explain it though. This is a mostly static scene, differences are larger on high-animation scenes. |
29th August 2017, 18:06 | #96 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
This test shows clear differences
- No prefilter - RemoveGrain(18) - RemoveGrain(21) - TTempSmooth - mClean(rn=10) - mClean(rn=0) TempSmooth gives no gain and in fact degrades picture. mClean wins. RemoveGrain on its own... 21 is more balance than 18, but within mClean 18 works. Yes, renoise helps. You can't "add things back" after interpolation because source and destination frames count don't match I think I've settled the case.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 29th August 2017 at 23:53. |
29th August 2017, 21:50 | #98 | Link | |||
Registered User
Join Date: Sep 2006
Posts: 1,657
|
Quote:
Quote:
Quote:
You're not helping the OP in any way, such that you tested his denoising script on videos that have no noise at all, you gave him false feedback on setting comparison, you exaggerated invisible differences that photoshop can't even see, and you hijacked his thread and forced his script onto your own little filter, which in itself is a big confusing mess that doesn't make sense. |
|||
29th August 2017, 23:52 | #99 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Watch your language. If you don't like what we're doing, just get the heck out.
Quote:
Please, if you got nothing constructive to say, shut up. Edit: I renamed it to "no prefilter" to avoid confusing anyone else If someone else views any screenshot other than mClean(rn=10) to give better results, I'm listening, but please have something to back up what you say. This test is pretty simple: comparing 6 versions of a script to see which gives better results. I don't care about how "things should be done", I only care about concrete results.
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 30th August 2017 at 00:01. |
|
29th August 2017, 23:57 | #100 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Guys, please be nice.
Both of you are valued members of the community and both totally respected, if something aint working quite right, then maybe in time it will, it dont need no negs. Consider this a good slap on the arse for both (Soon be Christmas )
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
Tags |
cleaning, denoise, denoiser, mclean |
|
|