Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th January 2018, 18:03   #1301  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by tormento View Post
First of all, thanks.

I use it as prefilter with SMDegrain.

Any idea of why video output size changed?

From changelog, nothing gives hints.
Hi... size changed? More info?
__________________
github.com
Khanattila is offline   Reply With Quote
Old 29th January 2018, 18:34   #1302  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by Khanattila View Post
Hi... size changed? More info?
I am encoding a movie right now. After that I will give you some comparison.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 30th January 2018, 11:21   #1303  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Ok, first time I try this with some method. It could sound picky, but I am curious about deterministic influence of programming over a algorithm, that in theory should give always same results.

I used these clips that I already uploaded. The first one has a very low bitrate, the second one is full of movement.

First of all a very simple script:
PHP Code:
SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE"2)
SetFilterMTMode("ChangeFPS"3)
SetFilterMTMode("DGSource"3)
LoadPlugin("D:\eseguibili\media\DGDecNV\x64\DGDecodeNV.dll")
DGSource("E:\in\1_58 Hitman's bodyguard\hitman01.dgi")
ChangeFPS(last,last,true)
KNLMeansCL() 
Results:
Code:
hitman01_KNL-1.1.0.mkv 1.465.303
hitman01_KNL-1.1.1.mkv 1.464.935

hitman04_KNL-1.1.0.mkv 11.701.482
hitman04_KNL-1.1.1.mkv 11.694.121
Then I tried to use KNLMeansCL as a prefilter, with this script (and these modifications):
PHP Code:
SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE"2)
SetFilterMTMode("ChangeFPS"3)

SetFilterMTMode("DGSource"3)
LoadPlugin("D:\eseguibili\media\DGDecNV\x64\DGDecodeNV.dll")
DGSource("E:\in\1_58 Hitman's bodyguard\hitman04.dgi")

ChangeFPS(last,last,true)

SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,truemotion=true,plane=4,chroma=true,lsb=true,mode=0)

Prefetch(6
Results:
Code:
hitman04_SMD-1.1.0.mkv 7.828.657
hitman04_SMD-1.1.1.mkv 7.794.803
And the question is: why has output size changed?

Shouldn't be the applied algorithm always the same?
__________________
@turment on Telegram

Last edited by tormento; 30th January 2018 at 12:14.
tormento is offline   Reply With Quote
Old 30th January 2018, 11:34   #1304  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,752
Why are you surprised? Obviously, KNLMeansCL alone filters the video in a different way than SMDegrain as a whole.

And is it even a different clip? (hitman01 vs. hitman04)


BTW, an algorithm has no rhythm.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 30th January 2018, 12:13   #1305  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by LigH View Post
Why are you surprised? Obviously, KNLMeansCL alone filters the video in a different way than SMDegrain as a whole
Comparison is between two versions of KNLMeansCL about same clip.

Do you think I would really ask why two different filters have different output sizes?

P.S: Thanks for english corrections.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 30th January 2018, 15:08   #1306  |  Link
jmartinr
Registered User
 
jmartinr's Avatar
 
Join Date: Dec 2007
Location: Enschede, NL
Posts: 301
Quote:
Originally Posted by tormento View Post
Code:
hitman04_SMD-1.1.0.mkv 7.828.657
hitman04_SMD-1.1.1.mkv 7.794.803
And the question is: why has output size changed?

Shouldn't be the applied algorithm always the same?
7828657 / 7794803 = 1.0043

That's still a very small difference.
__________________
Roelofs Coaching
jmartinr is offline   Reply With Quote
Old 30th January 2018, 15:24   #1307  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by tormento View Post
Comparison is between two versions of KNLMeansCL about same clip.

Do you think I would really ask why two different filters have different output sizes?

P.S: Thanks for english corrections.
Different approximations in some calculations in the kernel code.

KNLMeansCL/commit
__________________
github.com
Khanattila is offline   Reply With Quote
Old 30th January 2018, 19:23   #1308  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by Khanattila View Post
Different approximations in some calculations in the kernel code.
That wasn't in changelog

Thanks.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 30th January 2018, 20:03   #1309  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,752
Sorry, tormento, I thought you compared another pair of files...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 30th January 2018, 21:23   #1310  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
What does this mean?


Code:
video=ConvertToYUV444(video).KNLMeansCL(d=1, a=2, s=4, h=4, device_type="CPU", device_id=0).ConvertToYUV420
Atak_Snajpera is offline   Reply With Quote
Old 30th January 2018, 22:00   #1311  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by Atak_Snajpera View Post
What does this mean?


Code:
video=ConvertToYUV444(video).KNLMeansCL(d=1, a=2, s=4, h=4, device_type="CPU", device_id=0).ConvertToYUV420
add channels="YUV"
__________________
github.com
Khanattila is offline   Reply With Quote
Old 30th January 2018, 22:15   #1312  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806

Code:
video=ConvertToYUV444(video).KNLMeansCL(d=1, a=2, s=4, h=4, device_type="CPU", device_id=0,channels="YUV").ConvertToYUV420
Why CPU device does not support this mode?
Atak_Snajpera is offline   Reply With Quote
Old 30th January 2018, 23:04   #1313  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by Atak_Snajpera View Post

Code:
video=ConvertToYUV444(video).KNLMeansCL(d=1, a=2, s=4, h=4, device_type="CPU", device_id=0,channels="YUV").ConvertToYUV420
Why CPU device does not support this mode?
Implementation-defined, however, it is difficult for the CPUs to support that mode.

The program is GPU oriented, and GPU work well with INT8, INT16, INT32, HALF and FLOAT.
So it makes sense not to support intermediate precision like INT9, INT12 or INT14.
However, there is a special format INT_101010, where the channels of each element (10 bits per channel) are packed into a single INT32.
Unfortunately, it is not very supported.
__________________
github.com

Last edited by Khanattila; 31st January 2018 at 09:23.
Khanattila is offline   Reply With Quote
Old 31st January 2018, 12:35   #1314  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Ok. No problem. I was just curious...
Atak_Snajpera is offline   Reply With Quote
Old 2nd February 2018, 10:51   #1315  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by Atak_Snajpera View Post
Ok. No problem. I was just curious...
You are welcome
__________________
github.com
Khanattila is offline   Reply With Quote
Old 12th February 2018, 15:41   #1316  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Hi all, need some help with KNLMeansCL on AMD RX Vega64:

denoised Y clip comes with broken data (mostly zero, very few times random output). This is a known issue (https://github.com/Khanattila/KNLMeansCL/issues/26) and we will await the patch.

Meanwhile, we would like to know whether using RGB is a safe workaround or not(assume we don't want to touch UV planes) :

Code:
a = "your file with YUV-8bit format.m2ts"
src8 = core.lsmas.LWLibavSource(a,threads=1)
src16 = core.fmtc.bitdepth(src8, bits=16)

Y = core.std.ShufflePlanes(src16, 0, vs.GRAY)
nr16y = core.resize.Bicubic(Y, format=vs.RGB48,matrix_in_s="709")
nr16y = core.knlm.KNLMeansCL(nr16y, d=1, a=2, s=3, h=1.3, channels="RGB", device_type="GPU")
nr16y = core.resize.Bicubic(nr16y, format=vs.GRAY16,matrix_s="709")

nr16y.set_output()
It looks OK for a few samples but we would like to know whether it is indeed unaffected by the same issue. Thanks for help!
littlepox is offline   Reply With Quote
Old 12th February 2018, 16:38   #1317  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by littlepox View Post
Hi all, need some help with KNLMeansCL on AMD RX Vega64:

denoised Y clip comes with broken data (mostly zero, very few times random output). This is a known issue (https://github.com/Khanattila/KNLMeansCL/issues/26) and we will await the patch.

Meanwhile, we would like to know whether using RGB is a safe workaround or not(assume we don't want to touch UV planes) :

Code:
a = "your file with YUV-8bit format.m2ts"
src8 = core.lsmas.LWLibavSource(a,threads=1)
src16 = core.fmtc.bitdepth(src8, bits=16)

Y = core.std.ShufflePlanes(src16, 0, vs.GRAY)
nr16y = core.resize.Bicubic(Y, format=vs.RGB48,matrix_in_s="709")
nr16y = core.knlm.KNLMeansCL(nr16y, d=1, a=2, s=3, h=1.3, channels="RGB", device_type="GPU")
nr16y = core.resize.Bicubic(nr16y, format=vs.GRAY16,matrix_s="709")

nr16y.set_output()
It looks OK for a few samples but we would like to know whether it is indeed unaffected by the same issue. Thanks for help!
Hi littlepox, thanks for your report.
The main problem is I do not have a Vega GPU. In the country where I live the cost is very high, over 800€!
I also tried AMD's CodeXL, but for now, I do not see something strange. Future investigations are required.

I do not know if the workaround you suggest works or not, there is a problem with the new AMD OpenCL compiler for Vega.
There are probably some optimizations in place that are not clear.
__________________
github.com

Last edited by Khanattila; 12th February 2018 at 16:42.
Khanattila is offline   Reply With Quote
Old 25th February 2018, 12:14   #1318  |  Link
fadedmaple
Registered User
 
fadedmaple's Avatar
 
Join Date: Dec 2017
Posts: 8
GPU use ratio is Low

Hi,i`m use KNLMeansCL in vapoursynth with GTX1060 .
The system is Win10 and driver 390.77.
I found the GPU performance can not be fully realized.

the vpy script like this

src = core.lsmas.LWLibavSource(r'E:\OB\MEET_THE_FOCKERS_HDCLUB\BDMV\STREAM\00372.m2ts')
src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="Y",device_type = "gpu" )
src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="UV",device_type = "gpu" )

Only 0.3 fps ,And take up 10%-30% GPU usage。
Is it unable to run out of GPU or is it my own problem?
fadedmaple is offline   Reply With Quote
Old 25th February 2018, 15:56   #1319  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by fadedmaple View Post
Hi,i`m use KNLMeansCL in vapoursynth with GTX1060 .
The system is Win10 and driver 390.77.
I found the GPU performance can not be fully realized.

the vpy script like this

src = core.lsmas.LWLibavSource(r'E:\OB\MEET_THE_FOCKERS_HDCLUB\BDMV\STREAM\00372.m2ts')
src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="Y",device_type = "gpu" )
src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="UV",device_type = "gpu" )

Only 0.3 fps ,And take up 10%-30% GPU usage。
Is it unable to run out of GPU or is it my own problem?
Hi fadedmaple, can you post the whole script?
The situation you describe is typical of a CPU-bottleneck.
__________________
github.com
Khanattila is offline   Reply With Quote
Old 25th February 2018, 16:57   #1320  |  Link
fadedmaple
Registered User
 
fadedmaple's Avatar
 
Join Date: Dec 2017
Posts: 8
Quote:
Originally Posted by Khanattila View Post
Hi fadedmaple, can you post the whole script?
The situation you describe is typical of a CPU-bottleneck.
here the whole script ,nothing special

Code:
from vapoursynth import core
import vapoursynth as vs

src = core.lsmas.LWLibavSource(r'E:\OB\MEET_THE_FOCKERS_HDCLUB\BDMV\STREAM\00372.m2ts')

src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="Y",device_type = "gpu" )
src = core.knlm.KNLMeansCL(src,d = 3,a = 6, h = 1.2,channels="UV",device_type = "gpu" )

src.set_output()
If i set the parametes as default (d = 1,a = 2),the x265 get a faster speed,even faster than no denoise.
So I think it really is using my GPU.

I use vseditor to benchmark ,when set d=3 a=6 ,CPU usage is very low,why CPU-bottleneck?

Last edited by fadedmaple; 25th February 2018 at 17:24.
fadedmaple is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:30.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.