Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd June 2016, 12:31   #1801  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,256
Quote:
Originally Posted by Reel.Deel View Post
I started making some changes to the documentation and need some opinions. Here's the difference between the two: http://diff.pics/JbQx0fvGOTeM/1

I added a syntax and parameters section and I also highlighted the syntax and listed all the parameters and their descriptions. I would like to do this to all the internal filters. Yea or nay?
Yes. And a big thanks for dealing with the documentation which is usually the last one in the task queue, at least for the programmers point of view.
pinterf is offline   Reply With Quote
Old 23rd June 2016, 12:33   #1802  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,256
Quote:
Originally Posted by tormento View Post
Post edited and DGIs inserted.

Please look original thread and see all the trials I did.

Will try to strip down things. As far as I remember, I could even use prefilter 2 instead of 4 or x86 vs x64 and nothing changed.
Thank you for saving time for me. I run too much things parallel nowadays and each one is a challenge for me.
pinterf is offline   Reply With Quote
Old 23rd June 2016, 13:55   #1803  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by pinterf View Post
Thank you for saving time for me. I run too much things parallel nowadays and each one is a challenge for me.
Done encoding with any prefilter. Error occurs only with 4.

Uploaded resulting files.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 23rd June 2016, 16:49   #1804  |  Link
Groucho2004
Cantankerous Fossil
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,470
The more I think about it, the more I like the blank slate approach when it comes to how to go from here regarding AVS+.

In order to serve frames to a client, all that's needed is avisynth.dll and optionally a some (compatible) plugins in a directory. The user doesn't have to "install" Avisynth and switching between versions (and auto-load plugin directories) can be done in an instant without rebooting.
Considering this, re-compiling all compatible plugins catering for an all new interface shouldn't be such a big deal.
If a user really has to use some ancient plugin, he could just switch to the "official" AVS2.6.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 23rd June 2016, 17:47   #1805  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,173
Quote:
Originally Posted by Groucho2004 View Post
Considering this, re-compiling all compatible plugins catering for an all new interface shouldn't be such a big deal.
Starting with RemoveGrain that was last compiled with VS2005
MysteryX is offline   Reply With Quote
Old 23rd June 2016, 17:56   #1806  |  Link
Groucho2004
Cantankerous Fossil
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,470
Quote:
Originally Posted by MysteryX View Post
Starting with RemoveGrain that was last compiled with VS2005
Why? What's wrong with RGTools?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 23rd June 2016, 22:27   #1807  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Everybody please, there is no need to point out that a conversion layer can be written. As described in #1770, I know that the old interface can be worked around. But frankly, I will not invest my own time in developing and maintaining compatibility to something which we actually want to switch away from, once we switched away from it. I will not block such attempts, so if somebody comes up with working patches, I'll merge them. But those patches won't come from me.

As for accessing size_t members using int-functions: That will work in *most* cases, as long as plugins are well-behaved. Unfortunately, by some brain-dead decision in the early development of avs classic, the class members have been made public. Knowing the quality standards of many plugins, there is no guarantee that somebody does not access the fields directly, or maybe even create/dereference instances of VideoFrame objects. They shouldn't, most plugins probably don't, but this is a breaking change of the ABI, and I won't be doing code reviews of all plugins to determine which ones we broke and which ones we didn't.

It would be a different story if we had a central repository of all plugins in Git so that we can recompile them with a single click, as well as a package manager, so that if somebody updates the core, plugins will be updated automatically to the new interface. This was a GSoC-proposal once...
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 23rd June 2016, 23:48   #1808  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by tormento View Post
As pinterf is now an active contributor, I feel it is time to submit again an old problem I introduced months ago.

Here is the original sample from BD.

When I apply the following script, using AviSynth 2.6 MT, I have good encodings.

13HoursAV26.dgi

PHP Code:
SetMTMode(3)
SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\DGDecIM\dgdecodeim.dll")
DGSourceIM("E:\in\2_24 13 hours  The secret soldiers of Benghazi\13HoursAV26.dgi"silent=true)
ChangeFPS(last,last,true)
SetMTMode(2)
crop(01400, -140)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6
Result.

When I use AviSynth+, I have image corruption.

13HoursAVS+.dgi

PHP Code:
SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE"2)
SetFilterMTMode("ChangeFPS"3)
SetFilterMTMode("DGSource"3)
LoadPlugin("D:\eseguibili\media\DGDecNV\x64\DGDecodeNV.dll")
DGSource("E:\in\2_24 13 hours  The secret soldiers of Benghazi\13HoursAVS+.dgi")
#SetFilterMTMode("DGSourceIM", 3)
#LoadPlugin("D:\eseguibili\media\DGDecIM\x64\dgdecodeim.dll")
#DGSourceIM("E:\in\2_24 13 hours  The secret soldiers of Benghazi\13HoursCUT.dgi", silent=true)
ChangeFPS(last,last,true)
#Crop(0, 140, 0, -140)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)
Prefetch(8
prefilter=1 OK
prefilter=2 OK
prefilter=3 OK
prefilter=4 CORRUPTION

Memento:

PHP Code:
prefilter

[int"-1"01234] or [clip: - ]

-
off (Default)
light controlled gauss blur
mild median/gauss blur
strong median/gauss blur
dfttest (spatial)
KNLmeansCL (spatio-temporal GPU filter)
variable prefiltered clip input variable 
I have tried to use x86 version of AviSynth+, even DGIndexIM.

Results does not change.

The plugin versions I use are the following for both AviSynth and AviSynth+.

PHP Code:
[Avisynth CPP 2.6 plugins]
D:\Programmi\media\AviSynth+\plugins64\KNLMeansCL-0.7.6.dll  (n/a)
D:\Programmi\media\AviSynth+\plugins64\MaskTools-2.1b1-tp7.dll  (2.1.0.0)
D:\Programmi\media\AviSynth+\plugins64\MedianBlur2-0.94-tp7.dll  (n/a)
D:\Programmi\media\AviSynth+\plugins64\MVTools-2.7.0.22-pfmod.dll  (2.7.0.22)
D:\Programmi\media\AviSynth+\plugins64\RgTools-0.92.1-tp7.dll  (n/a)

[
Avisynth CPP 2.5 plugins]
D:\Programmi\media\AviSynth+\plugins64\DFTTest-1.9.4.dll  (1.9.4.0)
D:\Programmi\media\AviSynth+\plugins64\Dither-1.27.2.dll  (n/a
I can provide other movie examples too.

Ideas? Fixes?

P.S.: Previous post is here.
Instead of trying to find alternatives, I'd be happy if we found the bug.

Looking at the encodings, to me this looks like a clipping (overflow/underflow) bug, both at high and low pixel values. If that is true, once the binary filter causing this is isolated, the fix should be simple.
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 24th June 2016, 00:12   #1809  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by ultim View Post
Instead of trying to find alternatives, I'd be happy if we found the bug.

Looking at the encodings, to me this looks like a clipping (overflow/underflow) bug, both at high and low pixel values. If that is true, once the binary filter causing this is isolated, the fix should be simple.
Prefilter=4 means KNLMeansCL and the belonging part of SMDegrain. The strange thing is that it happens only with AVS+
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 24th June 2016, 00:28   #1810  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by tormento View Post
Prefilter=4 means KNLMeansCL and the belonging part of SMDegrain. The strange thing is that it happens only with AVS+
Not strange at all. It is with all likelyhood an internal filter in the Avs+ core. Most of those have been rewritten in Avs+ to get rid of inline assembly. And I do see SMDegrain making calls to internal functions along the prefilter=4 paths (too). KNLMeansCL might be the feature of prefilter=4, but I do not think this plugin is the cause of the problem.
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 24th June 2016, 00:53   #1811  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,293
Quote:
Originally Posted by ultim View Post
Not strange at all. It is with all likelyhood an internal filter in the Avs+ core. Most of those have been rewritten in Avs+ to get rid of inline assembly. And I do see SMDegrain making calls to internal functions along the prefilter=4 paths (too). KNLMeansCL might be the feature of prefilter=4, but I do not think this plugin is the cause of the problem.
I remember dogway say this happened if DGSourceIM + KNLMeansCL in the same script in avs+
__________________
My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 24th June 2016, 08:21   #1812  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,964
... so are two different plugins using the same hardware for acceleration, therefore might switch hardware operation modes of graphic chips independently of each other, especially when working multi-threaded. There is little surprise that this may corrupt the video when the GPU gets reprogrammed to decode the next frame while filtering the previous.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 24th June 2016, 09:46   #1813  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,764
Honestly, about interface, evolution does the job only half, to do thing properly to the end, i would have put all the pitch in ptrdiff_t. If you have to break existing things, break it for a full good purpose, not half. And also the Getpitch functions should return the same.
But, i agree about something. These critical datas shouldn't have been public, and should be accessed only via functions.

Well, personnaly, i would have done :
Code:
class VideoFrame {
public:
	volatile long refcount;
	VideoFrameBuffer* const vfb;
	const size_t offset;
	const ptrdiff_t pitch;
	const uint32_t row_size;
	const int32_t height;
	const size_t offsetU, offsetV;  // U&V offsets are from top of picture.
	const ptrdiff_t pitchUV;
	const uint32t row_sizeUV;
	const int32_t heightUV;

Last edited by jpsdr; 24th June 2016 at 10:02.
jpsdr is offline   Reply With Quote
Old 24th June 2016, 12:12   #1814  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by real.finder View Post
I remember dogway say this happened if DGSourceIM + KNLMeansCL in the same script in avs+
It happens with DGSource (NV) + KNLMeansCL too.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 24th June 2016, 12:13   #1815  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by LigH View Post
... so are two different plugins using the same hardware for acceleration, therefore might switch hardware operation modes of graphic chips independently of each other, especially when working multi-threaded. There is little surprise that this may corrupt the video when the GPU gets reprogrammed to decode the next frame while filtering the previous.
Ok and why only with AVS+ and not with AVS standard?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 24th June 2016, 12:22   #1816  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by tormento View Post
It happens with DGSource (NV) + KNLMeansCL too.
Can you provide a minimalistic script that reproduces the problem? Not with SMDegrain, but something without eval, without calling other scripts, without conditional code etc. As few lines as possible.
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 24th June 2016, 12:30   #1817  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Avisynth+

Quote:
Originally Posted by ultim View Post
Can you provide a minimalistic script that reproduces the problem? Not with SMDegrain, but something without eval, without calling other scripts, without conditional code etc. As few lines as possible.


Sorry, not in my capabilities.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 24th June 2016, 14:04   #1818  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 188
tormento

Avs+: r1841, MT-pfmod, x86
KNLMeansCL: http://rgho.st/6K5RRq2t8 (dunno how to check file version)
Script:
Code:
SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("ChangeFPS", 3)
SetFilterMTMode("DGSourceIM", 3)
DGSourceIM("13HoursCUT.dgi", silent=true)
ChangeFPS(last,last,true)
Crop(0, 140, 0, -140)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)
Prefetch(8)
Results: http://screenshotcomparison.com/comparison/176528
Please ensure that your PC is not infected by viruses, then try my KNLMeansCL (32-bit dll).
I didn't tried to check this script on my PC, only on notebook.
DJATOM is offline   Reply With Quote
Old 24th June 2016, 14:31   #1819  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,173
Quote:
Originally Posted by DJATOM View Post
tormento

Avs+: r1841, MT-pfmod, x86
KNLMeansCL: http://rgho.st/6K5RRq2t8 (dunno how to check file version)
Script:
Code:
SetMemoryMax(8000)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("ChangeFPS", 3)
SetFilterMTMode("DGSourceIM", 3)
DGSourceIM("13HoursCUT.dgi", silent=true)
ChangeFPS(last,last,true)
Crop(0, 140, 0, -140)
SMDegrain (tr=4,PreFilter=4,thSAD=400,contrasharp=false,refinemotion=false,plane=4,chroma=true,lsb=true,mode=6)
Prefetch(8)
Results: http://screenshotcomparison.com/comparison/176528
Please ensure that your PC is not infected by viruses, then try my KNLMeansCL (32-bit dll).
I didn't tried to check this script on my PC, only on notebook.
I've also been having issues with SMDegrain(prefilter=4) working terrible with MT; working faster with a single thread. I suppose this script will have the same problem.
MysteryX is offline   Reply With Quote
Old 24th June 2016, 14:44   #1820  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 188
Strange, but in avspmod x64 it only shows the first 800 pixels for height. Applying SMDegrain (with lsb) didn't change it, still 1920x800. 32-bit version works fine. Need to investigate.
Upd: made a new index and problem is fixed now (tested with avs+ r1873, x64; index made with DGIndexIM beta 50 X64). Still can't see any corruption :\

Last edited by DJATOM; 24th June 2016 at 15:09.
DJATOM is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.