Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
20th May 2019, 11:23 | #1101 | Link | ||
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
Avisynth Threading
Quote:
Quote:
From my own experience having a 64bit application and everything in the chain x64 hasn't provided significant performance gains over running its equivalent x86 app. Admittedly, I've not been attempting something so resource intensive as this though. @pinterf Thank you, will take a look at these and see if I can find x64 verisons of the other plugins John's script uses and see if I can get this running and compare to x86 performance. I've checked that my source files aren't root cause by using the parade example clip from port #1078 to test run John's script and I'm getting same performance. So I must be missing something or have something fundamental in play here, as I get almost the same low fps performance across 2 different machines that have different hardware and OS using the same Avisynth + VirtualDub setups. Will keep looking and also explore the x64 plugins. Last edited by SnillorZ; 20th May 2019 at 11:48. |
||
20th May 2019, 12:10 | #1102 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
Quote:
You can use the Avsmeter utility to see the increase in memory consumption for Prefetch 2, 3, ... values. Or set SetMemoryMax manually though it cannot count with the extra memory need for external plugins, still worth playing with it. |
|
20th May 2019, 14:57 | #1103 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Also (clutching at straws) suggest use DebugView, see if message from dll requiring Avstp is issued (Google "DebugView", from MicroSoft), execute DebugView and run script, view debugview window.
(I cant say what the error message might be, I'm running encode at the moment and so cannot remove dll to test. [or just check your plugins directory for dll] I'm guessin' that some of your filters use avstp (for multithreading), and if absent then can cause all kinds of crashes and such. Same suggestion to stormy1777 if he sees this. EDIT: I think message issued by some plugin (rgtools maybe) is something like "Avstp.dll missing, dropping down to single threaded". EDIT: Could also post result of "AvsMeter avsinfo". Quote:
Latest version is v1.0.3
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 20th May 2019 at 15:11. |
|
20th May 2019, 17:06 | #1104 | Link | ||
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
Avisynth+ threading using AVSTP
@pinterf @StainlessS Thanks chaps
Quote:
Quote:
RemoveDirt 0.9 AvstpWrapper: cannot find avstp.dll.Usage restricted to single threading. AVSMeter AVSinfo (with -c) Code:
AVSMeter 2.9.1 (x86) - Copyright (c) 2012-2019, Groucho2004 VersionString: AviSynth+ 0.1 (r2772, MT, i386) VersionNumber: 2.60 File / Product version: 0.1.0.0 / 0.1.0.0 Interface Version: 5 Multi-threading support: Yes Avisynth.dll location: C:\windows\SysWOW64\avisynth.dll Avisynth.dll time stamp: 2018-12-20, 12:06:28 (UTC) PluginDir2_5 (HKLM, x86): D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins PluginDir+ (HKLM, x86): D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins [CPP 2.6 Plugins (32 Bit)] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\ConvertStacked.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\DirectShowSource.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\ImageSeq.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\Shibatch.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\TimeStretch.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\VDubFilter.dll [2018-12-20] Custom plugin directory : D:\VideoTools\AVSplugins [CPP 2.5 Plugins (32 Bit)] D:\VideoTools\AVSplugins\autolevels.dll [0.6.0.0] D:\VideoTools\AVSplugins\avstp.dll [1.0.3.0] D:\VideoTools\AVSplugins\deflicker.dll [0.4.0.0] D:\VideoTools\AVSplugins\FFT3DFilter.dll [2.1.1.0] D:\VideoTools\AVSplugins\Motion_06Dec05B.dll [2005-12-06] D:\VideoTools\AVSplugins\mt_masktools.dll [2.0.32.0] D:\VideoTools\AVSplugins\MVTools.dll [2008-09-11] D:\VideoTools\AVSplugins\RemoveDirt.dll [2008-04-01] D:\VideoTools\AVSplugins\RemoveDirtSSE2.dll [0.9.0.0] D:\VideoTools\AVSplugins\RemoveGrain.dll [2005-05-01] D:\VideoTools\AVSplugins\RemoveGrainS.dll [2005-05-01] D:\VideoTools\AVSplugins\RemoveGrainSSE2.dll [2005-05-01] D:\VideoTools\AVSplugins\warpsharp.dll [2008-04-06] [CPP 2.6 Plugins (32 Bit)] D:\VideoTools\AVSplugins\DePan.dll [2.13.1.3] D:\VideoTools\AVSplugins\DePanEstimate.dll [2.10.0.2] D:\VideoTools\AVSplugins\ffms2.dll [2016-12-29] D:\VideoTools\AVSplugins\FredAverage_x86.dll [0.3.0.0] D:\VideoTools\AVSplugins\GamMac_x86.dll [1.10.0.0] D:\VideoTools\AVSplugins\mvtools2.dll [2.7.39.0] [Scripts (AVSI)] D:\VideoTools\AVSplugins\FFMS2.avsi [2015-05-22] [Uncategorized files] D:\VideoTools\AVSplugins\03_RemoveDirtMC.avs [2010-06-29] D:\VideoTools\AVSplugins\ffms2.lib [2016-12-29] I'm still progressing to try and test x64 Avisynth / VDub / plugins: I've found x64 versions or alternatives that can be used with a modified JohnM script. Just left with 3 to find/swap out: autolevels.dll Deflicker.dll warpsharp.dll - for which I've found AWarpSharp2 but this doesn't contain the unsharpmask function from John's script and I'm not sure what its equivalent is. Anyway must go and do some paid work for a bit Last edited by SnillorZ; 20th May 2019 at 17:09. Reason: Billy Fat Fingers can't typo too gud. |
||
21st May 2019, 05:19 | #1105 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Firstly, I have not found x64 version of these either.
autolevels.dll Deflicker.dll warpsharp.dll I'll post this first. Code:
[CPP 2.5 Plugins (32 Bit)] D:\VideoTools\AVSplugins\autolevels.dll [0.6.0.0] # I've got source for this, maybe I'll give it a go for x64. D:\VideoTools\AVSplugins\avstp.dll [1.0.3.0] D:\VideoTools\AVSplugins\deflicker.dll [0.4.0.0] D:\VideoTools\AVSplugins\FFT3DFilter.dll [2.1.1.0] # I have v2.6 dll, fft3dfilter_x86_2_4_0_0.dll (maybe check latest fftw3.dll and libfftw3f-3.dll too, for SysWOW64) # Does not seem to be used by Fred script D:\VideoTools\AVSplugins\Motion_06Dec05B.dll [2005-12-06] D:\VideoTools\AVSplugins\mt_masktools.dll [2.0.32.0] D:\VideoTools\AVSplugins\MVTools.dll [2008-09-11] # Remove one of these D:\VideoTools\AVSplugins\RemoveDirt.dll [2008-04-01] # This is the one in Fred zip (but is slower) [EDIT: Requires MSVCR71.dll runtime in SysWOW64] D:\VideoTools\AVSplugins\RemoveDirtSSE2.dll [0.9.0.0] # This is faster SSE2 [EDIT: Requires MSVCR71.dll runtime in SysWOW64] # Missing RemoveDirtS.dll from fred sourcecode/removedirt/removedirt does NOT req (VS2003) MSVCR71.dll runtime in SysWOW64 # But is from 2005. # Remove Two of these ( *** WARNING Will Robinson ***, Version from Fred script is 1st Oct 2006 - and requires MSVCR71.dll runtime in SysWOW64) D:\VideoTools\AVSplugins\RemoveGrain.dll [2005-05-01] # Standard compile [EDIT: Probably requires either (Vs2002)MSVCR70.dll or MSVCR71.dll] D:\VideoTools\AVSplugins\RemoveGrainS.dll [2005-05-01] # S version static linked, dont need runtime D:\VideoTools\AVSplugins\RemoveGrainSSE2.dll [2005-05-01] # SSE2, faster D:\VideoTools\AVSplugins\warpsharp.dll [2008-04-06] # Oops, mine seem to be from 2003 [Groucho2004 will probably get a kick out of reading above link again ] Here overrides anything above. Suggest remove all from Plugins directory, EXCEPT Code:
[CPP 2.6 Plugins (32 Bit)] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\ConvertStacked.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\DirectShowSource.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\ImageSeq.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\Shibatch.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\TimeStretch.dll [2018-12-20] D:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\VDubFilter.dll [2018-12-20] [CPP 2.5 Plugins (32 Bit)] D:\VideoTools\AVSplugins\avstp.dll [1.0.3.0] [CPP 2.6 Plugins (32 Bit)] D:\VideoTools\AVSplugins\ffms2.dll [2016-12-29] [Scripts (AVSI)] D:\VideoTools\AVSplugins\FFMS2.avsi [2015-05-22] # Optional Follow Freds instructions to install (Into suggested directories), eg Code:
D:\Fred_Restore\ helpfiles\ scripts\ sourcecode\ Might need to install install MSVCR71.dll to SysWOW64. If MSVCR71.dll NOT in SysWOW64, then can find it here(also MSVCR70.dll):- http://www.mediafire.com/file/1220u8...times.rar/file Dependency Walker 2.2(to check if dll needs runtimes):- http://www.dependencywalker.com/ Also can copy/move fftw3.dll to SysWOW64 (for Depan use). [should not then need to be in the script directory, thats where most people put it]. Then suggest go though ALL scripts and where any current relative directory mentioned, prepend with ".\", and Also maybe change path separator slash "/" to "\" (for no real reason) eg, Loadplugin("plugins/Depan.dll") to Code:
Loadplugin(".\plugins\Depan.dll") # and Import(".\plugins\03_RemoveDirtMC.avs") If you dont keep Fred Plugs (which are old versions) separate from daily use plugins, you will just have problems some other time. (would be best if scripts fixed up to be working with current plugs, probably be good to get rid of old mvtools). Gorra get some shuteye, let us know what happens ta-ta.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 21st May 2019 at 05:48. |
21st May 2019, 08:39 | #1106 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
1.) Probably these RemoveDirt dlls can be deleted as well
D:\VideoTools\AVSplugins\RemoveDirt.dll [2008-04-01] D:\VideoTools\AVSplugins\RemoveDirtSSE2.dll [0.9.0.0] RemoveDirt_MC uses only RestoreMotionBlocks from this plugin. I replaced them with RemoveDirt 0.9.2 version 2.) RemoveGrain.DLL, RemoveGrainS.DLL, RemoveGrainSSE.DLL: All these can possibly be replaced by v0.97 RgTools 3.) Then I'm using masktools2 instead of 2.0.0.32 mt_masktools 4.) MvTools2 instead of 2008 version of mvtools Note: do not keep DLLs containing similarly named plugin functions in one autoloaded directory neither load them all manually. Collision happens. E.g. when multiple DLLs containing "clense" are loaded, referencing "Clense" would call either the 15 year old version or the one from the latest SSE2 or AVX2 capable version. Some of these newer DLLs are using not only SSE2 but SSE4.1 and AVX2 instructions which can give you further speed up when your processor is capable for that. When you bought your machine in the last 5 years, probably that is the case. |
21st May 2019, 10:23 | #1107 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
@StainlessS
Please note that SnillorZ's "D:\VideoTools\AVSplugins" is not an auto-load directory so none of the functions from DLLs in that directory are added to the function pool during auto-load. Apart from that - everything that pinterf wrote in the previous post is good advice.
__________________
Groucho's Avisynth Stuff |
21st May 2019, 11:43 | #1108 | Link | ||
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
@StainlessS @pinterf @Groucho2004 thank you all.
Yes this: Quote:
Sorry, so to clarify, for all 3 scripts I'm testing they all load the plugins required explicitly - hence using a common "avsplugins" directory with plugins with different versions and runtime requirements etc. I just started working this way as I found it easier to work out which script required what, which plugins caused the scripts to bomb, dll dependencies (thanks Stainless, I'd picked up a recommendation for DepWalker, I think from one of your much earlier posts). So for VideoFred2012 Script, JohnMeyer2012 (MT) Script and John Meyer2017 (Avisynth+ Gamac) script am loading each plugin explicitly and using the corresponding version of Avisynth (using Groucho's SetAVS) Also to clarify, the VideoFred2012 script works ok and at a speed that as I understand it could not be improved upon without MT which some of the plugins it uses are not supported. However I'm preferring the results given by the modifications made in John Meyers scripts - Just checking I have everything set in my environment to get the best performance I can before forging ahead into 3000+ foots worth of footage and finding at a later point in time I could have saved myself a ton of time with a few simple changes. Quote:
Doh! I'd not thought of trying swapping out for some of the newer x86 plugins to test for performance increases in the original JM2017 script - I'd focused in on trying to get running in 64bit. So will try those also and report back. Hope you managed to get some beauty sleep StainlessS. Toodle-Pip. |
||
21st May 2019, 11:55 | #1109 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
Maybe someone could just port Unsharpmask from warpsharp as a script? Myrsloik did a port for VS https://github.com/myrsloik/WarpShar...ter/wss.py#L28
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
21st May 2019, 14:09 | #1111 | Link | |
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
Quote:
AVSMeter 2.9.1 (x86) - Copyright (c) 2012-2019, Groucho2004 VersionString: AviSynth+ 0.1 (r2772, MT, i386) VersionNumber: 2.60 File / Product version: 0.1.0.0 / 0.1.0.0 Interface Version: 5 Multi-threading support: Yes Avisynth.dll location: C:\windows\SysWOW64\avisynth.dll Avisynth.dll time stamp: 2018-12-20, 12:06:28 (UTC) Custom plugin directory : D:\VideoTools\AVSplugins\x86 [CPP 2.5 Plugins (32 Bit)] D:\VideoTools\AVSplugins\x86\autolevels.dll [0.6.0.0] D:\VideoTools\AVSplugins\x86\avstp.dll [1.0.3.0] D:\VideoTools\AVSplugins\x86\deflicker.dll [0.4.0.0] D:\VideoTools\AVSplugins\x86\warpsharp.dll [2008-04-06] [CPP 2.6 Plugins (32 Bit)] D:\VideoTools\AVSplugins\x86\DePan.dll [2.13.1.4] D:\VideoTools\AVSplugins\x86\DePanEstimate.dll [2.10.0.3] D:\VideoTools\AVSplugins\x86\fft3dfilter.dll [2.6.0.0] D:\VideoTools\AVSplugins\x86\FredAverage_x86.dll [0.3.0.0] D:\VideoTools\AVSplugins\x86\GamMac_x86.dll [1.10.0.0] D:\VideoTools\AVSplugins\x86\masktools2.dll [2.2.18.0] D:\VideoTools\AVSplugins\x86\mvtools2.dll [2.7.41.0] D:\VideoTools\AVSplugins\x86\RemoveDirt.dll [0.9.2.0] D:\VideoTools\AVSplugins\x86\RgTools.dll [0.97.0.0] |
|
21st May 2019, 14:25 | #1112 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
(I usually point both at same dir, and I dont like putting Avisynth supplied dll's in same dir as others, could cross pollinate dll's when duplicating dll sets to another ver$ avs plugins). I thought that old version dll's would prompt for futher advice on replace with recent, I expected further posts on that (but thought might be reason for fred/john using old versions). These are the avs specific dll's that I copy to system32/SysWOW64 Code:
AvsRecursion.dll fftw3.dll # same as below file, name edited only libfftw3f-3.dll libsndfile-1.dll Pinterf, let me take a crack at AutoLevels x64, it uses DDigit for metrics which has been updated a few times since AutoLevels source supplied version, If I get into trouble, I'll holla. Oodles and Oodles of Toodles EDIT: I would probably not be ambitious enouigh yet to attempt avs+ colorspaces for AutoLevels, but let me do standard v2.6 version, and if you wish to add avs+ colorspace stuff then you can do mod on that.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 21st May 2019 at 14:44. |
|
21st May 2019, 14:56 | #1113 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
Quote:
|
|
21st May 2019, 16:16 | #1115 | Link | |
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
@pinterf
Sorry, trying to follow multiple suggestions as methodically as I can without ending up with retrograde steps, so please hold the line caller. To clarify though all tests as per @johnmeyer suggestion Result4/Comparison4 - So no Autolevels or Colour Corrections. Just DirtRemoval, RemoveGrain, Stabalize, etc Quote:
No performance improvement though. However, in light of this I've gone back to testing some amendments to JohnMeyer2012 script (using AVS2.6MT) and managed to get 4+fps. Using same "script relevant" plugins as post #1111 - with the exception of RgTools.dll [0.97.0.0] which seems to cause a crash with 2.6MT? Encouraging uptick though. Last edited by SnillorZ; 21st May 2019 at 16:41. |
|
21st May 2019, 16:34 | #1116 | Link |
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
@videoFred, @johnmeyer, @StainlessS, @pinterf, @Groucho2004
Just wanted to say that in 20+ years of using forums (for both profession & personally) I've rarely come across a more welcoming and helpful bunch of members. In fact, sadly more often than not, the experience is the complete opposite and once you admit to being a noob and lurking around a thread and then dain to ask a question, find yourself be chased down the road by the flaming torches & pitchforks wielding mob. Anyway, just wanted to make that point. If any of you found yourself in my part of the world (English Lake District) would happily buy you a beer Cheers P.S. If you see my boss, you've not seen me. If you see my wife, tell her cant come away from the computer just yet as "I've found someone who's wrong on the Internet" |
21st May 2019, 17:02 | #1117 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Bout 35 years ago, was half way between between Newcastle and Carlisle (English Lake District), stopped at pub in middle of nowhere and went in for a beer. Manager told me that I should have been there the day before. Why, I asked. Because, they heard a terrlible loud noise in back garden (pretty much nothing [EDIT: except Kielder Forest of course] for 40/50/60 miles up to Scotland, big garden), and looking out of window saw a helicopter landing there. Out jumped Prince Charles and a handfull of bodyguards/lackies. Somebody wanted a piss, so they stopped for a rest period, and a pint. Nice to know that the other half have to piss as well, I thought that they just 'glistened', rather a lot. EDIT: Lovely day here in The Big Smoke, wish I could escape from this pub and go start on AutoLevels x64 mod.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 21st May 2019 at 17:55. |
|
21st May 2019, 17:19 | #1118 | Link |
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
@StainlessS
I wouldn't have been surprised if you'd said Charlie and T'other off-comers in their big metal bird had been chased off. Some places up here still like "Deliverance" country and you have to be careful where one puts one's banjo. |
21st May 2019, 18:20 | #1119 | Link | |
Registered User
Join Date: Jan 2019
Location: Lake District UK
Posts: 48
|
johnmeyer 2017 script - Result use workflow
@johnmeyer
Quote:
Can I clarify, after doing a R4 processing with your script, are you saying then run the result (VirtualDub>Save As>Uncompressed avi) through a script process using R1/2/3 (as preferred)? Using same script i.e. letting Stabilise/Denoise/Sharpening process again? or use a modified version to comment out some of these (I note your comment GamMac introduces some flicker) |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|