Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 Doom9's Forum Avisynth+
 Register FAQ Calendar Search Today's Posts Mark Forums Read

16th September 2013, 00:17   #2  |  Link
Groucho2004

Join Date: Mar 2006
Location: Barcelona
Posts: 5,054
Quote:
 Originally Posted by ultim Feedback?
A couple of observations after running a bunch of my test scripts:
Exception handling is different than the official (and my ICL) DLL. When Avisynth runs out of memory, this dialog is displayed:

The official version(s) properly throw an error message like this:
Code:
GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=12912704, max=2109149184, used=1564790336
I think we have run out of memory folks!
The performance is about the same as with the official DLL, sometimes a little bit better. The ICL DLL is in some rare cases much faster.
However, this is rather academic since the plugins and filters determine the performance by a large margin.

You should mention that the DLL is dynamically linked and that one needs the VC2012 runtime DLLs. I had to figure this out with Dependency Walker.

All this was tested on XP32, SP3.

16th September 2013, 07:08   #3  |  Link
ultim
AVS+ Dev

Join Date: Aug 2013
Posts: 359
Quote:
 Originally Posted by Groucho2004 A couple of observations after running a bunch of my test scripts: Exception handling is different than the official (and my ICL) DLL. ... You should mention that the DLL is dynamically linked and that one needs the VC2012 runtime DLLs. I had to figure this out with Dependency Walker. All this was tested on XP32, SP3.
Thanks, later today (after work) I'll reupload a version built with VS2005 to decrease the probability of VC runtimes needing to be installed. I'll also take a look at the exception handling problem.

Quote:
 Originally Posted by Groucho2004 The performance is about the same as with the official DLL, sometimes a little bit better. The ICL DLL is in some rare cases much faster.
That is about expected. As I mentioned in my original post, the optimizations are mostly to the initialization path. During frame-processing most time is spent in the plugins, BitBlt, and video frame allocation. I wasn't able to make BitBlt conclusively faster for now, and I've yet to take a closer look at the frame allocation path. Either way, most time is still spent in the plugins, so the only way to really speed Avisynth up is to improve the plugins.

Quote:
 Originally Posted by qyot27 If for no other reason, this certainly makes the sources dead simple to build for those of us used to cross-compiling.
Making it easy to build was the primary goal for now after all. Thanks for the additional instructions too. But as a note to others, you don't need the command line to compile if you are not familiar with it. Using CMake GUI and the VS IDE allows you to accomplish everything, even disabling the build of DirectShowSource (just unload the project in the IDE).

Quote:
 Originally Posted by Stephen R. Savage What would be great is if we could start working on making the AVS core 32/64-bit safe.
That is one of my long-term goals, it will probably take a lot of time though and I'm not working on that directly right now.

Last edited by ultim; 16th September 2013 at 07:52.

16th September 2013, 09:30   #4  |  Link
Groucho2004

Join Date: Mar 2006
Location: Barcelona
Posts: 5,054
Quote:
 Originally Posted by Groucho2004 You should mention that the DLL is dynamically linked and that one needs the VC2012 runtime DLLs.
Quote:
 Originally Posted by ultim Thanks, later today (after work) I'll reupload a version built with VS2005 to decrease the probability of VC runtimes needing to be installed.
I only mentioned this so people know that the VS2012 runtimes are a requirement.

17th September 2013, 05:26   #5  |  Link
qyot27
...?

Join Date: Nov 2005
Location: Florida
Posts: 1,268
Quote:
 Originally Posted by ultim Making it easy to build was the primary goal for now after all. Thanks for the additional instructions too. But as a note to others, you don't need the command line to compile if you are not familiar with it. Using CMake GUI and the VS IDE allows you to accomplish everything, even disabling the build of DirectShowSource (just unload the project in the IDE).
This is true. I just hate having to open up the Visual Studio IDE. Terminal-based compilation has me spoiled.

For that matter, my recommendation of hooking it into MSys isn't actually required either, but I'm not as comfortable with using variables and for loops through cmd.exe (also, the variables and for loops aren't necessary either if the user doesn't plan on using UPX packing or packing the generated .dlls up into a nicely-named archive). It can be done strictly through the Visual Studio Command Prompt.

 17th September 2013, 08:47 #6  |  Link ultim AVS+ Dev     Join Date: Aug 2013 Posts: 359 On a (not-so-) unrelated note, given that Avisynth's development has been mostly dormant with only very few contributors, it is not very motivating when somebody steps up with work and willingness to make the situation better, to tell him "go and work on that other project". Especially because nobody has said anything wrong about my proposals. If it had problems like breaking existing functionality or making code worse I'd understand. Or if IanB had reviewed them and told me "you should fix that differently because...", or any kind of constructive criticism, I'd go back and fix those things ASAP. But literally the only problem some of you had with my changes is that "there are too many", which makes me LOL in sadness. Should I beg for forgiveness that I contribute to the project at all? Last edited by ultim; 17th September 2013 at 09:40.
 16th September 2013, 01:44 #8  |  Link StainlessS HeartlessS Usurer     Join Date: Dec 2009 Location: Over the rainbow Posts: 9,132 If it dont compile well with later version compiler, then have a go at them. VS6 is just fine and any later problems come from later lack of compatibility. Really annoys me when you need the latest version IE v...whatever to view a web page, misguided at the very least. I'm sure that you mean well but it dont need that kind of upheaval. Several posts preceded my outburst, was aimed at Ultim. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 16th September 2013 at 01:51.
16th September 2013, 02:15   #9  |  Link
TheFluff
Excessively jovial fellow

Join Date: Jun 2004
Location: rude
Posts: 1,099
Quote:
 Originally Posted by StainlessS VS6 is just fine and any later problems come from later lack of compatibility.
this statement is so bizarrely wrong that I am unable to formulate a polite response to it

16th September 2013, 02:20   #10  |  Link
StainlessS
HeartlessS Usurer

Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,132
Quote:
 Originally Posted by TheFluff this statement is so bizarrely wrong that I am unable to formulate a polite response to it
Not much different there then.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

16th September 2013, 08:02   #11  |  Link
ultim
AVS+ Dev

Join Date: Aug 2013
Posts: 359
Quote:
 Originally Posted by StainlessS If it dont compile well with later version compiler, then have a go at them. VS6 is just fine and any later problems come from later lack of compatibility. Really annoys me when you need the latest version IE v...whatever to view a web page, misguided at the very least. I'm sure that you mean well but it dont need that kind of upheaval. Several posts preceded my outburst, was aimed at Ultim.
Hehe, I don't think I've made an upheaval, but if I did I'm sorry, I didn't intend to. But where I think you're wrong is that you are neglecting most of the community. Certainly, Avisynth doesn't strictly "need" a VS newer than VS6 to compile: compiled with VS6 it will work fine (not too surprising given it was mostly developed on VS6). But the people need newer VS versions. VS6 is so old that most people are used to newer environments, which by the way also brings better features for both debugging and comfort, as well as better compatibility for new OSs.

I find your comparison to a webpage needing the latest browser version highly flawed. Needing only a newer version just for the sake of upgrading sure is misguided. But I didn't upgrade the project to compile only with VS2013. I left VS2005 (which is 8 years old!) and anything newer than that fully supported, and I upgraded for the sake of making it easier for people to compile. That makes 4 different Visual Studio generations supported! If you really want an analogon to webpages, I'd say my move is more like MS trying to banish IE6, which everyone is happy about, except for people who still develop for IE6.

If you intend to keep Avisynth stuck to VS6, it will be an entry barrier for many people who'd otherwise try to contribute to the project. So this is why I think compatibility with VS2005+ is more important than VS6.

 16th September 2013, 03:34 #12  |  Link qyot27 ...?     Join Date: Nov 2005 Location: Florida Posts: 1,268 If for no other reason, this certainly makes the sources dead simple to build for those of us used to cross-compiling. Ah, what the heck. Build instructions suitable for copy-and-paste: This guide assumes a familiarity with a more *nix-centric command line compilation methodology, because it's easier to provide direct instructions for this that can just be copy/pasted. MSys and 7zip should already be installed, and the user should know what a phrase like 'adding the directory to the system PATH' means. [AviSynth+ prerequisites] Code: r1576 is the last milestone version that can built on XP with VS2010. Newer versions (which is what we care about) need at least VS2012. XP compatibility is still do-able by adjusting the toolset CMake uses. Install CMake: http://www.cmake.org/cmake/resources/software.html Add CMake's directory to the system PATH manually if the installer won't. Add 7zip to the PATH. Make sure that MSys' root directory (the one holding msys.bat) is also on the PATH. [DirectShowSource Prerequisites] DirectShowSource requires a couple of big pieces here that building the AviSynth+ core itself does not. It's not a requirement to build DirectShowSource, especially with the options of using either FFmpegSource2 or LSMASHSource, but the guide wouldn't be complete if I didn't cover it. DirectShowSource requires strmbase.lib, the C++ Base Classes library, which for some reason isn't included in a standard install of Visual Studio. The source code for the library is provided with the Windows SDK, and requires the user to build it first. Code: Download the Windows SDK 7.1: http://www.microsoft.com/en-US/download/details.aspx?Id=8442 Download the following ISO for x86: GRMSDK_EN_DVD.iso Verify against CRC32 or SHA1: CRC#: 0xBD8F1237 SHA1: 0xCDE254E83677C34C8FD509D6B733C32002FE3572 For convenience, you can use Pismo File Mount (if you've already got it installed for AVFS) to mount the ISO. Then just launch setup.exe and follow the wizard. Install only the Samples, uncheck everything else. Intellisense stays checked, though. Open VS2013 Express (or what is it, Community now?), and open the .sln file in the 7.1 SDK, at C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\multimedia\directshow\baseclasses Allow VS2013 to convert the project, switch it over to Release, and Build. That's it. For pre-SSE2 CPUs, make sure you switch the ARCH to either IA32 or SSE rather than leaving it disabled. This is buried in the Properties dialog. DirectShowSource also needs the DirectX SDK, so we'll grab that too. Code: Download the DirectX SDK (February 2010): http://www.microsoft.com/en-us/download/details.aspx?id=10084 Only install the headers and libs from it. Uncheck everything else. Unlike the strmbase.lib stuff, there's no post-install stuff required. To make the AviSynth+ build instructions more concise, we'll set a couple of environment variables. Open up MSys' /etc/profile and copy the following two lines into it somewhere: Code: export STRMBASELIB="C:/Program Files/Microsoft SDKs/Windows/v7.1/Samples/multimedia/directshow/baseclasses/Release/strmbase.lib" export DIRECTXSDK="C:/Program Files/Microsoft DirectX SDK (February 2010)/Include" Thankfully, all of this setup only needs to be done once. [Building AviSynth+] Start the Visual Studio Command Prompt You can use Visual Studio's stuff from MSys by launching MSys from the Visual Studio Command Prompt. So type 'msys' and hit Enter. Download the AviSynth+ source: Code: git clone git://github.com/AviSynth/AviSynthPlus.git mkdir avisynth_build cd AviSynthPlus Set up the packaging directory for later: Code: AVSDIRNAME=avisynth+_r$(git rev-list --count HEAD)-$(date --rfc-3339=date | sed 's/-//g') mkdir ../$AVSDIRNAME If DirectShowSource won't be built, comment out the reference to its subdirectory in plugins/CMakeLists.txt: Code: sed -i '4 s/^/#/' plugins/CMakeLists.txt Now, build AviSynth+. We'll do this out-of-tree so it's easy to clean up afterward: Code: cd ../avisynth_build For 32-bit: cmake -G "Visual Studio 12 2013" -T "v120_xp" -DDSHOWSRC_BASECLASSES_LIB="$STRMBASELIB" -DDSHOWSRC_DX_INCLUDE_PATH="$DIRECTXSDK" cmake --build . --config release mv Output avsplus_x86 mv avsplus_x86 ../$AVSDIRNAME rm -R * For 64-bit (DirectShowSource isn't really worth the effort on 64-bit, and most 64-bit users won't be on XP Pro 64-bit): cmake -G "Visual Studio 12 2013 Win64" cmake --build . --config release mv Output avsplus_x64 mv avsplus_x64 ../$AVSDIRNAME rm -R * cd ../ You can't build 32-bit and 64-bit at the same time. If you decided not to build DirectShowSource, omit the -DDSHOWSRC_BASECLASSES_LIB and -DDSHOWSRC_DX_INCLUDE_PATH options from the (32-bit) cmake command above. Packing the newly-built DLLs with 7zip: Code: 7za a -mx9$AVSDIRNAME.7z \$AVSDIRNAME Last edited by qyot27; 18th August 2015 at 04:47. Reason: Seriously, update these instructions
 16th September 2013, 09:43 #13  |  Link IanB Avisynth Developer   Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,173 It's obvious you have done a lot of work and are quite keen. You say you want your changes accepted upstream, yet you randomise the location of files and you make copious white changes to the sources. Hell, I would have love to restructure the tree to the way I think it should be, but I resist that urge very strongly. If you want to contribute changes you need to make them easy, very easy, extremely very easy to incorporate back into the original tree. If I download and apply a set of patches and compare the changes and see a small handful of very neat, very tidy and very concise changes, I will probably show a fair amount of interest and actively review the changes, checking the boundary conditions, inspecting the performance and considering the side effects. If I download a set of patches and they won't even apply because the files and directories have been moved and/or renamed, I probably won't bother looking any further. And if I do look further and I find a pant load of spurious white changes hiding the important differences then almost certainly I won't bother investigating further.
 16th September 2013, 11:40 #14  |  Link ultim AVS+ Dev     Join Date: Aug 2013 Posts: 359 It is true that there is a lot of noise. However, I disagree that they hide the useful differences, because I always kept them separate. For example, any time I moved files I did that in a separate commit without changing contents, so that Git can clearly tell that it was only a rename, and I only changed the files in the next commit. Never in the same commit so that Git recognize moved files and not flag them as removed/added. Also, important commits which fix bugs or affect well-defined features are also committed separately, to prevent them from mixing with "unimportant" changes. I was under the impression that you are also using Git, because your latest changes are in Git on GitHub. If you are in fact using Git, my changes will apply cleanly if you merge from my repository, and not export/import patchfiles. And even if you are using SVN, there are working bridges between Git and SVN so you can still pull my changes. So I don't really see your problem, especially because I *did* keep noise and useful changes in separate commits. Edit: Btw, telling me that I have randomized the file locations went a bit overboard. Anyone who checks out my repository can tell that the project structure is a thousand times cleaner than it ever was in the history of Avisynth. Last edited by ultim; 16th September 2013 at 14:22.
16th September 2013, 11:58   #15  |  Link
Groucho2004

Join Date: Mar 2006
Location: Barcelona
Posts: 5,054
Quote:
 Originally Posted by ultim I was under the impression that you are also using Git, because your latest changes are in Git on GitHub.
Avisynth is hosted here and as far as I know, Ian is only maintaining it using CVS.

 16th September 2013, 13:17 #16  |  Link ultim AVS+ Dev     Join Date: Aug 2013 Posts: 359 So Avisynth is only accepting contributions in the form of exported patch files? I have a full repository with cleanly separated changes and fully preserved history, and it is termed useless?
16th September 2013, 20:10   #17  |  Link
TheFluff
Excessively jovial fellow

Join Date: Jun 2004
Location: rude
Posts: 1,099
Quote:
 Originally Posted by ultim So Avisynth is only accepting contributions in the form of exported patch files? I have a full repository with cleanly separated changes and fully preserved history, and it is termed useless?
Avisynth is in many ways a project that is permanently stuck in the past, and all efforts have tried to modernize it are either forks (Avisynth-MT, Avisynth 64) or completely different projects (Avisynth 3, VapourSynth). You've found the reason for this the hard way.

Avisynth 2.6 is still a late 90's project at its core. Even when someone finally got around to breaking the API (after ten years), high bitdepth video still isn't supported, nor is 64-bit compilation (there are even a few comments in avisynth.h going "this will break on 64-bit!", gee thanks, it's almost 2014 now), all the old braindamaged API restrictions that sorta made sense in 2001 are still there and you still need to use CVS and support VC6, Windows 98 and pre-SSE CPU's because that's how it's always worked. Nobody is interested in developer friendly changes because nobody is developing Avisynth except poor IanB, and he understandably doesn't want to change his own environment.

If you want to get anything done, I suggest you either fork or start contributing to VapourSynth.

Last edited by TheFluff; 16th September 2013 at 20:20.

 16th September 2013, 20:28 #18  |  Link mandarinka Registered User     Join Date: Jan 2007 Posts: 739 Thanks for contributing to Avisynth! You really made it hard to merge upstream though, since you have a lot of changes at once and with the project files reshuffling, it is sort of like a "code dump" for the upstream/original developers. Ideally, the changes should be reviewed and then adopted, but in this form, it will be hard to do...
16th September 2013, 21:21   #19  |  Link
ultim
AVS+ Dev

Join Date: Aug 2013
Posts: 359
Quote:
 Originally Posted by mandarinka Thanks for contributing to Avisynth! You really made it hard to merge upstream though, since you have a lot of changes at once and with the project files reshuffling, it is sort of like a "code dump" for the upstream/original developers. Ideally, the changes should be reviewed and then adopted, but in this form, it will be hard to do...
I understand that my "patches" are a lot of code. However, nobody is forcing IanB to review and merge them in one night. My commits are on GitHub and they won't go anywhere. He could take them one-by-one, and review and merge them over a course of 2-3 weeks or more. Just because I published them all at once does not mean they need to be reviewed all at once, especially because they are broken up into lots of small commits, with only a few exceptions towards the beginning.

EDIT: I'd like to make it clear just once more, that me shuffling files should not make it that much harder to incorporate the changes. Because I have always done the "moving files" and "changing contents" in separate commits. So if upstream ever comes to a commit of mine which moved files, all he has to do is move files. Anything that changes contents will show up as a new commit, so there is never a case of a change being hidden just because the whole file shows up as new falsely. There is no friendlier or more respecting way to move files than this (except for the case when Ian moved to Git or SVN, in which case he wouldn't even have to move the files by hand).

Quote:
 Originally Posted by TheFluff ... and all efforts have tried to modernize it are either forks (Avisynth-MT, Avisynth 64) or completely different projects (Avisynth 3, VapourSynth). ... If you want to get anything done, I suggest you either fork or start contributing to VapourSynth.
Well, I guess it is time for me to present my full story. I have known Avisynth for a very long time, but until about 8-9 months ago, I only new about some of its forks. So I've began laying out plans how I could implement a better Avisynth, because no matter how much I liked Avisynth, its flaws did (and do) bother me. Off the top of my head my ideas were: built for multithreading in its core, video frames should be taggable by custom information, 64-bit compatbility, cross-platform support, just to name a few. In one word: very (too?) ambitious. I had absolutely no time to implement this though until a few months passed. About half a year ago then, I started laying out an improved .avs scripting language, with full support for old scripts, but better language features, like proper multiline if-blocks, multiple return values, and some more. Actually, that was the sole reason I ported the Gold Parser to C++. But I've done more than just port Gold Parser, I've even written and tested its language definition input file for my improved .avs syntax, in correct EBNF(-like) form and being able to parse!

Then, about this time, a "strange" (LOL) idea struck me: What if others have also started to make something that I was just planning to? So I fired up Google, and have found VapourSynth and Avxsynth. I saw that VapourSynth set mostly the same goals as I did, which was cool, but I was *very* annoyed by its choice of Python. Yes, it is a more powerful language, but also much harder to learn and use, and the syntax as implemented in VapourSynth is just too verbose. Anyway, I decided it is better if I help out one of the projects instead of just starting yet a another fork/alternative, and my decision fell to Avisynth. So after yet another few months of doing-other-things (diploma thesis, some free software, and looking for work), here I am contributing to Avisynth.

Btw, I updated my announcement post with a new binary that only requires a much older VC++ runtime. I also looked at the exception handling reported by Groucho2004, and right now my best bet is that the "old" behavior is pure luck. Because even in the old code (and there is no difference to this in my changes), if we run out of memory, new memory must still be reserved for the red error clip (unless there is a frame in the free-list). Now, if there was not enough memory to reserve a new frame, there is also the probability that there won't be enough memory to construct the error clip either -> in which case the code can only crash, both old and new. I have verified my code on countless other exceptions and for me they always correctly resulted in an error clip, and the only time my build crashed was when constructing the error clip failed due to low memory.

Last edited by ultim; 16th September 2013 at 21:54.

16th September 2013, 22:05   #20  |  Link
TheFluff
Excessively jovial fellow

Join Date: Jun 2004
Location: rude
Posts: 1,099
Quote:
 Originally Posted by ultim I saw that VapourSynth set mostly the same goals as I did, which was cool, but I was *very* annoyed by its choice of Python. Yes, it is a more powerful language, but also much harder to learn and use, and the syntax as implemented in VapourSynth is just too verbose.
As Stephen says, VapourSynth isn't tied to its scripting language like Avisynth is. VapourSynth is really just an API; the Python scripting component is just a module that calls that API. You can implement a new VapourSynth API wrapper in any language you want.