Avisynth+

[Khanattila]

Originally Posted by vivan

Of cource it's 16 bit per channel because no one sane is going to implement 5-bit rgb in 2016.

Oops, I really misunderstood.

[pinterf]

Originally Posted by Groucho2004

I just tried it with about 500 plugins

Do you usually do extreme sports?

Regarding avisynth things, I'm still waiting for more error reports.
jpsdr already found two of them.

The development version on my git contains the following changes:

• New: arrays (you can read about them at a few pages back)
• New: CombinePlanes, like ShufflePlanes in Vapoursynth
  (though I like the ShufflePlanes name more)
• ExtractY, ExtractU, ... V, R, G, B, A:
  shortcuts for extracting single planes from planar YUV(A) or RGB(A) (ExtractU/V is UToY8/VToY8)
• VirtualDubFilter: really allow param type 'd' and 'l' (maps them into 'f' and 'i' for avisynth - we don't have double and long types)
• YToUV accepts an extra alpha clip source if target is YUVA
• much faster YUV444<->Planar RGB conversion
• fix: planar RGB(A) turnleft/turnright
• fix: PlaneToY("Y")

Then there is a new RgTools (waiting for release), forked from tp7's repository.
The new version supports 10,12,14,16 bit and float formats, and accepts planar RGB besides YUV (with all valid bit-depths of course)

[StainlessS]

Originally Posted by pinterf

Do you usually do extreme sports?

I have been known to have about 260 plugs in-situ, although more recently is limited to about 160 (+ specials on occasion)

New: arrays (you can read about them at a few pages back)

You have given New Hope, thank you kind sir, may all of your memories, be good ones.

EDIT: (I'm still sticking currently with Standard AVS, but am quite excited because of all of your Good Stuff, will swap one day)
Peace and Love

[ilko]

Hi there, I can finally post after one week…

Using NSIS, I've compiled an alternative avs+ installer based on the latest r2294 release. Basically this was for my personal use but maybe some people will find it interesting.

Note : x64 systems only.


Additional features

• FFMpegSource2 v2.23

Optional :

• A selection of plugins with their documentation
• A selection of scripts (incl. Interframe2, QTGMC)
• A basic script template ready to use
• New icons, black for regular scripts, blue for auto-load scripts or classic original style
• New context menu entry for the templates
• Open scripts with Notepad++
• Open .ass with Notepad++
• Open .ass files in Aegisub menu item
• Full integration of .ass subtitle files (incl. a dedicated icon)
• AviSynth+ language syntax recognition file for Notepad++

Downloads

Any feedback will be appreciated. Thanks.

[pinterf]

Originally Posted by Groucho2004

I think AVS+ should limit the number of plugins it enumerates during initialization just as 2.6.x. I just tried it with about 500 plugins (only about 60 unique plugins but renamed several times) and VDub, AVSMeter, mpc-hc either crash, stop working without error message or tell you they can't load the script.
The limit in 2.6.x is 50, so 100 may be a sensible number.

Edit: It seems to simply run out of memory (I tested on a 32 bit OS).

Using avsmeter 2.44, x86 avisynth+ r2318 internal build
0.) Renaming the given dll to xx_100.DLL .. xx_600.dll
1.) Run a simple script using avsmeter
2.) Get avsmeter - avsinfo

Summary
- avsmeter -avsinfo has problems with too much DLLs
- Loading DLL's consume memory, it depends on the plugin.
e.g. I don't know why 500 pcs of RgTools eat a relatively large memory
- nice to have feature from avsmeter: list approx idle memory consumption after DLL loading

Test #1 (mvtools2.dll 2.7.5.22)
1.) avsmeter testscript.avs
Memory consumption was 242MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

Listed all CPP 26 plugins, even the last one
...
C:\Program Files (x86)\AviSynth\plugins\xx_599.dll [2.7.5.22]
C:\Program Files (x86)\AviSynth\plugins\xx_600.dll [2.7.5.22]

Then
Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_138.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_139.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Test #2 (rgtools.dll 0.93)
1.) avsmeter testscript.avs worked
Memory consumption was 1045MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

Listed all CPP 26 plugins, even the last one

..
C:\Program Files (x86)\AviSynth\plugins\xx_599.dll [0.9.3.0]
C:\Program Files (x86)\AviSynth\plugins\xx_600.dll [0.9.3.0]

There were NO avsmeter error messages like with mvtools2

Test #3 (nnedi3.dll 0.9.4.30)
1.) avsmeter testscript.avs worked (but spent a few minutes in "Querying avisynth info")
Memory consumption was 181MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

It took some minutes until I got the list (that nnedi3 DLL is huge)

Same, as with mvtools2, I got
Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_138.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_139.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Test #4 (knlmeanscl.dll)
1.) avsmeter testscript.avs worked
Memory consumption was 227MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

There were NO avsmeter error messages like with mvtools2 and nnedi3

[Groucho2004]

Originally Posted by pinterf

Test #1 (mvtools2.dll 2.7.5.22)
1.) avsmeter testscript.avs
Memory consumption was 242MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

Listed all CPP 26 plugins, even the last one
...
C:\Program Files (x86)\AviSynth\plugins\xx_599.dll [2.7.5.22]
C:\Program Files (x86)\AviSynth\plugins\xx_600.dll [2.7.5.22]

Then
Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_138.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_139.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

That's most likely due to static linking of this DLL, see Myrsloik's comment above in #2615.

Originally Posted by pinterf

Test #2 (rgtools.dll 0.93)
1.) avsmeter testscript.avs worked
Memory consumption was 1045MB with, 168 MB without the 500+ DLL's

That may be a memory leak in RGTools, but it's just a wild guess.

Originally Posted by pinterf

Test #3 (nnedi3.dll 0.9.4.30)
1.) avsmeter testscript.avs worked (but spent a few minutes in "Querying avisynth info")
Memory consumption was 181MB with, 168 MB without the 500+ DLL's

2.) avsmeter -avsinfo

It took some minutes until I got the list (that nnedi3 DLL is huge)

Same, as with mvtools2, I got
Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_138.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

Cannot load file 'C:/Program Files (x86)/AviSynth/plugins/xx_139.dll'. Platform returned code 1114:
dynamic link library (DLL) intialization routine failed.

See comment on mvtools2

[pinterf]

Originally Posted by Groucho2004

That's most likely due to static linking of this DLL, see Myrsloik's comment above in #2615.


That may be a memory leak in RGTools, but it's just a wild guess.


See comment on mvtools2

I don't know either what can leak there.
In this phase, only PluginInit is called, right?
Functions implemented in the DLL are added to Avisynth's function list. Only 6 of them exist:

    env->AddFunction("RemoveGrain", "c[mode]i[modeU]i[modeV]i[planar]b", Create_RemoveGrain, 0);
    env->AddFunction("Repair", "cc[mode]i[modeU]i[modeV]i[planar]b", Create_Repair, 0);
    env->AddFunction("Clense", "c[previous]c[next]c[grey]b", Create_Clense, 0);
    env->AddFunction("ForwardClense", "c[grey]b", Create_ForwardClense, 0);
    env->AddFunction("BackwardClense", "c[grey]b", Create_BackwardClense, 0);
    env->AddFunction("VerticalCleaner", "c[mode]i[modeU]i[modeV]i[planar]b", Create_VerticalCleaner, 0);

The test script uses nothing from rgtools.

[Groucho2004]

Originally Posted by pinterf

I don't know either what can leak there.
In this phase, only PluginInit is called, right?

Correct. Not even the constructors are called.
One explanation would be that the other two DLLs are statically linked and Avisynth stops enumerating after the first DLL initialization failure. Have you tried building mvtools2 with dynamic linking to the MS runtimes? It might turn out that it uses just as much memory as RGTools if the enumeration finishes.

[jpsdr]

nnedi3 is build with runtime library option /MT, not /MD.
I must said that i don't realy know very precisely what's the difference, but i've read once that it would be better. I don't remember exactly what the "better" was, probably not having to install redistribuables, but not sure.
Now, if you said that finaly /MD is better, i don't mind switching for the next releases.

[real.finder]

Originally Posted by jpsdr

nnedi3 is build with runtime library option /MT, not /MD.
I must said that i don't realy know very precisely what's the difference, but i've read once that it would be better. I don't remember exactly what the "better" was, probably not having to install redistribuables, but not sure.
Now, if you said that finaly /MD is better, i don't mind switching for the next releases.

http://forum.doom9.org/showthread.ph...06#post1736106

in my opinion for icl make it MT and for msvc make it MD

[Groucho2004]

Originally Posted by jpsdr

nnedi3 is build with runtime library option /MT, not /MD.
I must said that i don't realy know very precisely what's the difference, but i've read once that it would be better. I don't remember exactly what the "better" was, probably not having to install redistribuables, but not sure.
Now, if you said that finaly /MD is better, i don't mind switching for the next releases.

There are at least 3 reasons I can think of why dynamically linking DLLs against the MS runtimes is preferable:

1. It consumes less system resources when the DLL is loaded multiple times

2. See this article on MSDN, particularly these paragraphs:

Using the statically linked CRT implies that any state information saved by the C runtime library will be local to that instance of the CRT. For example, if you use strtok, _strtok_l, wcstok, _wcstok_l, _mbstok, _mbstok_l when using a statically linked CRT, the position of the strtok parser is unrelated to the strtok state used in code in the same process (but in a different DLL or EXE) that is linked to another instance of the static CRT. In contrast, the dynamically linked CRT shares state for all code within a process that is dynamically linked to the CRT. This concern does not apply if you use the new more secure versions of these functions; for example, strtok_s does not have this problem.

Because a DLL built by linking to a static CRT will have its own CRT state, it is not recommended to link statically to the CRT in a DLL unless the consequences of this are specifically desired and understood. For example, if you call _set_se_translator in an executable that loads the DLL linked to its own static CRT, any hardware exceptions generated by the code in the DLL will not be caught by the translator, but hardware exceptions generated by code in the main executable will be caught.

...

If you have more than one DLL or EXE, then you may have more than one CRT, whether or not you are using different versions of Visual C++. For example, statically linking the CRT into multiple DLLs can present the same problem. Developers encountering this problem with static CRTs have been instructed to compile with /MD to use the CRT DLL. If your DLLs pass CRT resources across the DLL boundary, you may encounter issues with mismatched CRTs and need to recompile your project with Visual C++.

3. Dynamic linking makes sure that the code of the latest runtimes is used (provided that the latest runtimes are installed).

[Groucho2004]

Originally Posted by real.finder

in my opinion for icl make it MT and for msvc make it MD

I'd like to know how you came to that assessment. The Intel compiler uses the same libraries as the MS compiler and therefore requires the same runtimes (and additional Intel runtime libs).

[jpsdr]

Interesting informations, thanks. According to theses, I'll switch to /MD for next releases.
Never thought of reason number 3, even if when you realise it, it's obvious...

[dipje]

Originally Posted by Groucho2004

3. Dynamic linking makes sure that the code of the latest runtimes is used (provided that the latest runtimes are installed).

Not always true these days with the side-by-side system. Look at the side of the WinSxS folder, a lot of versions of the same DLL files are kept there just to prevent the old Win95/98 DLL hell problem .

[Groucho2004]

Originally Posted by dipje

Not always true these days with the side-by-side system. Look at the side of the WinSxS folder, a lot of versions of the same DLL files are kept there just to prevent the old Win95/98 DLL hell problem .

That nightmare was abandoned by MS with the introduction of VS2010. It's only relevant for VS2005/8.

[pinterf]

O.K., as we have MvTools 10-16 bit, RgTools 10-16bit + float + planar RGB, let's get back to Avisynth+.

For those who can't wait until I prepare a proper release and documentation, please find here a r2337 test build.
edit: replaced the link to the latest dev version

http://www.mediafire.com/file/h6nlac...splus-r2337.7z

Changes since r2294:

20161208 r2337dev

• C interface array compatibility vol#2 (zero size arrays) (AvsPMod)
• Merge, MergeChroma, MergeLuma: AVX2 (planar)
• Possibly a bit faster text overlay

20161207 r2333dev

• Overlay fix

20161206 r2331dev

• YUY2 PlaneToY finally works
• C interface compatible array-type AVSValue handling (avspmod issue)

20161205 r2327dev

• BlankClip parameter "colors" accepts exact color values to use
  Color order: Y,U,V,A or R,G,B(,A)
  These color values are used as-is, not scaled or converted in any way.
  Reason: old colors parameter is int (32 bit) cannot hold three or four 16 bit or float values
  Example: BlankClip(width=1920,height=1080,length=1000,pixel_type="RGB64", colors=[64000,32768,1231,65535])
• ExtractY, PlaneToY("Y") accepts YUY2 clip
• ExtractR, ExtractG, ExtractB, ExtractA,
  and
  PlaneToY("R"), PlaneToY("G"), PlaneToY("B"), PlaneToY("A")
  functions are accepting packed RGB input (RGB24/32/48/64)
  They are converted to planar RGB on-the-fly before plane extraction
• Histogram "levels" works from Planar RGB.
  Color legends show R, G and B.
  bits=8..12 parameter is still available for finer ultra-wide histogram display

20161201 r2322dev

• constant script arrays
  

  array_variable = [[1,2,3],[4,5,8],"hello"]
  dictionary = [["one",1],["two",2]]
  empty = []
  subarray = array_variable[0]
  val = subarray[2]
  val2 = array_variable[1,3]
  str = array_variable[2]
  n = ArraySize(array_variable) #3
  n2 = ArraySize(empty) #0
  val3 = dictionary["two"]

• arrays as filter parameters (named and unnamed):
  new 'a' type or use '.' (any) and check AVSValue IsArray()
  todo: maybe .+ or *+ syntax?
• Planar RGB <-> YUV: SSE2 (SSE4)
• Planar RGB <-> Packed RGB32/64: SSE2

20161120:

• make PlanarRGB TurnLeft, TurnRight work again.
  (too strict check in PlaneToY)

20161116

• new functions for plane extraction
  ExtractX, where X = R,G,B,Y,U,V,A
• Extended function
  old: YToUV(clip clipU, clip clipV [, clip clipY ] )
  new: YToUV(clip clipU, clip clipV [, clip clipY [, clip clipA] ] )
  
  YToUV accepts optional alpha clip after Y clip
  
  Example
  

  U = source.UToY8()
  V = source.VToY8()
  Y = source.ConvertToY()
  A = source.AddAlphaPlane(128).PlaneToY("A")
  # swaps V, U and A, Y
  YToUV(V,U,A,Y).Histogram("levels").Info().RemoveAlphaPlane()

• New function
  CombinePlanes(clip1 [,clip2, clip3, clip4], string planes [, string source_planes, string pixel_type, string sample_clip])
  
  Combines planes of source clip(s) into a target clip
  
  If sample_clip is given, target clip properties are copied from that clip
  If no sample_clip is provided, then clip1 provides the template for target clip
  An optional pixel_type string (e.g."YV24", "YUV420PS", "RGBP8") can override the base video format.
  
  If the source clip count is less than the given planes defined, then the last available clip is used as a source.
  
  string planes
  the target plane order (e.g. "YVU", "YYY", "RGB")
  missing target planes will be undefined in the target
  
  string source_planes (optional)
  the source plane order, defaulting to "YUVA" or "RGBA" depending on the video format
  
  Example#1
  

  #combine greyscale clips into YUVA clip
  U8 = source.UToY8() # or ExtractU
  V8 = source.VToY8() # or ExtractV
  Y8 = source.ConvertToY() # or ExtractY
  A8 = source.AddAlphaPlane(128).PlaneToY("A") # or ExtractA
  CombinePlanes(Y8, U8, V8, A8, planes="YUVA", source_planes="YYYY", sample_clip=source) #pixel_type="YUV444P8"

  Example#2
  

  # Copy planes between planar RGB(A) and YUV(A) without any conversion
  # yuv 4:4:4 <-> planar rgb
  source = last.ConvertBits(32) # 4:4:4
  cast_to_planarrgb = CombinePlanes(source, planes="RGB", source_planes="YUV", pixel_type="RGBPS")
  # get back a clip identical with "source"
  cast_to_yuv = CombinePlanes(cast_to_planarrgb, planes="YUV", source_planes="RGB", pixel_type="YUV444PS")

  Example#3
  

  #create a black and white planar RGB clip using Y channel
  #source is a YUV clip
  grey = CombinePlanes(source, planes="RGB", source_planes="YYY", pixel_type="RGBP8")

  Example#4
  

  #copy luma from one clip, U and V from another
  #source is the template
  #sourceY is a Y or YUV clip
  #sourceUV is a YUV clip
  grey = CombinePlanes(sourceY, sourceUV, planes="YUV", source_planes="YUV", sample_clip = source)

edit: test version updated

[LigH]

How much faster could "benchmark" scripts like QTGMC be when optimized for all these new features? Just starting with support of planar YUV with fine chroma subsampling...

[Motenai Yoda]

with this last test build avspmod doesn't work, at least on my spec

[pinterf]

Originally Posted by Motenai Yoda

with this last test build avspmod doesn't work, at least on my spec

I'm trying this version (x86) and it works, at least I'm getting the right preview.
http://forum.doom9.org/showpost.php?...postcount=1148

However there is an error window:

Error parsing ArrayGet plugin parameters: unknown character 'a'
Error parsing ArrayGet plugin parameters: unknown character 'a'
Error parsing ArraySize plugin parameters: unknown character 'a'
Error parsing BlankClip plugin parameters: unknown character 'a'
Error parsing BlankClip plugin parameters: unknown character 'a'
Exception WindowsError: 'exception: access violation reading 0x4049D7D0' in 
  <bound method AVS_Value.__del__ of [None, None, None]> ignored
Exception WindowsError: 'exception: access violation reading 0x4030FFF0' in 
  <bound method AVS_Value.__del__ of [None, None, None]> ignored

AvsPMod is enumerating all internal functions and parses their parameters. This parameter type 'a' is new to this Avisynth Plus version, it is for array-type parameters. But it seems it didn't break AvsPMod for me.
As for the exceptions, I cannot tell what they are meaning.
Any other experiences?

Edit:
regarding the exceptions.
it must be a C interface that may interfere with the array handling. Array-type AVSValue now copies all array elements when assigned to another AVSValue.

The old AVSValue stores only a pointer on the child AVSValue elements.
When AVSValue is free'd the elements are still held in memory and won't get freed.

For the new behaviour, AVSValue can hold elements that can also be arrays, they can even be nested. All AVSValue elements of an array are copied, not just the reference. When a child element is array, it is also copied, etc.... What happens when an array-type AVSValue is freed? Avisynth core now resursively frees up all AVSValue within an array, because it can do it safely, as the array creation and copy ensures that the elements were really copied.

I guess that when C Interface uses avs_release_value, this interferes with this logic.
AvsPMod allocates the main array-type AVSValue, allocates array element AVSValues on its own. When avs_release_value is called, it tries to free up array elements that were not allocated by avs core. It's not clear for me, what happens, never used C interface, but this is my assumption, I have to investigate this.

[pinterf]

Ok, let's try this one:

http://www.mediafire.com/file/90d7qa...splus-r2331.7z

Changes since r2294: see above. Last change:

20161206 r2331dev

• YUY2 PlaneToY
• C interface compatible array-type AVSValue handling (avspmod issue)