Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
2nd November 2016, 02:18 | #2541 | Link |
...?
Join Date: Nov 2005
Location: Florida
Posts: 1,419
|
What kind of problem(s) exactly? tp7 mentioned the probable need to add something better in the core to deal with string formatting and concatenation a couple years ago, but that may not be the same thing you're referring to.
|
2nd November 2016, 03:52 | #2542 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
I was referring to memory usage, ie not freeing of temporary strings, strings used in eg any single instance of a Scriptclip frame
are not released until Avisynth closure, despite them never being used again. We should be able to automatically release local strings that are created in any script function and are not returned as the function result. I dont know if memory used for other non string local variables additionally are not released until closure. EDIT: Plugin writers can rely upon strings returned via SaveString() to be still available later on, so to be able to release strings allocated via plugins, there must be another function akin to SaveString() that hands over total ownership to Avisynth and such memory cannot be later accessed by the plugin. The original SaveString(), copies the string into Avisnth allocated memory. How to return Strings (I'm guessin that you dont need this, perhaps some do):- http://forum.doom9.org/showthread.ph...ng#post1633936 Creating an expanded system environment string + file is not I think such a big problem, (does not need to be implemented in core, although GetSystemEnv() could be a good idea). Code:
AVSValue __cdecl RT_GetSystemEnv(AVSValue args, void* user_data, IScriptEnvironment* env) { // char * myName="RT_GetSystemEnv: "; const char *s=args[0].AsString(); char * sysenv = getenv(s); if(sysenv==NULL) sysenv=""; return env->SaveString(sysenv); } this is string formatting as implemented in RT_Stats RT_string Code:
AVSValue __cdecl RT_String(AVSValue args, void* user_data, IScriptEnvironment* env) { char *myName="RT_String: "; const char *s = args[0].AsString(); // text int arrsz = args[1].ArraySize(); int esc = args[2].AsInt(1); if(esc < 0 || esc > 2) env->ThrowError("%sEsc range 0 -> 2 only",myName); enum { CHICKEN=64 }; // what size buffer we need ? int i,mem=strlen(s) + 1 + CHICKEN; for(i=0;i<arrsz;++i) { if(args[1][i].IsString()) { const char *st=args[1][i].AsString(); mem += strlen(st) + 1 + CHICKEN; } else { mem += 8 + CHICKEN; // no particular reason why so big, just chicken factor. } } char *pbuf = new char[(mem+1)*2]; if(pbuf==NULL) env->ThrowError("%sCannot allocate memory",myName); char *ptem=pbuf+(mem+1); // temp buffer const unsigned char * r= (const unsigned char *)s; char *p=pbuf; int c,ix=0; int t=0; // Parse text and insert variables while(c=*r) { if(c=='%') { ++r; if(*r=='\0') { *p++ ='%'; } else if(*r=='%') { *p++=*r++; // replace escaped double % with single } else { if(ix>=arrsz) { delete [] pbuf; env->ThrowError("%sExpecting data arg (%d)",myName,ix+1); } char *tp=ptem; *tp++='%'; if(*r=='-' || *r=='+' || *r=='0' || *r==' ' || *r=='#') // flags *tp++=*r++; if(*r=='*') { // int holds length t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; if(t!=3) { delete [] pbuf; env->ThrowError("%sUnsupported data type, Expecting Width as Int (%d)",myName,ix+1); } tp+=sprintf(tp,"%d",args[1][ix].IsInt()); ++r; // skip '*' ++ix; // next data } else { while(*r>='0' && *r<='9') { *tp++ = *r++; } } if(*r=='.') { *tp++ = *r++; // precision prefix if(*r=='*') { // int holds length t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; if(t!=3) { delete [] pbuf; env->ThrowError("%sUnsupported data type, Expecting Precision as Int (%d)",myName,ix+1); } tp+=sprintf(tp,"%d",args[1][ix].AsInt()); ++r; // skip '*' ++ix; // next data } else { while(*r>='0' && *r<='9') { *tp++ = *r++; } } } t=args[1][ix].IsBool() ?1: \ args[1][ix].IsString() ?2: \ args[1][ix].IsInt() ?3: \ args[1][ix].IsFloat() ?4: \ 0; // type if( (*r=='c' ) || (*r=='C' ) || // char as int (*r=='d' || *r=='i') || // int (*r=='o' || *r=='u' || *r=='x' || *r=='X')) { // unsigned int if(t!=3) { delete [] pbuf; env->ThrowError("%sType='%c', Expecting Int data (%d)",myName,*r,ix+1); } *tp++=*r++; *tp='\0'; p+=sprintf(p,ptem,args[1][ix].AsInt()); ++ix; // next data } else if(*r=='e' || *r=='E' || *r=='f' || *r=='g' || *r=='G') { // double if(t!=4&&t!=3) { delete [] pbuf; env->ThrowError("%sType='%c', Expecting Float (%d)",myName,*r,ix+1); } *tp++=*r++; *tp='\0'; p+=sprintf(p,ptem,args[1][ix].AsFloat()); ++ix; // next data } else if((*r=='s')||(*r=='S')) { // string if(t!=2&&t!=1) { delete [] pbuf; env->ThrowError("%sType='s', Expecting String (%d)",myName,ix+1); } *tp++=*r++; *tp='\0'; if(t==1) { // Bool p+=sprintf(p,ptem,args[1][ix].AsBool()?"True":"False"); } else { // String p+=sprintf(p,ptem,args[1][ix].AsString()); } ++ix; // next data } else { delete [] pbuf; env->ThrowError("%sUnknown format type '%c' (%d)",myName,*r,ix+1); } } } else if(c == '\\' && esc == 1) { ++r; c=*r; // abfnrtv switch (c) { case '\0' : *p++='\\'; break; // copy single backslash at end of string case '\\' : *p++=*r++; break; // replace double backslash with single backslash case 'n' : ++r; *p++='\n'; break; case 'r' : ++r; *p++='\r'; break; case 't' : ++r; *p++='\t'; break; case 'v' : ++r; *p++='\v'; break; case 'f' : ++r; *p++='\f'; break; case 'b' : ++r; *p++='\b'; break; case 'a' : ++r; *p++='\a'; break; default : *p++='\\'; *p++=*r++; break; // anything else we copy backslash and whatever follows } } else { *p++=*r++; } } *p=0; // nul term if(ix<arrsz) { delete [] pbuf; env->ThrowError("%sUnexpected data arg (%d)",myName,ix+1); } // dprintf("RT_String: MEM Estimate=%d : Actual=%d\n",mem+1,strlen(pbuf)); if(esc==2) { p=pbuf; r=(const unsigned char*)p; while(c=*r) { if(c=='\\') { ++r; c=*r; // abfnrtv switch (c) { case '\0' : *p++='\\'; break; // copy single backslash at end of string case '\\' : *p++=*r++; break; // replace double backslash with single backslash case 'n' : ++r; *p++='\n'; break; case 'r' : ++r; *p++='\r'; break; case 't' : ++r; *p++='\t'; break; case 'v' : ++r; *p++='\v'; break; case 'f' : ++r; *p++='\f'; break; case 'b' : ++r; *p++='\b'; break; case 'a' : ++r; *p++='\a'; break; default : *p++='\\'; *p++=*r++; break; // anything else we copy backslash and whatever follows } } else { *p++=*r++; } } *p=0; // nul term } AVSValue ret = env->SaveString(pbuf,p-pbuf); delete [] pbuf; return ret;
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 2nd November 2016 at 04:37. |
2nd November 2016, 12:23 | #2543 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,309
|
Quote:
A couple of hours I went on a bad direction. First I was thinking about using an existing feature: unnamed function parameters already accepted array-like parameter values, by the '+' specifier, e.g. in StackHorizontal, Spline or Select. Code:
{ "StackHorizontal", BUILTIN_FUNC_PREFIX, "cc+", StackHorizontal::Create }, { "Spline", BUILTIN_FUNC_PREFIX, "[x]ff+[cubic]b", Spline }, { "Select", BUILTIN_FUNC_PREFIX, "i.+", Select }, And the broader support in AVSValue mechanism (constructor, etc.) failed. When I wanted to return such an array value, e.g. store it in a variable, the content of the array disappeared outside the internal method, because after instancing a function or method the array member AVSValues were freed up. Another problem was that the existing array parameter type has special handling internally, the array content is "flattened" at some point. So I had to introduce a "real" script array type, now I call it "A" in order not to mix it with the existing 'a' internal array specifier. It is a vector<AVSValue> Because of the working mechanism of '+' parameter specifier, which is putting together a one-level array AVSValue (type=='a') for the functions, I introduced a # (since it resembles the + sign) specifier that results a new script-array (type=='A') instead of that. In my sample, BlankClip can have a real array, that can be used for a named parameter. The definition is like this (A= the new script array type) Code:
{ "BlankClip", BUILTIN_FUNC_PREFIX, "[]c*[length]i[width]i[height]i[pixel_type]s[fps]f[fps_denominator]i[audio_rate]i[channels]i[sample_type]s[color]i[color_yuv]i[clip]c[colors]A", Create_BlankClip }, Code:
{ "Array", BUILTIN_FUNC_PREFIX, ".#", ArrayCreate }, // # instead of +: creates script array { "IsArray", BUILTIN_FUNC_PREFIX, ".", IsArray2 }, { "ArrayGet", BUILTIN_FUNC_PREFIX, "Ai", ArrayGet }, { "ArrayGet", BUILTIN_FUNC_PREFIX, "As", ArrayGet }, { "ArraySize", BUILTIN_FUNC_PREFIX, "A", ArraySize }, Script function can check the script-array type with IsArray2 at the moment. At least I had a little insight to the parser, too. Anyhow, this part was nicely written, I hope I can easily change to [] for definition and indexing arrays. A little off-topic, but I relaxed with this problem after my weekend 85K trail running race Fun after fun. BTW did you know that there is built-in for and while? |
|
2nd November 2016, 13:20 | #2544 | Link | ||
Avisynth language lover
Join Date: Dec 2007
Location: Spain
Posts: 3,431
|
Quote:
Actually, on further thought, I'm not sure you really need # anyway. Can't the built-in Array() function be defined as ".*", and simply copy its arguments into an new type 'A' AVSValue? Quote:
In the early stages of development, I gave ultim permission to incorporate my GScript code into Avisynth+. |
||
3rd November 2016, 08:52 | #2545 | Link |
Registered User
Join Date: Jan 2008
Posts: 42
|
Hi all,
I recently swapped avisynth for avisynth+ Yesterday I modified one of my avs Scripts. Just added some new pics - basically three more ImageSource(...) But now I got: "Could not map shared memory into local space" in VirtualDub. Any ideas ? EDIT: I fall back to std. Avisynth 2.6.1 ... no problems anymore ... |
3rd November 2016, 12:49 | #2546 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,309
|
Quote:
- avs script - whether the script works alone w/o vdub (to rule out avs+ install problems, such as missing VS2015 update 3 redistributables) - type and size of the images - virtualdub version - avisynth+ version |
|
3rd November 2016, 13:07 | #2547 | Link | |
German doom9/Gleitz SuMo
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,751
|
Quote:
If I understood may24 correctly, the script worked well with a few images less loaded, but he crossed some threshold by adding some more sources? |
|
3rd November 2016, 13:16 | #2548 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,309
|
Quote:
I found that error message and all I could see that a MapViewOfFile after a successful CreateFileMapping has failed with a specific buffer size. |
|
3rd November 2016, 16:18 | #2551 | Link |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,308
|
Just when i finaly found it...
Thanks anyway. Now, i have to figure out why... why... why... the compiler don't want to compile minmax.h anymore, throwing me a lot of errors about the template T ! And i have to implement a test for detecting VS version, because i still want to be able to compile with VS2010, which the avx intrinsic don't allow anymore... But first, i have to be able to build with VS2015. I can workaround minmax with #define macros, but i would better understand why sudenly it don't want to build anymore... Last edited by jpsdr; 3rd November 2016 at 16:22. |
3rd November 2016, 16:25 | #2552 | Link | |
Registered User
Join Date: Jan 2014
Posts: 2,309
|
Quote:
PS, I have cleaned up the resizers a bit (after you found unused avx path test code) and added a real avx2 path, I bet it is still not as fast as zimg, but at least it got cleaner. Just have commited, check my MT repo. |
|
3rd November 2016, 16:55 | #2553 | Link | ||
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,308
|
Quote:
Quote:
zimg... If it works with direct full square ... (argh... don't remember the word now), it will be impossible to multi-thread by splitting. And a more general question. In image/video processing, the 1rst idea (at least, for me) which comes to mind when talking of multi-threading is splitting the picture, not multiply it. So, my question is : Why when introducing multi-threading in avisynth, nobody seems to have thought at the 1rst obvious idea, internaly MT the core filters by splitting the picture, instead of trying to run several GetFrame at the same time...? Filters that are pure memory transfert (like doubleweave, or TurnL/R) may not benefit of MT (there is more chances of memory bandwith ... again don't remember the correct word ... issue,struggle), but the others may. So... Why...? Last edited by jpsdr; 3rd November 2016 at 17:08. |
||
3rd November 2016, 22:02 | #2557 | Link |
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
No, jpsdr, you didn't suddenly come up with the obvious solution that nobody else has considered. There are reasons nobody uses slice based threading. The most obvious one is that if you're working with video you already have your input split up in neat discrete chunks for you: they're called frames. Splitting these chunks further just adds complexity for no real gain at all. I'm pretty sure it's also worse performance wise (for many reasons, especially for spatial filters where you might have to handle overlapping slices) but I haven't benchmarked it.
|
Thread Tools | Search this Thread |
Display Modes | |
|
|