Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 27th June 2016, 05:19   #1841  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
DitherTools isn't working well with MT.

Try this script repeatedly and it will crash in dither.dll, when all dither functions are set to either MT=1 or 2
Code:
Dither_convert_8_to_16()
SMDegrain(thsad=200, prefilter=2, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
Prefetch(8)
A solution I found is to set DitherPost to MT mode 3. However, it then causes a severe slowdown when using SMDegrain(prefilter=4) where performance goes from 2.2 to 1.6fps.

Without setting DitherPost to MT mode 3, I can't get rid of the crashes, so it would be nice for someone to investigate.
MysteryX is offline  
Old 27th June 2016, 05:51   #1842  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,746
Issues with DitherPost have been reported before; I remember the statement that it is sufficient to limit one function of this family if there are several dither function used, may not matter much which of them...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline  
Old 27th June 2016, 08:02   #1843  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by MysteryX View Post
DitherTools isn't working well with MT.
Do you have avstp.dll in your auto-load directory? If so, remove it and try again.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline  
Old 27th June 2016, 08:09   #1844  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by Groucho2004 View Post
Do you have avstp.dll in your auto-load directory? If so, remove it and try again.
No I don't have that
MysteryX is offline  
Old 27th June 2016, 08:37   #1845  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,297
Quote:
Originally Posted by Chikuzen View Post
@ultim or someone

I have a question about multi-threading.

Is it possible to implement a temporal IIR filter as MT_NICE_FILTER ?
I am affraid you have to be MT_SERIALIZED with this kind of filter, you can't even be MT_MULTI_INSTANCE. At least, from what i've understood.
But there is also a MT_MODE_COUNT, i don't know what it does.
jpsdr is offline  
Old 27th June 2016, 12:16   #1846  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Here's another bug. I tried using Deshaker.
http://www.guthspot.se/video/deshaker.htm

Here's a script that will run the first pass and generate a log file to be used for the second pass.
Code:
P="Encoder\"
LoadVirtualDubPlugin(P+"deshaker.vdf", "deshaker", preroll=0)
LoadPlugin(P+"LSMASHSource.dll")
file="Source.MOV"
LWLibavVideoSource(file, cache=True)
ConvertToRGB32()
deshaker("19|1|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|1|3|2|8|30|300|4|Preview_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")
PointResize(8,8)
To run the second pass, re-insert the deshaker line in your full script wherever you want and change the 2nd argument from 1 to 2 to run the 2nd pass from that log file.

With AviSynth 2.6 it works fine. With AviSynth+, I get a crash in the destructor. Whenever I close the preview I get a crash.

Deshaker doesn't seem to work with MT but a single thread tops all 8 CPU cores anyway.

Last edited by MysteryX; 27th June 2016 at 12:38.
MysteryX is offline  
Old 27th June 2016, 13:56   #1847  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
The VD plugin bug seems easy to fix so I thought I might take a look myself. Trying to compile AVS+ in VS2015, however, there is this error: redefinition of 'y'. This code really doesn't look right... it's using y for the loop and redefining y twice more within the loop. Pinterf I'll let you look into that one.

For now I'll rename the 2 redefinitions as y2.

Code:
static void af_horizontal_yuy2_c(BYTE* p, int height, int pitch, int width, int amount) {
  const int center_weight = amount*2;
  const int outer_weight = 32768-amount;
  for (int y = height; y>0; --y) 
  {
    BYTE yy = p[0];
    BYTE uv = p[1];
    BYTE vu = p[3];
    int x;
    for (x = 0; x < width-2; ++x) 
    {
      BYTE y = ScaledPixelClip(p[x*2+0] * center_weight + (yy + p[x*2+2]) * outer_weight);
      yy   = p[x*2+0];
      p[x*2+0] = y;
      BYTE w = ScaledPixelClip(p[x*2+1] * center_weight + (uv + p[x*2+5]) * outer_weight);
      uv   = vu;
      vu   = p[x*2+1];
      p[x*2+1] = w;
    }
    BYTE y     = ScaledPixelClip(p[x*2+0] * center_weight + (yy + p[x*2+2]) * outer_weight);
    yy       = p[x*2+0];
    p[x*2+0] = y;
    p[x*2+1] = ScaledPixelClip(p[x*2+1] * center_weight + (uv + p[x*2+1]) * outer_weight);
    p[x*2+2] = ScaledPixelClip(p[x*2+2] * center_weight + (yy + p[x*2+2]) * outer_weight);
    p[x*2+3] = ScaledPixelClip(p[x*2+3] * center_weight + (vu + p[x*2+3]) * outer_weight);

    p += pitch;
  }
}
MysteryX is offline  
Old 27th June 2016, 14:27   #1848  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
The VDub destructor bug isn't so easy to fix; but shouldn't be hard for someone who understands the code better.

Code:
  // VDubFilter.cpp
  void CallEndProc() {
    if (fd->endProc) {
      int result = fd->endProc(&fa, &g_filterFuncs);
      if (result != 0) {
        throw AvisynthError("VirtualdubFilterProxy: error calling endProc");
      }
    }
  }
Basically, Deshaker's EndProc is trying to access "something" within fa that is no longer accessible at that time.

Commenting both of these lines does solve the problem, but I doubt it is the right fix
Code:
// in FilterAdd
fm->env->AtExit(FreeFilterDefinition, fd);

// in LoadVirtualdubPlugin
env->AtExit(FreeFilterModule, fm);
The destructor needs to be called before these 2 functions.

I'll let you fix it before I start adding a "release timer" to fix the sequence

Last edited by MysteryX; 27th June 2016 at 14:51.
MysteryX is offline  
Old 27th June 2016, 14:58   #1849  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by MysteryX View Post
For now I'll rename the 2 redefinitions as y2.
In my source it is fixed already in focus.cpp.
https://github.com/pinterf/AviSynthP...ters/focus.cpp
Which branch or version are you working on?
pinterf is offline  
Old 27th June 2016, 15:03   #1850  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by MysteryX View Post
Trying to compile AVS+ in VS2015, however, there is this error: redefinition of 'y'.
Code:
static void af_horizontal_yuy2_c(BYTE* p, int height, int pitch, int width, int amount) {
  const int center_weight = amount*2;
  const int outer_weight = 32768-amount;
  for (int y = height; y>0; --y) 
  {
    BYTE yy = p[0];
    BYTE uv = p[1];
    BYTE vu = p[3];
    int x;
    for (x = 0; x < width-2; ++x) 
    {
      BYTE y = ScaledPixelClip(p[x*2+0] * center_weight + (yy + p[x*2+2]) * outer_weight);
      yy   = p[x*2+0];
      p[x*2+0] = y;
      BYTE w = ScaledPixelClip(p[x*2+1] * center_weight + (uv + p[x*2+5]) * outer_weight);
      uv   = vu;
      vu   = p[x*2+1];
      p[x*2+1] = w;
    }
    BYTE y     = ScaledPixelClip(p[x*2+0] * center_weight + (yy + p[x*2+2]) * outer_weight);
    yy       = p[x*2+0];
    p[x*2+0] = y;
    p[x*2+1] = ScaledPixelClip(p[x*2+1] * center_weight + (uv + p[x*2+1]) * outer_weight);
    p[x*2+2] = ScaledPixelClip(p[x*2+2] * center_weight + (yy + p[x*2+2]) * outer_weight);
    p[x*2+3] = ScaledPixelClip(p[x*2+3] * center_weight + (vu + p[x*2+3]) * outer_weight);

    p += pitch;
  }
}
Earlier compilers didn't complain about re-definitions if they had a different scope. That led to sloppy programming.
The above function is just horrible, it also defines a variable with different types (BYTE y, int y).
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline  
Old 27th June 2016, 15:08   #1851  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by MysteryX View Post
The VDub destructor bug isn't so easy to fix; but shouldn't be hard for someone who understands the code better.

Code:
  // VDubFilter.cpp
  void CallEndProc() {
    if (fd->endProc) {
      int result = fd->endProc(&fa, &g_filterFuncs);
      if (result != 0) {
        throw AvisynthError("VirtualdubFilterProxy: error calling endProc");
      }
    }
  }
Basically, Deshaker's EndProc is trying to access "something" within fa that is no longer accessible at that time.

Commenting both of these lines does solve the problem, but I doubt it is the right fix
Code:
// in FilterAdd
fm->env->AtExit(FreeFilterDefinition, fd);

// in LoadVirtualdubPlugin
env->AtExit(FreeFilterModule, fm);
The destructor needs to be called before these 2 functions.

I'll let you fix it before I start adding a "release timer" to fix the sequence
Did you checked the last commit? It is not yet in the last pfmod release, there were changes integrated from classic avs regarding "Fix VDub plugin race on cleanup"
pinterf is offline  
Old 27th June 2016, 15:10   #1852  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Pinterf, I took your branch.

The destruction sequence is the problem. This is something to be very careful about so it's better if I don't touch it; it touches the core of AVS and will affect every plugin.

Code:
ScriptEnvironment::~ScriptEnvironment() {

  closing = true;

  // Before we start to pull the world apart
  // give every one their last wish.
  at_exit.Execute(this);
Here it wants to execute these functions before pulling the world apart, and with VDubFilter we must call the destructor before calling these functions.
MysteryX is offline  
Old 27th June 2016, 15:17   #1853  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by pinterf View Post
Did you checked the last commit? It is not yet in the last pfmod release, there were changes integrated from classic avs regarding "Fix VDub plugin race on cleanup"
I see, I forgot to manually select the right branch after downloading the code.

Now it compiles fine right away. The VDub bug is still the same. Good luck Pinterf.

Last edited by MysteryX; 27th June 2016 at 15:25.
MysteryX is offline  
Old 27th June 2016, 21:16   #1854  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Let's look at this silly code fragment
Code:
lsmashvideosource(film, stacked=true, format="YUV444P16")  
# Open Prores422 10 bit as YUV444P16
ConvertStackedToNative()
Spline64Resize(400,300)
Crop(9,7,-9,-7)
FlipHorizontal().FlipVertical()
TurnLeft().TurnRight().Turn180()
PointResize(3200,2400)
GrayScale()
BilinearResize(1600,1200)
ConvertNativeToStacked()
source1=last
...
Nothing special, we are in 2016, UHD era, with 14-16 bit linear uncompressed raw in cameras. I was a bit surprized when this very basic 16 bit workflow started to work on my bench. Funny last minute techdemo from Avisynth. A kind of Second Reality
For everything else there's DitherTools.
pinterf is offline  
Old 27th June 2016, 22:51   #1855  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 595
Code:
lsmashvideosource(film, stacked=false, format="YUV444P16")  
# Open Prores422 10 bit as YUV444P16
Spline64Resize(400,300)
...
Why do you take a side trip and waste your time?
You can open the source as NativeFormat
__________________
my repositories

Last edited by Chikuzen; 27th June 2016 at 22:58.
Chikuzen is offline  
Old 27th June 2016, 22:52   #1856  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,746
Quote:
Originally Posted by pinterf View Post
A kind of Second Reality
You might know that Smash Designs created a C=64 version of the Future Crew PC VGA demo. Maybe that's how we used to feel with only 8 bits per component...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline  
Old 27th June 2016, 23:04   #1857  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by Chikuzen View Post
Code:
lsmashvideosource(film, stacked=false, format="YUV444P16", dr=true)  
# Open Prores422 10 bit as YUV444P16
Spline64Resize(400,300)
...
Why do you take a side trip and waste your time?
You can open the source as NativeFormat
We discussed this in #avs-plus@irc.rizon.net

Currently lsmashvideosource without stacked will out in interleavexbit not in NativeFormat, so this will need something like ConvertInterleaved16ToNative() and ConvertNativeToInterleaved16()

Interleaved16 will be like the clip that out from dither_out()

more clearly details in flash3kyuu_deband.txt, input_mode and output_mode and stacked bool in f3kdb_dither
__________________
See My Avisynth Stuff

Last edited by real.finder; 27th June 2016 at 23:33.
real.finder is offline  
Old 28th June 2016, 03:33   #1858  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
VDub destructor crash fixed

Fixed crash on exit due to FreeFilterModule being called before VirtualdubFilterProxy's destructor
Code:
diff --git a/plugins/VDubFilter/VDubFilter.cpp b/plugins/VDubFilter/VDubFilter.cpp
index df512ee..7d97fef 100644
--- a/plugins/VDubFilter/VDubFilter.cpp
+++ b/plugins/VDubFilter/VDubFilter.cpp
@@ -365,10 +365,11 @@ typedef struct FilterDefinition {
 
 class FilterDefinitionList {
 public:
+	FilterModule* fm;
   FilterDefinition* fd;
   FilterDefinitionList* fdl;
 
-  FilterDefinitionList(FilterModule* fm, FilterDefinition* _fd) : fd(_fd), fdl(fm->fdl) { };
+  FilterDefinitionList(FilterModule* _fm, FilterDefinition* _fd) : fm(_fm), fd(_fd), fdl(fm->fdl) { };
 };
 
 //////////
@@ -575,6 +576,7 @@ public:
 class VirtualdubFilterProxy : public GenericVideoFilter {
   PVideoFrame src, dst, last;
   VFBitmap vbSrc, vbDst, vbLast;
+  FilterDefinitionList* const fdl;
   FilterDefinition* const fd;
   FilterStateInfo fsi;
   FilterActivation fa;
@@ -602,9 +604,11 @@ class VirtualdubFilterProxy : public GenericVideoFilter {
   }
 
 public:
-  VirtualdubFilterProxy(PClip _child, FilterDefinition* _fd, AVSValue args, IScriptEnvironment* env)
-    : GenericVideoFilter(_child), fd(_fd), fa(vbDst, vbSrc, &vbLast)
+  VirtualdubFilterProxy(PClip _child, FilterDefinitionList* _fdl, AVSValue args, IScriptEnvironment* env)
+    : GenericVideoFilter(_child), fdl(_fdl), fd(_fdl->fd), fa(vbDst, vbSrc, &vbLast)
   {
+	if (!fd)
+		env->ThrowError("VirtualdubFilterProxy: No FilterDefinition structure!");
     if (!vi.IsRGB32())
       throw AvisynthError("VirtualdubFilterProxy: only RGB32 supported for VirtualDub filters");
 
@@ -763,16 +767,29 @@ public:
 
   ~VirtualdubFilterProxy() {
     CallEndProc();
+	FreeFilterModule(fdl->fm);
     if (vbSrc.hdc)
       ReleaseDC(NULL, vbSrc.hdc);
   }
 
+  void __cdecl FreeFilterModule(FilterModule* fm) {
+	  for (FilterDefinitionList* fdl = fm->fdl; fdl; fdl = fdl->fdl) {
+		  delete fdl->fd;
+		  fdl->fd = 0;
+	  }
+
+	  fm->deinitProc(fm, &g_filterFuncs);
+	  FreeLibrary(fm->hInstModule);
+	  if (fm->prev)
+		  fm->prev->next = fm->next;
+	  if (fm->next)
+		  fm->next->prev = fm->prev;
+	  delete fm;
+  }
+
   static AVSValue __cdecl Create(AVSValue args, void* user_data, IScriptEnvironment* env) {
     FilterDefinitionList* fdl = (FilterDefinitionList*)user_data;
-
-    if (!fdl->fd) env->ThrowError("VirtualdubFilterProxy: No FilterDefinition structure!");
-
-    return new VirtualdubFilterProxy(args[0].AsClip(), fdl->fd, args, env);
+    return new VirtualdubFilterProxy(args[0].AsClip(), fdl, args, env);
   }
 };
 
@@ -803,24 +820,6 @@ FilterDefinition *VDcall FilterAdd(FilterModule *fm, FilterDefinition *pfd, int
 }
 
 
-void __cdecl FreeFilterModule(void* user_data, IScriptEnvironment* env) {
-  FilterModule* fm = (FilterModule*)user_data;
-
-  for (FilterDefinitionList* fdl = fm->fdl; fdl; fdl = fdl->fdl) {
-    delete fdl->fd;
-    fdl->fd = 0;
-  }
-
-  fm->deinitProc(fm, &g_filterFuncs);
-  FreeLibrary(fm->hInstModule);
-  if (fm->prev)
-    fm->prev->next = fm->next;
-  if (fm->next)
-    fm->next->prev = fm->prev;
-  delete fm;
-}
-
-
 AVSValue __cdecl LoadVirtualdubPlugin(AVSValue args, void*, IScriptEnvironment* env) {
   const char* const szModule = args[0].AsString();
   const char* const avisynth_function_name = args[1].AsString();
@@ -879,7 +878,6 @@ AVSValue __cdecl LoadVirtualdubPlugin(AVSValue args, void*, IScriptEnvironment*
     fm->next->prev = fm;
 
   env->SetGlobalVar("$LoadVirtualdubPlugin$", (const char*)fm);
-  env->AtExit(FreeFilterModule, fm);
 
   return AVSValue();
 }
Git isn't allowing me to push the changes; is there something I'm doing wrong or you just have to add those changes yourself?
Code:
git.exe push -v --progress "origin" master:MT-pfmod

remote: Permission to pinterf/AviSynthPlus.git denied to mysteryx93.
fatal: unable to access 'https://github.com/pinterf/AviSynthPlus.git/': The requested URL returned error: 403
Pushing to https://github.com/pinterf/AviSynthPlus.git

Last edited by MysteryX; 28th June 2016 at 03:36.
MysteryX is offline  
Old 28th June 2016, 05:41   #1859  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,416
Quote:
Originally Posted by MysteryX View Post
Git isn't allowing me to push the changes; is there something I'm doing wrong or you just have to add those changes yourself?
Code:
git.exe push -v --progress "origin" master:MT-pfmod

remote: Permission to pinterf/AviSynthPlus.git denied to mysteryx93.
fatal: unable to access 'https://github.com/pinterf/AviSynthPlus.git/': The requested URL returned error: 403
Pushing to https://github.com/pinterf/AviSynthPlus.git
You have to have your own account on Github, use Github's tools* to spawn your own AviSynthPlus repo, and then push up to your own account. You can then open a pull request through Github to have the change integrated.

*well, technically you don't need to use Github's tools to do it so long as the SSH remotes are correct and you've authenticated your keys on your Github account, but it won't show up on the Network graph.
qyot27 is offline  
Old 28th June 2016, 06:20   #1860  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by qyot27 View Post
You have to have your own account on Github, use Github's tools* to spawn your own AviSynthPlus repo, and then push up to your own account. You can then open a pull request through Github to have the change integrated.

*well, technically you don't need to use Github's tools to do it so long as the SSH remotes are correct and you've authenticated your keys on your Github account, but it won't show up on the Network graph.
OK I created a new Fork into my account, but MT-pfmod doesn't show up in the list of branches
MysteryX is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:21.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.