Avisynth+

[Reel.Deel]

A while back NightSprinter reportred this problem with crt_display while using MT.

Originally Posted by NightSprinter

This is odd, running a Core i7 920 (4 cores, 8 threads) with everything done as posted on the tutorial. However, when I enable prefetch, I get an error along the lines of "I don't know what 'zb' means". I'm using the latest AVS+ installer from the wiki page, and the filter script chain I'm using is the CRT simulation filter that's on this forum as well. I'm including the script in .txt format

I recently tried with this script and get a similar error "I don't know what 'z3' means". Disable MT and the script works just fine.

Import("MTModes.avsi")
Import("crt_display.avsi")

SetFilterMTMode("DEFAULT_MT_MODE", 2)

ColorBars()
Trim(0,500)

ar = 10.0 / 11.0
crt_display (2*ar, 2, ppp=1*ar, blurh=2.0)

Prefetch(4)

[jpsdr]

Question about the multi-threading parts.
In case filter is MT_NICE, or MT_MULTI_INSTANCE. Especialy for this last case, are the multiple instances of the filter real different instances, or should i worry about eventualy copy constructor issues ?

[pinterf]

Strange.

O.K. (Default MT = 3 + Eval + Function Call)

SetFilterMTMode("DEFAULT_MT_MODE", 3)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
	src
	r  = ShowRed ("Y8")
	g  = ShowGreen ("Y8")
	b  = ShowBlue ("Y8")
	Eval("Interleave (r, g, b)") 
}

Fail, (Default MT = 2 + Eval + Function call)
# I dont' know what 'r' means

SetFilterMTMode("DEFAULT_MT_MODE", 2)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
	src
	r  = ShowRed ("Y8")
	g  = ShowGreen ("Y8")
	b  = ShowBlue ("Y8")
	Eval("Interleave (r, g, b)") 
}

O.K. (Default MT = 2 + NO Eval + Function call)

SetFilterMTMode("DEFAULT_MT_MODE", 2)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
	src
	r  = ShowRed ("Y8")
	g  = ShowGreen ("Y8")
	b  = ShowBlue ("Y8")
	#Eval("Interleave (r, g, b)")
	Interleave (r, g, b) # ok
}

O.K (Default MT = 2 + Eval + no function call)

SetFilterMTMode("DEFAULT_MT_MODE", 2)
ColorBars()
Trim(0,2500)

#test()

r  = ShowRed ("Y8")
g  = ShowGreen ("Y8")
b  = ShowBlue ("Y8")
Eval("Interleave (r, g, b)")

Prefetch(1)

[MysteryX]

Originally Posted by jpsdr

Question about the multi-threading parts.
In case filter is MT_NICE, or MT_MULTI_INSTANCE. Especialy for this last case, are the multiple instances of the filter real different instances, or should i worry about eventualy copy constructor issues ?

With MT_MULTI_INSTANCE, they are completely separate instances. The only way the various instances can communicate with each other is through global variables.

[bilditup1]

Originally Posted by bilditup1

I will continue doing tests and reporting back, especially with respect to how well 1080i works and regarding matrix operations.

Tested 1080i60 with downscaling to 360p30 and matrix conversion from 709 to 601, and getting some weird results. Download screenshots and scripts here. I think the file naming/numbering is intuitive and you guys should be able to figure it out.

Four passes:

• just straight downscaling (Spline36) and deinterlacing, no matrix operations. This yielded 10.17fps (including x264 compression).
• using ColorMatrix for 709->601. This was done in almost real-time, yielding 10.06fps.
• using Dither for 709->601 and downscaling (still using Spline36 kernel). Slowed things down significantly, only got 6.32fps, which I guess is to be expected.
• using AviSynthShader for 709->601 and downscaling (using Bicubic kernel). Slowed things down even more, only got 5.85fps - again, to be expected with a lowly HD4600 - and MeGUI crashed at 99.83% (there's a screenshot of this in the zip).
  • I had intended to do comparisons with this last also, after muxing the 264. However I had neglected to delete the original encoding job from the MeGUI queue, and as a result the original file got destroyed once the queue commenced - instead of the muxing job, it tried to do the encode again, since when MeGUI itself totally crashes, the job that causes the crash is never marked as done, and shows up as 'waiting' in the queue - and I didn't want to wait for the encode to finish before I posted here. In any case, it's not really relevant to me right now, so I left it be.

A few observations:

• CPU, when I bothered to check, was running at at least 60%.
• I had to disable MT for the comparison clips (using FFMS2), otherwise the overlays with the frame-numbers and such got garbled and destroyed.
• 1080i in and of itself does not appear to be an issue.
• But, from these clips it looks like ColorMatrix might in fact still be causing problems?
  • In frame 0, the scrolling text is corrupted on the bottom right, only in the ColorMatrix one (not in the others or the source).
  • At frame 39287, there are big green horizontal bars across the upper middle and bottom of the frame.
  • Both of these only show up in an actual encode, not in the mere preview (using AvsPmod, for example).
  • There might be other problems, but they were too quick to catch.
• Dither has some weirdness, too. The first two frames look like some interlaced abortion of some kind. But, again, this only happens in an actual encode.
• All of them had trouble at 29300...but this turned out to be in the source. I've included it because it's a pretty weird artifact and I'm curious as to what could cause it.
• Due to the nature of these problems - crashing/not crashing with no modifications to the avs, problems showing up in the encode but not the preview, &c - I'm beginning to suspect that perhaps my overclock is too aggressive. So I think you may need to take all of these with a grain of salt. I'll be fiddling with that and trying again once I get back in town.

[Groucho2004]

Originally Posted by bilditup1

Tested 1080i60 with downscaling to 360p30 and matrix conversion from 709 to 601, and getting some weird results. Download screenshots and scripts here. I think the file naming/numbering is intuitive and you guys should be able to figure it out.

Four passes:

• just straight downscaling (Spline36) and deinterlacing, no matrix operations. This yielded 10.17fps (including x264 compression).
• using ColorMatrix for 709->601. This was done in almost real-time, yielding 10.06fps.
• using Dither for 709->601 and downscaling (still using Spline36 kernel). Slowed things down significantly, only got 6.32fps, which I guess is to be expected.
• using AviSynthShader for 709->601 and downscaling (using Bicubic kernel). Slowed things down even more, only got 5.85fps - again, to be expected with a lowly HD4600 - and MeGUI crashed at 99.83% (there's a screenshot of this in the zip).
  • I had intended to do comparisons with this last also, after muxing the 264. However I had neglected to delete the original encoding job from the MeGUI queue, and as a result the original file got destroyed once the queue commenced - instead of the muxing job, it tried to do the encode again, since when MeGUI itself totally crashes, the job that causes the crash is never marked as done, and shows up as 'waiting' in the queue - and I didn't want to wait for the encode to finish before I posted here. In any case, it's not really relevant to me right now, so I left it be.

A few observations:

• CPU, when I bothered to check, was running at at least 60%.
• I had to disable MT for the comparison clips (using FFMS2), otherwise the overlays with the frame-numbers and such got garbled and destroyed.
• 1080i in and of itself does not appear to be an issue.
• But, from these clips it looks like ColorMatrix might in fact still be causing problems?
  • In frame 0, the scrolling text is corrupted on the bottom right, only in the ColorMatrix one (not in the others or the source).
  • At frame 39287, there are big green horizontal bars across the upper middle and bottom of the frame.
  • Both of these only show up in an actual encode, not in the mere preview (using AvsPmod, for example).
  • There might be other problems, but they were too quick to catch.
• Dither has some weirdness, too. The first two frames look like some interlaced abortion of some kind. But, again, this only happens in an actual encode.
• All of them had trouble at 29300...but this turned out to be in the source. I've included it because it's a pretty weird artifact and I'm curious as to what could cause it.
• Due to the nature of these problems - crashing/not crashing with no modifications to the avs, problems showing up in the encode but not the preview, &c - I'm beginning to suspect that perhaps my overclock is too aggressive. So I think you may need to take all of these with a grain of salt. I'll be fiddling with that and trying again once I get back in town.

You are posting in the wrong thread. None of this is related to AVS+.

[bilditup1]

Originally Posted by Groucho2004

You are posting in the wrong thread. None of this is related to AVS+.

It most certainly is. The latest MT_modes list still has warnings about processing 1080i and about using the ColorMatrix plugin. These warnings have been there for years and nobody has bothered to verify them in ages. See the discussion starting here.

[Groucho2004]

Originally Posted by bilditup1

It most certainly is.

Originally Posted by bilditup1

I'm beginning to suspect that perhaps my overclock is too aggressive.

Even if it were, do really think anyone will bother to look at your results after the above statement?

[bilditup1]

Originally Posted by Groucho2004

Even if it were, do really think anyone will bother to look at your results after the above statement?

I have only vague suspicions though. If this is a stability issue I have not seen other symptoms. BOINC runs all the time without errors, programs open the way they should, no BSoDs or random restarts. I included that disclaimer for completion's sake.

[ryrynz]

Originally Posted by Groucho2004

You are posting in the wrong thread. None of this is related to AVS+.

Did you really need to quote that entire unrelated post? Lol.

Originally Posted by bilditup1

I have only vague suspicions though. If this is a stability issue I have not seen other symptoms. BOINC runs all the time without errors, programs open the way they should, no BSoDs or random restarts. I included that disclaimer for completion's sake.

That doesn't matter.. Instability can often go unseen. Go back to defaults and retest.

[bilditup1]

Originally Posted by ryrynz

That doesn't matter.. Instability can often go unseen. Go back to defaults and retest.

...yes...hence my suspicions...I was just saying that there wasn't a traditional smoking gun just yet. I will in fact be retesting a few times and at stock, plus hopefully on another machine entirely if I can manage it.
I will also split off the rest of this discussion to its own thread. Not here to make trouble - just trying to make a positive contribution.

[MysteryX]

Ever since I added Dither Tools in this script, sometimes it works and sometimes it crashes the encoder or MPC-HC or freezes VirtualDub.

I tried running the same under AviSynth 2.6 MT and opened it 10-15 times without any crash or freeze. I'm guessing the most likely cause is a difference in MT modes for one of the DitherTools filters.

P="Encoder\"
Import(P+"AviSynthMT.avsi")
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"Dither.dll")
Import(P+"Dither.avsi")
LoadPlugin(P+"KNLMeansCL.dll")
LoadPlugin(P+"svpflow1.dll")
LoadPlugin(P+"svpflow2.dll")
Import(P+"InterFrame2.avsi")
Import(P+"ResizeX.avsi")
LoadPlugin(P+"Shaders\Shader.dll")
Import(P+"Shaders\Shader.avsi")

file="VCD.mpg"
LWLibavVideoSource(file, cache=False, threads=1)
AudioDub(LWLibavAudioSource(file, cache=False))
Crop(0, 0, -8, -0)
Dither_convert_8_to_16()
Dither_resize16(Width, Height/2, kernel="Spline36", csp="YV24")
KNLMeansCL(D=1, A=2, h=2.1, cmode=true, device_type="GPU", device_id=0, lsb_inout=true)
SuperResXBR(5, 1, 0.15, XbrStr=2.7, XbrSharp=1.3, MatrixIn="601", lsb_in=true, lsb_out=true)
Dither_resize16(Width, Height/2, kernel="Spline36", csp="YV12", invks=true)
DitherPost()
InterFrame(Cores=8, Tuning="Smooth", NewNum=60, NewDen=1, GPU=true)
SuperResXBR(3, 1, 0.15, XbrStr=2.7, XbrSharp=1.3, fWidth=1012, fHeight=778, fKernel="Bicubic", fB=0, fC=.75)
ResizeX(1004, 768, 0, 5, -8, -5)
Prefetch(8)

By the way, LWLibavVideoSource runs with MT mode 3 which means there is a single instance? In that case, I don't need to specify 1 thread and I should just let it manage its own threats, correct?

[MysteryX]

I have set all DitherTools functions to MT mode 2. The first time I opened it, it freezed again, but afterwards I couldn't get it to freeze again. Dither Tools will require more testing for its MT compatibility/stability level.

By the way, at the top of the MT modes list, there should be a link to where the list actually came from and how to update it. When I want to refer changes about it, I don't even remember where it came from and who wrote it.

[pinterf]

This "I dont' know what 'r' means" is not an easy bug hunt.
At least now I know how to reproduce it.

All these conditions must be me met:

1. MT is on (Prefetch(x))
2. Eval is set to MT_MULTI_INSTANCE
   either by SetFilterMTMode("Eval", MT_MULTI_INSTANCE)
   or
   SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
3. Eval is within a function
4. Within Eval the last filter is NonCacheableVideoFilter
   (responds != 0 to SetCacheHints(CACHE_DONT_CACHE_ME, 0))
   Such filters: Trim, DuplicateFrame, Interleave, etc.
5. The last filter in Eval in the function is NICE_FILTER

There were quite a few permutations of these possible cases.

Sample scripts.

Fail. "I dont' know what 'r' means"
Eval MT mode 2 + function call + last filter in Eval is non cacheable and NICE_FILTER (Interleave)

SetFilterMTMode("Eval", 2)
#or SetFilterMTMode("DEFAULT_MT_MODE", 2)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
  src
  r  = ShowRed ("Y8")
  g  = ShowGreen ("Y8")
  b  = ShowBlue ("Y8")
  Eval("Interleave (r, g, b)") 
}

O.K. - MT mode 2 + last filter in Eval is cacheable (StackVertical)

SetFilterMTMode("Eval", 2)
#or SetFilterMTMode("DEFAULT_MT_MODE", 2)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
  src
  r  = ShowRed ("Y8")
  g  = ShowGreen ("Y8")
  b  = ShowBlue ("Y8")
  Eval("a=Interleave (r, g, b)
         StackVertical(a,a)") 
}

O.K. - Eval MT mode 1

SetFilterMTMode("Eval", 1)
ColorBars()
Trim(0,2500)

test()

Prefetch(1)

Function test(clip src)
{
  src
  r  = ShowRed ("Y8")
  g  = ShowGreen ("Y8")
  b  = ShowBlue ("Y8")
  Eval("Interleave (r, g, b)") 
}

[pinterf]

Originally Posted by MysteryX

I have set all DitherTools functions to MT mode 2. The first time I opened it, it freezed again, but afterwards I couldn't get it to freeze again. Dither Tools will require more testing for its MT compatibility/stability level.

By the way, at the top of the MT modes list, there should be a link to where the list actually came from and how to update it. When I want to refer changes about it, I don't even remember where it came from and who wrote it.

Can it be that the memory is simply not enough? What happens is you downsize the original video to 320x200? Or use only Prefetch(4)?

[Reel.Deel]

Originally Posted by pinterf

This "I dont' know what 'r' means" is not an easy bug hunt.
At least now I know how to reproduce it...

I found another scenario where it happens but this time without MT.

This works, but if I change SMDegrain() to SMDegrain(globals=1) I get "I don't know what 'bv2' means". I do not know how to reproduce this without SMDegrain but if I find an alternative I'll let you know.

ColorBars(pixel_type="YV12")
SMDegrain ()

[MysteryX]

Setting DitherTools to MT=3 appears to work so far. Will have to test which filter breaks with MT=1 or MT=2.

Memory usage is only ~900MB.

[MysteryX]

DitherPost appears to be the one crashing. This works so far.

SetFilterMTMode("DitherPost",          MT_SERIALIZED)

The other DitherTools filters appear to work with MT_NICE_FILTER although I haven't tested all of them.

[pinterf]

Originally Posted by MysteryX

DitherPost appears to be the one crashing. This works so far.

SetFilterMTMode("DitherPost",          MT_SERIALIZED)

The other DitherTools filters appear to work with MT_NICE_FILTER although I haven't tested all of them.

Setting one filter to a stricter MT mode affects the timing and parallel execution conditions of the other filters in the chain, too. So unfortunately you cannot say that DitherPost is culprit. We can only state that bottlenecking MT at DitherPost solves this problem.

[pinterf]

Originally Posted by Reel.Deel

I found another scenario where it happens but this time without MT.

This works, but if I change SMDegrain() to SMDegrain(globals=1) I get "I don't know what 'bv2' means". I do not know how to reproduce this without SMDegrain but if I find an alternative I'll let you know.

ColorBars(pixel_type="YV12")
SMDegrain ()

Same happens to avisynth 2.6, because globals=1 is for read globals and Process

bv2 is a variable that a former processing phase outputs as global

You have to run a first phase in order to bv2 and the other variables exist.

ColorBars(pixel_type="YV12")
SMDegrain (globals=3) # 3 - Output globals only, don't process 
SMDegrain (globals=1) # 1 - Read globals and Process