Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 11th May 2016, 06:00   #1541  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
I have this script. When I open in MPC-HC x86, it loads fine and plays slowly.

When I seek, however, 50-80% of the times it completely freezes the player.

Code:
P="Encoder\"
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"KNLMeansCL.dll")
LoadPlugin(P+"Shaders\Shader.dll")
Import(P+"Shaders\Shader.avsi")
LoadPlugin(P+"svpflow1.dll")
LoadPlugin(P+"svpflow2.dll")
Import(P+"InterFrame2.avsi")
Import(P+"ResizeX.avsi")

SetFilterMTMode("DEFAULT_MT_MODE",2)
SetFilterMTMode("LWLibavVideoSource",3)
SetFilterMTMode("LWLibavAudioSource",3)
SetFilterMTMode("KNLMeansCL",3)
file="Input.mpg"
LWLibavVideoSource(file, cache=false, threads=1)
AudioDub(LWLibavAudioSource(file, cache=false))
Crop(0, 0, -8, -0)
ConvertToYV24()
KNLMeansCL(D=1, A=2, h=2.1, cmode=true, device_type="GPU", device_id=0)
SuperResXBR(2, 0.6, 0, XbrStr=2, XbrSharp=1.2, MatrixIn="601")
ConvertToYV12()
InterFrame(Cores=8, Tuning="Smooth", NewNum=60, NewDen=1, GPU=true)
SuperResXBR(2, 0.6, 0, XbrStr=2, XbrSharp=1.2, fWidth=1012, fHeight=778)
ResizeX(1004, 768, 0, 5, -8, -5)
Prefetch(8)
MysteryX is offline  
Old 11th May 2016, 06:35   #1542  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Isn't the freezing due to cache=false?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline  
Old 11th May 2016, 07:35   #1543  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by Boulder View Post
Isn't the freezing due to cache=false?
No.

If I replace with AviSource, then it freezes all the time.

If I remove InterFrame, then it loads and seeks fine; which is weird because SVP itself is working perfectly.

I made sure to copy the latest SVP DLLs from SVP's plugin folder into my Encoder folder, but that's not helping.
MysteryX is offline  
Old 11th May 2016, 08:27   #1544  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by MysteryX View Post
I have this script. When I open in MPC-HC x86, it loads fine and plays slowly.
When I seek, however, 50-80% of the times it completely freezes the player.
1.) What is the resolution of your input file? In Avs+ 32bit, using 1280x720 source I have memory problems.
Seems that when other (non-avsplus-core) processes occupy significant amount of memory after the script starts, and the total nears to 3GB, could not balance and free-up enough memory.

2.) My source is 384x288.With this original size there are no memory problems.
Your script still doesn't run for me even with prefetch(1) (from avsmeter. The usual "reading frames")
If I set only one of these to mode 3
Code:
SetFilterMTMode("ConvertFromShader", 2)
SetFilterMTMode("ConvertToShader", 2)
SetFilterMTMode("ExecuteShader", 2)
SetFilterMTMode("Shader", 2)
SetFilterMTMode("SVSmoothFps", 3)
then it works.
pinterf is offline  
Old 11th May 2016, 08:47   #1545  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,462
Quote:
Originally Posted by MysteryX View Post
Memory usage is 150% higher than the x86 version and performance is 15% slower even without Interframe

Definitely no gain here.

Thanks for the testing.

I myself have decided not to bother with AviSynth 64-bit any more. Last time I did (which, granted, was a while back; but your data doesn't look promising), it was crashing all over the place. We got VapourSynth now.
__________________
Gorgeous, delicious, deculture!
asarian is offline  
Old 11th May 2016, 09:59   #1546  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by pinterf View Post
1.) What is the resolution of your input file? In Avs+ 32bit, using 1280x720 source I have memory problems.
Seems that when other (non-avsplus-core) processes occupy significant amount of memory after the script starts, and the total nears to 3GB, could not balance and free-up enough memory.

2.) My source is 384x288.With this original size there are no memory problems.
Your script still doesn't run for me even with prefetch(1) (from avsmeter. The usual "reading frames")
If I set only one of these to mode 3
Code:
SetFilterMTMode("ConvertFromShader", 2)
SetFilterMTMode("ConvertToShader", 2)
SetFilterMTMode("ExecuteShader", 2)
SetFilterMTMode("Shader", 2)
SetFilterMTMode("SVSmoothFps", 3)
then it works.
Very accurate assessment.
This Shader thingy is monstrous and uses insane amounts of memory.
The SVP stuff is quite buggy and also uses the GPU and should therefore run in mode 3, I suppose. Not sure if "SetFilterMTMode("SVSmoothFps", 3)" is sufficient, though.
Groucho2004 is offline  
Old 11th May 2016, 10:39   #1547  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
No change if I set GPU=false in Interframe parameter.
I think that we face an unlucky timing, where the problem of a missing critical section or causes the problem.
I will definitely look into it because I like solving puzzles.
pinterf is offline  
Old 11th May 2016, 10:53   #1548  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by asarian View Post
Thanks for the testing.

I myself have decided not to bother with AviSynth 64-bit any more. Last time I did (which, granted, was a while back; but your data doesn't look promising), it was crashing all over the place. We got VapourSynth now.
Regarding the statement of much larger memory usage: it was tested with the old x64 build vs new x86 version.
(Because the script did not run on the current x64 build)

When we make the script runnable (setting MT mode 3, as I wrote previously), x64 version has lower memory need than the x86 version.

But it is true that the x86 version is 10-20% faster in this specific situation (based on comparison with my r1850 builds for x64 and x86, but it should be the same with r1849). Reason? One or more plugin do not have optimized x64 assembly path and use C code instead? Maybe.
For QTGMC I have a 4-8% speed gain using avs+ x64 version.
pinterf is offline  
Old 11th May 2016, 11:06   #1549  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by pinterf View Post
1.) What is the resolution of your input file? In Avs+ 32bit, using 1280x720 source I have memory problems.
Seems that when other (non-avsplus-core) processes occupy significant amount of memory after the script starts, and the total nears to 3GB, could not balance and free-up enough memory.
I use 352x288 input file.

Quote:
Originally Posted by pinterf View Post
Your script still doesn't run for me even with prefetch(1) (from avsmeter. The usual "reading frames")
If I set only one of these to mode 3
Code:
SetFilterMTMode("ConvertFromShader", 2)
SetFilterMTMode("ConvertToShader", 2)
SetFilterMTMode("ExecuteShader", 2)
SetFilterMTMode("Shader", 2)
SetFilterMTMode("SVSmoothFps", 3)
then it works.
For me it works with AVS x86.

Quote:
Originally Posted by Groucho2004 View Post
Very accurate assessment.
This Shader thingy is monstrous and uses insane amounts of memory.
The SVP stuff is quite buggy and also uses the GPU and should therefore run in mode 3, I suppose. Not sure if "SetFilterMTMode("SVSmoothFps", 3)" is sufficient, though.
Yet there's nothing better to replace them, and they both produce great results SVP works good in mode 2. In fact, it's entirely designed and optimized for multi-threading as it must run in real-time... often with 16+ threads for real-time playback. SVP itself is now actually very stable with the latest build of AVS+. It's working better than ever. Not sure why we're having problems with it here. Maybe the script calling it (InterFrame) didn't get updated and that's where the difference is?

Last edited by MysteryX; 11th May 2016 at 11:10.
MysteryX is offline  
Old 11th May 2016, 12:53   #1550  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Chainik said this about SVP

Quote:
Originally Posted by Chainik
There's a little mess with SVP libs versions and MT mode 1 support.
- all pre-Vapoursynth libs are MT mode 2 only
- builds 128 - 132 are compatible with MT mode 1 BUT there's a mysterious memory corruption somewhere in svpflow2 which gives us that error
- as a temporary fix we released svpflow2.dll .135 as an old (pre-VS) version upgraded for the latest SVP Manager, so it is NOT compatible with MT mode 1

The script with SetFilterMTMode("SVSmoothFps", 1) using svpflow2.dll ver.135 will definitely freeze on start.


However I really don't see any improvement in using MT mode 1 with build 132 libs.
Here's how it works:

"old" Avisynth-only libs in MT=2: N instanes of AVS's VideoFilters running at the same time.

"new" multi-platform libs contain a platform-independent thread-safe "cores"
- Vapoursynth: 1 thread safe VS filter uses 1 "core"
- Avisynth MT=2: N single-threaded VideoFilters share 1 thread safe "core"
- Avisynth MT=1: 1 thread safe VideoFilter uses 1 "core" - just like in Vapoursynth version

The point is there's just one "core" is all cases.
MysteryX is offline  
Old 11th May 2016, 14:26   #1551  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Quote:
Originally Posted by MysteryX View Post
Adding this causes the x86 script to freeze on start.
Hmm, if I set your shader filters to mode 3 and SVP to mode 1 it doesn't freeze, tested with 32 and 64-bit. Both perform about the same. I see that pinterf has to set SVSmoothFps to mode 3 to get it to work, something is really odd here.

One thing I did notice while testing is that performance is about the same with MT and with it disabled. Just out of curiosity, what's the performance of your script when you run it with MT disabled?


Quote:
Originally Posted by asarian View Post
I myself have decided not to bother with AviSynth 64-bit any more. Last time I did (which, granted, was a while back; but your data doesn't look promising), it was crashing all over the place. We got VapourSynth now.
I don't think MysteryX's result are a proper depiction of AviSynth+ MT. With QTGMC I get better performance with AVS+ than with VS.
Reel.Deel is offline  
Old 11th May 2016, 14:33   #1552  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by Reel.Deel View Post
what's the performance of your script when you run it with MT disabled?
With MT I get 9.1fps
Without MT I get 2.2fps

That's because I have 8 cores. Without MT, only 1/8th of my computing power is used. Plus with AviSynthShader, the CPU has to wait for the GPU to finish processing so the CPU is idle a lot of the time; alternating between the CPU and the GPU working but not both simultaneously.

Quote:
Originally Posted by Reel.Deel View Post
I don't think MysteryX's result are a proper depiction of AviSynth+ MT. With QTGMC I get better performance with AVS+ than with VS.
I do get slightly better performance with AviSynth+ than AviSynth 2.6 MT, and considerably lower memory usage, but only with the x86 version.

However, my results aren't very representative of the effectiveness of CPU frame processing because the bottleneck is GPU memory transfers. It does, however, test the memory usage, effectiveness and stability of memory transfers. I agree that a script where the CPU is the bottleneck would be a better representation of the computing effectiveness of AviSynth+.
MysteryX is offline  
Old 11th May 2016, 14:42   #1553  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
I had posted benchmark results of x64 which were not good, however x86 works well. It would be more fair to show the good results with x86.

Curiously enough, after updating SVPFlow to the latest version, it started crashing in AVS+. After reverting back to the version that comes with InterFrame, it works again (??)

AND, after reverting back to that version of SVPFlow, I'm no longer getting freezes when seeking. That's odd because SVP itself is working perfectly fine with AVS+.

AviSynth 2.6
Code:
Frames processed:               304 (0 - 303)
FPS (min | max | average):      1.391 | 1000000 | 9.087
Memory usage (phys | virt):     823 | 1200 MiB
Thread count:                   139
CPU usage (average):            11%
AviSynth+ r1849
Code:
Frames processed:               303 (0 - 302)
FPS (min | max | average):      1.032 | 1000000 | 9.121
Memory usage (phys | virt):     544 | 992 MiB
Thread count:                   155
CPU usage (average):            13%
AviSynth+ is very slightly faster and the speed is more consistent. Memory usage is clearly lower.

Again, running it in x64 is a whole other story.

Last edited by MysteryX; 11th May 2016 at 14:49.
MysteryX is offline  
Old 11th May 2016, 15:42   #1554  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Ok, MysteryX, check PM.
It works in x64, with your plugins in mode 2 and spvflow in mode 1 (spv version 4.0.0.128)
(but only one test is not a test)

Last edited by pinterf; 11th May 2016 at 16:30. Reason: spv version added
pinterf is offline  
Old 13th May 2016, 09:57   #1555  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 595
When a temporal filter calls child->GetFrame() more than one times, the speed of the filter will be changed by the order of that calls on MT mode.

I noticed this when I rewrote ReduceFlicker for avs2.6/avs+.

my repository
binary(32bit)

Code:
LoadPlugin("ReduceFlicker26.dll")
SetFilterMTMode("ReduceFlicker", MT_NICE_FILTER)
ColorBars(1920, 1080, "YV12").Spline36Resize(1280, 720)
ReduceFlicker(strength=3, aggressive=true, grey=false, opt=1, raccess=true)
prefetch(4)
When I set raccees(I added this option to confirm this behavor) to false, almost all prefetch loses its power.
(490fps -> 175fps on my i7-4790 desktop)

Probably, it'll be also same as this that prefetch doesn't work for TemporalSoften().
__________________
my repositories

Last edited by Chikuzen; 14th May 2016 at 04:21.
Chikuzen is offline  
Old 13th May 2016, 20:39   #1556  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
First I wanted to say that hmmm, interesting phenomenon. If someone would develop a filter chain visualizer I would be grateful.

Then a memory came to mind, from times when I was bothering with the cache problem thingy.

I've seen an adaptive pattern matching in the prefetch code. As I recall that the algorithm tries to guess the request pattern, e.g. frames are requested by one (1-2-3-4...), two (2-4-6-8... what is it used for, I don't know, SelectEven?), or backward(100-99-98-97...), etc.
If there is enough data for a rule, it "locks" on this pattern and continues the prefetch using this prediction.

(And I noticed that Frame 0 is not prefetched, prefetching starts with frame 1 .... Tried to fix it for the sake of perfectness but did not have the knowledge and patience for this)
pinterf is offline  
Old 13th May 2016, 20:43   #1557  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Long, long time ago, I remember a project named "AVE AviSynth Visual Editor". I doubt it ever reached practical relevance...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline  
Old 16th May 2016, 08:19   #1558  |  Link
kypec
User of free A/V tools
 
kypec's Avatar
 
Join Date: Jul 2006
Location: SK
Posts: 826
Could any moderator be so kind and include link to pinterf's Git repo with latest Avisynth+ builds on the first page of this thread, please? It took me quite some time to dig those information from his posts scattered across dozens of the last pages...
kypec is offline  
Old 16th May 2016, 08:52   #1559  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Maybe I could use a signature line
pinterf is offline  
Old 16th May 2016, 08:58   #1560  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by pinterf View Post
Maybe I could use a signature line
I'd see something more simple for you, such as

> Me
MysteryX is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.