Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th September 2017, 17:22   #3581  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,172
Quote:
Originally Posted by Groucho2004 View Post
Not sure why your example has a .mp4 extension but opening a script "おどるポンポコリン.avs" works fine in mpc-hc even without switching the locale to Japanese. This is on XP.
Sorry I meant "おどるポンポコリン.avs" I'm using Windows 10 x64. Anything I could look for to debug the issue?
MysteryX is offline   Reply With Quote
Old 5th September 2017, 17:30   #3582  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
Quote:
Originally Posted by MysteryX View Post
Sorry I meant "おどるポンポコリン.avs" I'm using Windows 10 x64. Anything I could look for to debug the issue?
Maybe this could be the problem?

Edit: Alternatively, change the system locale to Japanese and check if you still can't load the file in mpc-hc.

Last edited by Groucho2004; 5th September 2017 at 18:54.
Groucho2004 is offline   Reply With Quote
Old 5th September 2017, 17:56   #3583  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
Quote:
Originally Posted by StainlessS View Post
Atak_Snajpera,

What happens if you get virtual memory use over 2048MiB ?
Speed progressively drops after 2000 MiB mark. AVS meter silently crashes or terminates around 3.5 GiB (around 32 bit memory limit) . Speed drops to 0.5 fps before silent crash.
Atak_Snajpera is offline   Reply With Quote
Old 5th September 2017, 18:19   #3584  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,196
I've always thought that there was something not quite right with either avs or windows use of pagefile (dont know which).

EDIT: What do the 'big guns' think could be wrong ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th September 2017 at 18:22.
StainlessS is offline   Reply With Quote
Old 5th September 2017, 18:37   #3585  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
Quote:
Originally Posted by Atak_Snajpera View Post
Speed progressively drops after 2000 MiB mark. AVS meter silently crashes or terminates around 3.5 GiB (around 32 bit memory limit) . Speed drops to 0.5 fps before silent crash.
There have been a lot of improvements regarding exception handling in AVSMeter since 2.2.6. This won't prevent the crash but it might provide more info as to where it occurs.
Groucho2004 is offline   Reply With Quote
Old 5th September 2017, 19:24   #3586  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
More data but this time on nice graph



Quote:
Originally Posted by Groucho2004 View Post
There have been a lot of improvements regarding exception handling in AVSMeter since 2.2.6. This won't prevent the crash but it might provide more info as to where it occurs.
Just generic Access Violation after 3.5 GiB mark
Atak_Snajpera is offline   Reply With Quote
Old 6th September 2017, 10:51   #3587  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,941
Well, you used AVSMeter 2.6.2 (x86), so, a 32-bit process.

AVSMeter64 will run a 64-bit process.

Yet ... if you have LAA binaries, utilizing above 2.0 and up to 3.5 GB RAM should not cause so much speed penalty, that's a different issue (bold guess: maybe using a plugin which is not really LAA compatible, thus system calls occur a lot to try to prevent issues?).
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 6th September 2017 at 10:55.
LigH is offline   Reply With Quote
Old 6th September 2017, 11:29   #3588  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
Quote:
Originally Posted by LigH View Post
Well, you used AVSMeter 2.6.2 (x86), so, a 32-bit process.

AVSMeter64 will run a 64-bit process.

Yet ... if you have LAA binaries, utilizing above 2.0 and up to 3.5 GB RAM should not cause so much speed penalty, that's a different issue (bold guess: maybe using a plugin which is not really LAA compatible, thus system calls occur a lot to try to prevent issues?).
I'm a bit puzzled by your post. What makes a plugin LAA compatible? LAA only applies to the calling executable (AVSMeter, x264, VDUB, ...). Setting that linker flag in Avisynth.dll or plugins is pointless.

I have no idea why Avisynth+ doesn't exit gracefully with a proper out of memory message, I guess it's a plugin misbehaving in combination with multi-threaded AVS+.

Last edited by Groucho2004; 6th September 2017 at 11:34.
Groucho2004 is offline   Reply With Quote
Old 6th September 2017, 11:44   #3589  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,941
I'm sorry about my lack of in-depth knowledge here ... long ago, I read a brief remark about compilers or linkers abusing the most significant bit of addresses (which will be unused in a 32-bit process environment) as a behavioral flag, which I did not understand in detail, but since then I am unsure if there might be code – even in a DLL – which might rely on this bit serving as a flag, rather than being ignored. Very vague, I know. But some people use strange memory addressing trickery. I would not be able to exclude this as "half-wit nightmares", unless pointed to a convincing reason why I don't need to bother.

On the other hand, I can easily imagine issues regarding threading, despite the list of default plugin functions threading modes being so much tuned already.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 6th September 2017, 14:21   #3590  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
Quote:
Originally Posted by LigH View Post
I'm sorry about my lack of in-depth knowledge here ... long ago, I read a brief remark about compilers or linkers abusing the most significant bit of addresses (which will be unused in a 32-bit process environment) as a behavioral flag, which I did not understand in detail, but since then I am unsure if there might be code even in a DLL which might rely on this bit serving as a flag, rather than being ignored. Very vague, I know. But some people use strange memory addressing trickery. I would not be able to exclude this as "half-wit nightmares", unless pointed to a convincing reason why I don't need to bother.
Don't sell yourself short, there are indeed risks involved although I think that following simple coding guidelines should avoid these.
Groucho2004 is offline   Reply With Quote
Old 6th September 2017, 16:50   #3591  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,245
@Atak_Snajpera: could you please put this line at the beginning of the script?
SetLogParams("log.txt", LOG_DEBUG)

Then look into the generated file. When there are a lot of messages for shrinking caches, it can cause heavy slow down.

And another question: what are your speed numbers when you specify e.g. SetMemoryMax(2000) or SetMemoryMax(3000)?
pinterf is offline   Reply With Quote
Old 6th September 2017, 18:17   #3592  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
Quote:
@Atak_Snajpera: could you please put this line at the beginning of the script?
SetLogParams("log.txt", LOG_DEBUG)
Default
Code:
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MSuper() by script.
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MAnalyse() by script.
---------------------------------------------------------------------
WARNING: Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax().
---------------------------------------------------------------------
WARNING: A plugin or the host application might be causing memory leaks.
2000
Code:
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MSuper() by script.
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MAnalyse() by script.
---------------------------------------------------------------------
WARNING: Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax().
---------------------------------------------------------------------
WARNING: A plugin or the host application might be causing memory leaks.
Code:
AVSMeter 2.2.6 (x86)
AviSynth+ 0.1 (r2508, MT, i386) (0.1.0.0)
Loading script...

Number of frames:                  420
Length (hh:mm:ss.ms):     00:00:14.014
Frame width:                      1920
Frame height:                     1080
Framerate:                      29.970 (5000000/166833)
Colorspace:                       YV12

Frames processed:               420 (0 - 419)
FPS (min | max | average):      0.397 | 472295 | 12.13
Memory usage (phys | virt):     2686 | 2880 MiB
Thread count:                   53
CPU usage (average):            87%

Time (elapsed):                 00:00:34.632
3000
Code:
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MSuper() by script.
---------------------------------------------------------------------
INFO: Ignoring unnecessary MT-mode specification for mvtools2_MAnalyse() by script.
---------------------------------------------------------------------
WARNING: A plugin or the host application might be causing memory leaks.
Code:
AVSMeter 2.2.6 (x86)
AviSynth+ 0.1 (r2508, MT, i386) (0.1.0.0)
Loading script...

Number of frames:                  420
Length (hh:mm:ss.ms):     00:00:14.014
Frame width:                      1920
Frame height:                     1080
Framerate:                      29.970 (5000000/166833)
Colorspace:                       YV12

Frames processed:               420 (0 - 419)
FPS (min | max | average):      0.300 | 472295 | 13.40
Memory usage (phys | virt):     2627 | 2819 MiB
Thread count:                   53
CPU usage (average):            92%

Time (elapsed):                 00:00:31.334
What is max safe value for SetMemoryMax in 32 bit? 3500 or 4000?
Atak_Snajpera is offline   Reply With Quote
Old 6th September 2017, 18:24   #3593  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
@Atak_Snajpera
Which version of mvtools2 are you using?
Groucho2004 is offline   Reply With Quote
Old 6th September 2017, 18:27   #3594  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
Atak_Snajpera is offline   Reply With Quote
Old 6th September 2017, 18:35   #3595  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
You might get some speedup by using the latest version from pinterf.
Groucho2004 is offline   Reply With Quote
Old 7th September 2017, 08:50   #3596  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,245
Quote:
Originally Posted by Groucho2004 View Post
You might get some speedup by using the latest version from pinterf.
Yes, there were many fixes and changes since mvtools2 2.7.13.22 and the latest 2.7.22 probably fixes this one as well:
"WARNING: A plugin or the host application might be causing memory leaks."

I'd say the safe limit is 3000, but you have to experiment. AviSynth+ maintains its own memory consumption (frames, caches) internally but cannot count with a situation when a plugin is allocating so much memory that the total consumption is over limit.
pinterf is offline   Reply With Quote
Old 7th September 2017, 09:20   #3597  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
Your version of mvtools2 solves performance drop after 2000 MiB mark. I don't even have to specify SetMemoryMax values at all
Atak_Snajpera is offline   Reply With Quote
Old 7th September 2017, 17:59   #3598  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,161
Simple question. Which one is better?

Code:
#Prefetch
video=Prefetch(video,16)

#Triming
video=Trim(video,0,1771)
or

Code:
#Triming
video=Trim(video,0,1771)

#Prefetch
video=Prefetch(video,16)
Atak_Snajpera is offline   Reply With Quote
Old 7th September 2017, 18:36   #3599  |  Link
Groucho2004
►◄
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,459
Quote:
Originally Posted by Atak_Snajpera View Post
Simple question. Which one is better?

Code:
#Prefetch
video=Prefetch(video,16)

#Triming
video=Trim(video,0,1771)
or

Code:
#Triming
video=Trim(video,0,1771)

#Prefetch
video=Prefetch(video,16)
Whatever gives you more performance and/or less memory usage. AVSMeter is your friend.

It may not matter at all.
Groucho2004 is offline   Reply With Quote
Old 8th September 2017, 20:16   #3600  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,172
This isn't working with Avisynth 2.6
Code:
if (!vi.IsY() && !vi.Is420() && !vi.Is422() && !vi.Is444() && !vi.IsRGB())
    env->ThrowError("ConvertToShader: Source format is not supported.");
Is420 returns false, but IsYV12 works. I made sure to update headers to the latest version.
MysteryX is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:18.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.