Internaly multi-threaded resampling functions - Page 2

Groucho2004 · 13th August 2016, 23:55

Another one (i5 2500K @ 4GHz, XP32, AVS+ r2085):

Internal resizers:

Code:

[Runtime info]
Frames processed:               1000 (0 - 999)
FPS (min | max | average):      1.743 | 255682 | 17.60
Memory usage (phys | virt):     293 | 295 MiB
Thread count:                   13
CPU usage (average):            99%

Time (elapsed):                 00:00:56.812


[Script]
colorbars(width = 1920 * 2, height = 1080 * 2, pixel_type = "yv12").killaudio().assumefps(25, 1).trim(0, 999)
BilinearResize(width() - 64, height() - 64)
BicubicResize(width() - 64, height() - 64)
LanczosResize(width() - 64, height() - 64)
Lanczos4Resize(width() - 64, height() - 64)
BlackmanResize(width() - 64, height() - 64)
Spline16Resize(width() - 64, height() - 64)
Spline36Resize(width() - 64, height() - 64)
Spline64Resize(width() - 64, height() - 64)
GaussResize(width() - 64, height() - 64)
SincResize(width() - 64, height() - 64)
Prefetch(4)

Plugin resizers:

Code:

[Runtime info]
Frames processed:               1000 (0 - 999)
FPS (min | max | average):      6.480 | 16.84 | 16.66
Memory usage (phys | virt):     76 | 79 MiB
Thread count:                   89
CPU usage (average):            99%

Time (elapsed):                 00:01:00.031


[Script]
colorbars(width = 1920 * 2, height = 1080 * 2, pixel_type = "yv12").killaudio().assumefps(25, 1).trim(0, 999)
BilinearResizeMT(width() - 64, height() - 64, threads = 4)
BicubicResizeMT(width() - 64, height() - 64, threads = 4)
LanczosResizeMT(width() - 64, height() - 64, threads = 4)
Lanczos4ResizeMT(width() - 64, height() - 64, threads = 4)
BlackmanResizeMT(width() - 64, height() - 64, threads = 4)
Spline16ResizeMT(width() - 64, height() - 64, threads = 4)
Spline36ResizeMT(width() - 64, height() - 64, threads = 4)
Spline64ResizeMT(width() - 64, height() - 64, threads = 4)
GaussResizeMT(width() - 64, height() - 64, threads = 4)
SincResizeMT(width() - 64, height() - 64, threads = 4)

TheFluff · 14th August 2016, 01:13

Quote:

Originally Posted by Chikuzen

benchmark?
I did. http://pastebin.com/ZCNnN5RW.

Code:

38.607 sec [77.706fps] #threads=1, prefetch=1
36.943 sec [81.206fps] #threads=2, prefetch=1

Is this an error in the benchmark or does it basically not scale at all with two threads?

jpsdr · 14th August 2016, 10:01

As i'm often doing this at work during lunch break, and i have only at it a 2 cores CPU without HT, when i've checked on it between 1 or 2 threads, there was a significant difference, and yes, looking also at the result, it scales with 2 threads.

I'm very busy today, but tomorrow i'll try on my PC some benchmark also...

jpsdr · 15th August 2016, 10:21

Some tests on my PC :

Code:

[OS/Hardware info]
OS version:                   Windows 7 (x64) Service Pack 1 (Build 7601)
CPU (brand string):           Intel(R) Core(TM) i7-6950X CPU @ 3.00GHz
CPU (code name):              Unknown Core 2
CPU clock (measured):         3624 MHz
CPU cores / Logical cores:    10 / 20


[Avisynth info]
Avisynth VersionString:       AviSynth+ 0.1 (r2151, MT, x86_64)
Avisynth VersionNumber:       2.60
File version:                 0.1.0.0
Avisynth Interface Version:   6
Multi-threading support:      Yes
Linker/compiler version:      14.0

[Clip info]
Number of frames:                10000
Length (hh:mm:ss.ms):     00:06:40.000
Frame width:                      3776
Frame height:                     2096
Framerate:                      25.000 (25/1)
Colorspace:                       YV12

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      118.3 | 149685 | 492.7
Memory usage (phys | virt):     1196 | 1191 MiB
Thread count:                   41
CPU usage (average):            99%

Time (elapsed):                 00:00:20.295


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36Resize(width()-64,height()-64)
Prefetch(20)

Code:

[OS/Hardware info]
OS version:                   Windows 7 (x64) Service Pack 1 (Build 7601)
CPU (brand string):           Intel(R) Core(TM) i7-6950X CPU @ 3.00GHz
CPU (code name):              Unknown Core 2
CPU clock (measured):         3624 MHz
CPU cores / Logical cores:    10 / 20


[Avisynth info]
Avisynth VersionString:       AviSynth+ 0.1 (r2151, MT, x86_64)
Avisynth VersionNumber:       2.60
File version:                 0.1.0.0
Avisynth Interface Version:   6
Multi-threading support:      Yes
Linker/compiler version:      14.0

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      459.2 | 505.2 | 502.5
Memory usage (phys | virt):     63 | 58 MiB
Thread count:                   61
CPU usage (average):            92%

Time (elapsed):                 00:00:19.900


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36ResizeMT(width()-64,height()-64)
#Prefetch(20)

Code:

[OS/Hardware info]
OS version:                   Windows 7 (x86) Service Pack 1 (Build 7601)
CPU (brand string):           Intel(R) Core(TM) i7-6950X CPU @ 3.00GHz
CPU (code name):              Unknown Core 2
CPU clock (measured):         3623 MHz
CPU cores / Logical cores:    10 / 20


[Avisynth info]
Avisynth VersionString:       AviSynth 2.60, build:Mar 31 2015 [16:38:54]
Avisynth VersionNumber:       2.60
File version:                 2.6.0.6
Avisynth Interface Version:   6
Multi-threading support:      No
Linker/compiler version:      6.0


[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      21.46 | 43.44 | 43.26
Memory usage (phys | virt):     46 | 44 MiB
Thread count:                   1
CPU usage (average):            5%

Time (elapsed):                 00:03:51.170


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36Resize(width()-64,height()-64)

Code:

[OS/Hardware info]
OS version:                   Windows 7 (x86) Service Pack 1 (Build 7601)
CPU (brand string):           Intel(R) Core(TM) i7-6950X CPU @ 3.00GHz
CPU (code name):              Unknown Core 2
CPU clock (measured):         3623 MHz
CPU cores / Logical cores:    10 / 20


[Avisynth info]
Avisynth VersionString:       AviSynth 2.60, build:Mar 31 2015 [16:38:54]
Avisynth VersionNumber:       2.60
File version:                 2.6.0.6
Avisynth Interface Version:   6
Multi-threading support:      No
Linker/compiler version:      6.0

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      404.0 | 484.2 | 481.5
Memory usage (phys | virt):     47 | 45 MiB
Thread count:                   41
CPU usage (average):            90%

Time (elapsed):                 00:00:20.769


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36ResizeMT(width()-64,height()-64)

Groucho2004 · 15th August 2016, 10:59

Quote:

Originally Posted by jpsdr

Intel(R) Core(TM) i7-6950X

Nice CPU, bloody expensive.

jpsdr · 15th August 2016, 11:04

Quote:

Originally Posted by Groucho2004

Nice CPU, bloody expensive.

It helps when brother works on Intel...

real.finder · 18th August 2016, 02:38

intel SSE4.2 Release gave me 0x7e error in load plugin for both x64 and x86

the cpu is core i7 X 980 in os windows server 2008 R2, same intel SSE4.2 Release for nnedi3 always work

jpsdr · 18th August 2016, 08:45

I'll check when back at home if i see anything strange in the build profile. My standard PC is no more than SSE4.2 (i7@850), and didn't have issue with it. Do you have the last Intel Redistribuable version ? (Don't know if it can be related...).

EDIT : Can you check if you have the same issue with either the AutoYUY2 filter and/or my VDub filters ?

real.finder · 18th August 2016, 15:29

Quote:

Originally Posted by jpsdr

I'll check when back at home if i see anything strange in the build profile. My standard PC is no more than SSE4.2 (i7@850), and didn't have issue with it. Do you have the last Intel Redistribuable version ? (Don't know if it can be related...).

EDIT : Can you check if you have the same issue with either the AutoYUY2 filter and/or my VDub filters ?

AutoYUY2 SSE4.2 gave same error

Quote:

Intel Redistribuable

I don't see it before, don't you build it as static? maybe you build nnedi3 as static but the others not?

jpsdr · 18th August 2016, 16:13

I think all are build as static (i'll chek later). Get and install the Intel Redistribuable, and check if it's this.

real.finder · 18th August 2016, 16:33

Quote:

Originally Posted by jpsdr

I think all are build as static (i'll chek later). Get and install the Intel Redistribuable, and check if it's this.

same thing

Groucho2004 · 18th August 2016, 17:00

Quote:

Originally Posted by real.finder

same thing

You could try troubleshooting with Dependency Walker and AVSMeter (with the "-avsinfo" switch).

jpsdr · 18th August 2016, 18:25

I've checked the build profile, and found for now nothing unusual, and i must confess that i have no idea what error 0x7e is...

After, you still have the standard no Intel releases, they are here as safeguard for such kind of cases. And in the best case, speed difference would probably be only a few percent.
Do you have the same issue with my VDub filters ?

burfadel · 19th August 2016, 07:28

Quote:

Originally Posted by Chikuzen

benchmark?
I did. http://pastebin.com/ZCNnN5RW.

What if you did thread 4, prefetch 4 (for example) instead of threads/prefetch as 8/1 or 1/8?

MysteryX · 19th August 2016, 08:14

Why are you guys spending so much time on this if it gives lower performance than the regular filters with Prefetch?

jpsdr · 19th August 2016, 09:12

Quote:

Originally Posted by MysteryX

Why are you guys spending so much time on this if it gives lower performance than the regular filters with Prefetch?

Mine #24 is a little higher on avs+, and a looot on standard avisynth. So basicaly, it depends on people. I never said this was for avs+ only, and that was also the purpose of it. After, everyone is free to use what they want. Personnaly, i rather keep the memory usage low, even at the cost of more threads created, but others are totaly free to have a different point of view, of course.

I've take a look at fk3db to see if i can do the same thing, but it's a lot more complex to follow, and the C++ is a little to much ++ for me...

Groucho2004 · 19th August 2016, 10:00

Quote:

Originally Posted by jpsdr

Mine #24 is a little higher on avs+, and a looot on standard avisynth. So basicaly, it depends on people. I never said this was for avs+ only, and that was also the purpose of it. After, everyone is free to use what they want. Personnaly, i rather keep the memory usage low, even at the cost of more threads created, but others are totaly free to have a different point of view, of course.

I too think it's a very good alternative for "classic" Avisynth users. It scales well, uses very little memory and it's easy to substitute the standard resizers. Nicely done.

FranceBB · 19th August 2016, 11:54

Tested using an i7 4 core, 8 threads, 3.60 GHz, 8 MB cache l3, AVX 2, using avisynth trying to upscale a progressive material via Spline64Resize and Spline64ResizeMT to 4K and encode it with x265.
It indeed increases the speed as the "pitch" reached by the standard Spline64 is 3.05, and is stable at 2.88 fps, while Spline64MT reaches a "pitch" of 3.84 and is stable at 3.24 fps, which is a very good improvement for me!
I tested it using an old processor as well, which is a monocore dual thread, which supports up to SSE3 trying to downscale from 1080p to 720p via Spline64 and I can tell ya that the Mt Version takes longer to start and it doesn't have any noticeable benefits. So... it really depends on your CPU. I'm gonna try it on an AMD 6 core as soon as I can, as AMD handles multithreading differently compared to Intel (in a worse way, most of the time) and maybe AMD CPUs will be the ones who will benefit most from it.

Oh, by the way, you said that you tried to do it with f3kdb, right? If you have time, how about doing it with the old good LSFmod as well? *_*

Groucho2004 · 19th August 2016, 13:09

Quote:

Originally Posted by FranceBB

Tested using an i7 4 core, 8 threads, 3.60 GHz, 8 MB cache l3, AVX 2, using avisynth trying to upscale a progressive material via Spline64Resize and Spline64ResizeMT to 4K and encode it with x265.

It seems to me that you mainly measured the encoding speed.

jpsdr · 19th August 2016, 20:29

I've made a new version with optimized threads repartition according physical cores.

Some tests with it :

Code:

[OS/Hardware info]
OS version:                   Windows 7 (x64) Service Pack 1 (Build 7601)
CPU (brand string):           Intel(R) Core(TM) i7-6950X CPU @ 3.00GHz
CPU (code name):              Unknown Core 2
CPU clock (measured):         3624 MHz
CPU cores / Logical cores:    10 / 20


[Avisynth info]
Avisynth VersionString:       AviSynth+ 0.1 (r2172, MT, x86_64)
Avisynth VersionNumber:       2.60
File version:                 0.1.0.0
Avisynth Interface Version:   6
Multi-threading support:      Yes
Linker/compiler version:      14.0

[Clip info]
Number of frames:                10000
Length (hh:mm:ss.ms):     00:06:40.000
Frame width:                      3776
Frame height:                     2096
Framerate:                      25.000 (25/1)
Colorspace:                       YV12

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      471.4 | 509.3 | 506.4
Memory usage (phys | virt):     63 | 58 MiB
Thread count:                   41
CPU usage (average):            60%

Time (elapsed):                 00:00:19.749


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36ResizeMT(width()-64,height()-64,threads=10)
#Spline36Resize(width()-64,height()-64)
#Prefetch(20)

Code:

[Clip info]
Number of frames:                10000
Length (hh:mm:ss.ms):     00:06:40.000
Frame width:                      3776
Frame height:                     2096
Framerate:                      25.000 (25/1)
Colorspace:                       YV12

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      460.0 | 502.9 | 501.0
Memory usage (phys | virt):     64 | 59 MiB
Thread count:                   61
CPU usage (average):            90%

Time (elapsed):                 00:00:19.962


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
Spline36ResizeMT(width()-64,height()-64,threads=0)
#Spline36Resize(width()-64,height()-64)
#Prefetch(20)

Code:

[Clip info]
Number of frames:                10000
Length (hh:mm:ss.ms):     00:06:40.000
Frame width:                      3776
Frame height:                     2096
Framerate:                      25.000 (25/1)
Colorspace:                       YV12

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      118.2 | 217181 | 450.0
Memory usage (phys | virt):     624 | 619 MiB
Thread count:                   31
CPU usage (average):            63%

Time (elapsed):                 00:00:22.222


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
#Spline36ResizeMT(width()-64,height()-64,threads=0)
Spline36Resize(width()-64,height()-64)
Prefetch(10)

Code:

[Clip info]
Number of frames:                10000
Length (hh:mm:ss.ms):     00:06:40.000
Frame width:                      3776
Frame height:                     2096
Framerate:                      25.000 (25/1)
Colorspace:                       YV12

[Runtime info]
Frames processed:               10000 (0 - 9999)
FPS (min | max | average):      125.1 | 191873 | 493.3
Memory usage (phys | virt):     1196 | 1191 MiB
Thread count:                   41
CPU usage (average):            99%

Time (elapsed):                 00:00:20.273


[Script]
Colorbars(width=1920*2,height=1080*2,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,9999)
#Spline36ResizeMT(width()-64,height()-64,threads=0)
Spline36Resize(width()-64,height()-64)
Prefetch(20)

And winners awards are :
The fastest : MT version with 10 threads (not what i hoped...

)
The more memory used : Core version with 20 threads.
The more threads created : MT version with 20 threads.

Well, it depends of CPU and maybe a little tunning also.
Well, finaly, it's up to you to choose whatever version you want to use...

18th August 2016, 02:38	#27 \| Link
real.finder Registered User Join Date: Jan 2012 Location: Mesopotamia Posts: 2,587	intel SSE4.2 Release gave me 0x7e error in load plugin for both x64 and x86 the cpu is core i7 X 980 in os windows server 2008 R2, same intel SSE4.2 Release for nnedi3 always work __________________ See My Avisynth Stuff

18th August 2016, 08:45	#28 \| Link
jpsdr Registered User Join Date: Oct 2002 Location: France Posts: 2,316	I'll check when back at home if i see anything strange in the build profile. My standard PC is no more than SSE4.2 (i7@850), and didn't have issue with it. Do you have the last Intel Redistribuable version ? (Don't know if it can be related...). EDIT : Can you check if you have the same issue with either the AutoYUY2 filter and/or my VDub filters ? Last edited by jpsdr; 18th August 2016 at 12:08.

19th August 2016, 08:14	#35 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Why are you guys spending so much time on this if it gives lower performance than the regular filters with Prefetch? __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

14th August 2016, 10:01	#23 \| Link
jpsdr Registered User Join Date: Oct 2002 Location: France Posts: 2,316	As i'm often doing this at work during lunch break, and i have only at it a 2 cores CPU without HT, when i've checked on it between 1 or 2 threads, there was a significant difference, and yes, looking also at the result, it scales with 2 threads. I'm very busy today, but tomorrow i'll try on my PC some benchmark also...

18th August 2016, 16:13	#30 \| Link
jpsdr Registered User Join Date: Oct 2002 Location: France Posts: 2,316	I think all are build as static (i'll chek later). Get and install the Intel Redistribuable, and check if it's this.

18th August 2016, 18:25	#33 \| Link
jpsdr Registered User Join Date: Oct 2002 Location: France Posts: 2,316	I've checked the build profile, and found for now nothing unusual, and i must confess that i have no idea what error 0x7e is... After, you still have the standard no Intel releases, they are here as safeguard for such kind of cases. And in the best case, speed difference would probably be only a few percent. Do you have the same issue with my VDub filters ?

19th August 2016, 11:54	#38 \| Link
FranceBB Broadcast Encoder Join Date: Nov 2013 Location: Royal Borough of Kensington & Chelsea, UK Posts: 2,903	Tested using an i7 4 core, 8 threads, 3.60 GHz, 8 MB cache l3, AVX 2, using avisynth trying to upscale a progressive material via Spline64Resize and Spline64ResizeMT to 4K and encode it with x265. It indeed increases the speed as the "pitch" reached by the standard Spline64 is 3.05, and is stable at 2.88 fps, while Spline64MT reaches a "pitch" of 3.84 and is stable at 3.24 fps, which is a very good improvement for me! I tested it using an old processor as well, which is a monocore dual thread, which supports up to SSE3 trying to downscale from 1080p to 720p via Spline64 and I can tell ya that the Mt Version takes longer to start and it doesn't have any noticeable benefits. So... it really depends on your CPU. I'm gonna try it on an AMD 6 core as soon as I can, as AMD handles multithreading differently compared to Intel (in a worse way, most of the time) and maybe AMD CPUs will be the ones who will benefit most from it. Oh, by the way, you said that you tried to do it with f3kdb, right? If you have time, how about doing it with the old good LSFmod as well? _