Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
![]() |
#1 | Link |
Registered User
Join Date: Nov 2016
Posts: 151
|
faster CPU = faster conversion?
I'm using a Threadripper 1950x with 64GB RAM 3200mhz; it's quite fast, but lately I've started to make projects involving 10bit UHD HDR, and conversion time is obviously increased a lot from 8bit HD SDR.
Now I'm thinking about the new AMD 7950x (still don't know if 32 or 64GB); according to CPUbenchmark, it's more or less twice faster than mine, both on single and multi thread: https://www.cpubenchmark.net/compare...-Ryzen-9-7950X I'm curious: this will apply also to avisynth conversions? I mean, if for example with the 1950x it takes 24hrs, would it take around 12hrs using the 7950x? Or there is something else to take in account, as RAM speed and type, CPU family (Threadripper vs Ryzen) etc? Thanks! |
![]() |
![]() |
![]() |
#2 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
![]() ![]() There are so many variables to this, what app's you're using, what filters & scripts you're using, the list goes on. ATM, about the best you can do (unless you've got very deep pockets), would be the new Ryzen 7950X3D, which may or may not be faster than its "little" brother, the 7950X. As the lucky owner of both a 7950X & an i9 13900KF, the choice is up to you, as they are so close, it's pretty hard to pick. Having said all that, the Intel option CAN be a LOT cheaper, as you can go for a lesser chipset motherboard, and DDR4 RAM, I doubt that it would affect encoding performance, as overclocking does generate too much more heat, and they already run @ 100°C, so you need GOOD cooling for both the AMD & Intel's. As for RAM, I find that 32Gb is plenty, but again, if you've got deep pockets, then go for 64Gb. The one possible advantage is that the Ryzen's support AVX512. Do your research, and choose well......
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|
![]() |
![]() |
![]() |
#3 | Link |
Registered User
Join Date: Dec 2001
Posts: 1,217
|
I would look at what you are encoding and see if its utilizing multiple threads well, or is it largely using a few cores.
Avisynth is mostly CPU limited, and while it has multi-threading, it doesn't really utilize it to the fullest. There are some plugins that can make use of GPU as well. I think upgrading the CPU would definitely give you a big boost, but would it cut speeds in half? I dunno. Before upgrading the CPU, I would recommend looking at other ways to boost your processing speed if its important to you. First, have you tried vapoursynth? It might be able to make better use of multiple cores and improve the processing speed. For any plugins you are using, are there other plugins that do a similar task, and have you tried them out? Are there any GPU accelerated plugins that you can use? Is it possible to rearrange the order of the commands in your script? For example, if you are upscaling your video and then applying a heavy plugin, what if you use that plugin before upscaling? Or if you are doing conversions between colorspace or bitdepths, have you tried using those commands at different points in your script? Some plugins might be much faster when working in certain colorspaces. Also, if you see that your cpu is only using half the threads, Could you run 2 videos at the same time? |
![]() |
![]() |
![]() |
#4 | Link |
Registered User
Join Date: Nov 2016
Posts: 151
|
About deep pockets: actually they are not deep at all!
![]() Was thinking about the 7950x because it seems the CPU with the best price/speed ratio; I read a bit about the new 7950X3D and the main feature seems to be lower consumption; still, with its price being 25% or more than 7950x, would bring me to think about the latter instead; for the MB I was thinking about a cheap X670, and RAM something around 4000mhz as the ones with higher speed cost much more offering not too drastic speed increase. About my encodes: I'm talking about restoration projects - the last one I'm doing right now involves a (blown up) UHD WEB regrading, taking DVD as color reference. During the years I think (hope?) I already improved my skills to the point that I guess it's hard to "squeeze" more speed from my scripts; I started arranging the tasks in the best possible way, then I stepped to Avisynth+, then lately to Avisynth+ 64bit; the last step was the best IMHO to improve the speed and avoid RAM problems - that were not present using mere 8bit HD SDR sources. Another is to use MP_pipeline; I usually split my script into more parts (like part 1 take care of frames 00000 to 15000, part to 15001 to 30000 and so on); setting up 4GB each, I reach almost always 100% CPU and RAM is almost full (even if I don't know why each script "eats" about 6/7GB RAM); this increase speed usually to 400/500% using 8 parts. |
![]() |
![]() |
![]() |
#5 | Link | ||
Registered User
Join Date: Jul 2018
Posts: 1,000
|
Quote:
Today desktops are mostly about load windows (11) and open Word and web-browser not very slow. When someone need good performance workstation it is about HP Z8 Fury G5 that top loaded looks like 2 TB ECC RAM (still DDR not HBM), 2 kW power supply, good W-Xeon like W-3375 38 cores 76 threads and several (up to 4) full-size data compute accelerator boards of professional class. Top single chip workstation Xeon may be w9-3495X 56 cores/112 threads, 8 channels of DDR5. Though the pricing of complete small undertable box is about $XXXXX. It is still not supercomputer in a separate cooled machine room. Quote:
Test script is Code:
LoadPLugin("vsTTempSmooth.dll") LoadPlugin("AddGrainC.dll") BlankClip(15, 1920, 1080, color=color_gray) #BlankClip(15, 3840, 2160, color=color_gray) ConvertToYV12() AddGrain(10) Loop(10000000) vsTTempSmooth(maxr=2) # using auto-detected default SIMD - AVX2 for intel (?) and AVX512 for AMD (?) Prefetch(16) # equal for both AMD and intel ? The Xeon Gold 6134 (8 cores / 16 threads) with maxr=7 gives about 90fps at 2160p and 378fps at 1080p. And with maxr=2 - 300 and 1200 fps (about proportional to frame sample count). It looks 6 channels of DDR4-2666 RAM and 24 MB of L3 cache not very greatly helps. The results of much cheaper i5-11600 is even not 2 times lower. Last edited by DTL; 4th March 2023 at 23:26. |
||
![]() |
![]() |
![]() |
#6 | Link | ||||
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
Quote:
Quote:
I'm trying to be polite, I can't simply just load that, and test it for you ![]() What do you use ?? Quote:
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
||||
![]() |
![]() |
![]() |
#7 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Well, if you're happy with it, why ask...it's just that you need deep pockets these days to get something that will do the job fast (without going way overboard on house priced systems)
Quote:
I've recently started upscaling some older footage, with Topaz AI, and specific avisynth scripts, etc, etc, and getting reasonably good results.
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|
![]() |
![]() |
![]() |
#8 | Link |
Registered User
Join Date: Nov 2016
Posts: 151
|
Even if I'm pretty satisfied, IF I can sell my actual hardware (at least CPU+cooler+MB) and get a 2x faster setup shelling a relatively small amount of money, why not?
Said so, I'd never spend $$$$ for EPYC or new Threadripper CPUs... you see, the 7950x seems the best price/speed ratio around right now. Script splitting: as I wrote before. Let's say I have a 120000 frame source to process. I first write my script as I want. Then I create X scripts that are the same as main script, with only added a trim line. If with one unsplit script I have for example 0.2fps, with X=8 split scripts I can get 0.8fps (and never 0.2fps * X = 1.6fps) Got it? Topaz AI: I like it, but often I got too plastic-y result; if there are two or more sources, I much prefer to average: https://forum.fanres.com/thread-4733.html |
![]() |
![]() |
![]() |
#9 | Link | |||
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
Quote:
Quote:
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|||
![]() |
![]() |
![]() |
#10 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,726
|
On not fully parallelized filters if you trim the file in multiple parts and encode them at the same time only to append them at the end you can boost your speed by making use of the whole CPU. For instance, a 0.8fps job split in 2 would run at 0.8 and 0.8, but it would have half the frames to encode per process, so it would be as if you had just one encode running at 1.6fps.
You can do the same trick as many times as you want until you saturate the CPU, but since it's never that precise it won't be exactly twice the speed for twice the resource consumption but a bit less, so 0.8fps single would be like 1.4fps double (while mathematically it should be 1.6). I know cause I'm aware of other people who do the same (although I don't). Last edited by FranceBB; 5th March 2023 at 02:21. |
![]() |
![]() |
![]() |
#11 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
![]() So what apps would one use, to do the above procedure, that's what I'm curious about. And correct me if I'm wrong, but isn't that basically the same thing that RipBot264 does with it's Distributed Encoding, and Staxrip with its "chunks" encoding option ?? Sounds like a lot of mucking around, when there's app out there, that do it already ???
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|
![]() |
![]() |
![]() |
#13 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
![]() EDIT:- Staxrip does indeed support AV1 (AOMEnc, Rav1c & SVT-AV1) Have you heard of FastFlix ??
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 Last edited by TDS; 5th March 2023 at 04:57. |
|
![]() |
![]() |
![]() |
#14 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,000
|
"What do you mean, "ended"."
The last 'enthusiasts edition' desktops intels were about 9 or 10 series. It was sort of 'small Xeons for home' with up to 4 memory channels and with 8..10+ cores and AVX512. After these series we have only degradation - no more 4 memory channels and no more AVX512 and a few of 'performance' cores in single chip. Intel-11 already have a limited memory channels to 2 even at best desktop editions and small core numbers. In 2017 we still see SkyLake-X for HEDT with up to 18 cores and all AVX512 and 4 memory channels. Now in 2023 we have only 2 memory channels and cutoff AVX512 at low-end desktops intels even in i9-13900*. "that script would not work with my go-to encoding app," I mean run it with simple avsmeter64.exe. At least several seconds to see stable average fps. It uses only AVS and 1 more external plugin AddGrainC (I hope it is typically installed at most of users - http://avisynth.nl/index.php/AddGrainC . Last edited by DTL; 5th March 2023 at 05:49. |
![]() |
![]() |
![]() |
#15 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
OK, I just had a quick look in my "plugins" folder, and indeed I have both, in question. I've been using AMD for a LONG time, so I missed all the HEDT Intel's, it's only VERY recently I went back to Intel, mainly to compare it with the 7950X, which as least has AVX512... So this is how I'm used to scripts (for RipBot), and I generally only use AVSMeter within it the app. Code:
#Custom SetCacheMode(1) SetMemoryMax(16384) LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll") LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll") LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll") LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\MedianBlur2\MedianBlur2.dll") Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs") Import("%AVISYNTHPLUGINS%\PD_TOOLS\LSF-PLUS\LSFplus.avs") Import("%AVISYNTHPLUGINS%\PD_TOOLS\RESIZERS-PACK\Resizers Pack.avs") Import("%AVISYNTHPLUGINS%\PD_TOOLS\SHARPENERS-PACK\Sharpeners Pack.avs") Import("%AVISYNTHPLUGINS%\PD_TOOLS\SMDEGRAIN\SMDegrain cpu.avs") Import("%AVISYNTHPLUGINS%\PD_TOOLS\Zs_RF_Shared\Zs_RF_Shared.avs") pre=ex_Median(video,mode="IQMV") video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter=pre,str=1.2,refinemotion=true)
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|
![]() |
![]() |
![]() |
#16 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,000
|
" I just had a quick look in my "plugins" folder, and indeed I have both,"
For testing it is required to update vsTTempSmooth to latest release of https://github.com/Asd-g/AviSynth-vs...ases/tag/1.2.3 . The AddGrainC should be enough any old version (it used only to fill some cached frames and do nothing after Loop()). Having all plugins installed so simply save Code:
SetCacheMode(1) SetMemoryMax(16384) LoadPlugin("%AVISYNTHPLUGINS%\AddGrainC\AddGrainC.dll") LoadPlugin("%AVISYNTHPLUGINS%\vsTTempSmooth\vsTTempSmooth.dll") # I hope it is the right path where you have plugins installed ? I never use global plugins dir and always simply put all .dlls and script in same dir. BlankClip(15, 1920, 1080, color=color_gray) # for 1080p test #BlankClip(15, 3840, 2160, color=color_gray) # for 2160p test ConvertToYV12() AddGrain(10) Loop(10000000) vsTTempSmooth(maxr=2) # using auto-detected default SIMD - AVX2 for intel (?) and AVX512 for AMD (?) Prefetch(32) # equal for both AMD and intel ? "to compare it with the 7950X, which as least has AVX512..." 7950x is 2 memory channels only and sort of AVX512-ready (can execute without invalid instruction crash but performance may be close to AVX2 because of only 256bit dispatch ports ?). It is expected 'full-width AVX512' is some next generations of Zen from AMD for desktops. " there's nothing like living in the past, the "good 'ole days" are long behind us." It looks we will never more see at poor degraded desktops something like i9-7980XE from 2017 - simple i9 but 18 cores 36 threads, 2.6/4.2 GHz and 4xDDR4 memory channels with all cores AVX512 and 2xFMA512 per core. It may be a candidate to try if found cheap enough secondhand. Nowdays any serious HEDT is Workstation-class Xeon like W-series. Last edited by DTL; 5th March 2023 at 07:20. |
![]() |
![]() |
![]() |
#17 | Link | |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
Quote:
Cheers.
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 |
|
![]() |
![]() |
![]() |
#18 | Link | |
Registered User
Join Date: Jul 2018
Posts: 1,000
|
Quote:
For example I have some old workstation-class motherboard (it have PCIe x16 for external video card and not very loaded with integrated networking) with 5000-series intel chipset and 2xE5420 Xeons (SSE-only) and 16 GB ECC DDR2 RAM in 4 channels (can be cheaply upgraded to 64 GB with 8x8 GB modules). It come secondhand at about $100 or less in begining of 2022 with free power supply about 300..400W (though very noisy and require separate room to install for overnight computing). So it is 4+4 cores system and 2x12MB L2 cache of increased number of ways (poor desktop CPUs also typically have poorer cache design with lower number of ways so easier to hit cache set overload condition) and good 4-channels memory controller. If put several such systems in a rig it may be cheaper solution in compare with single chip platform of several next generations newer. Also as real SDRAM performance is about no-progress in decades from 199x (about 50..100ns latency from SDR SDRAM in 199x to DDR5 in 202x) the more real separate SDRAM channels you have the better may be real multithreading performance. Several cheap secondhand boards with 4 real SDRAM channels in each may outperform some new 2-channel only system. Last edited by DTL; 5th March 2023 at 09:20. |
|
![]() |
![]() |
![]() |
#19 | Link |
Formally known as .......
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 830
|
So after the mention of the old i9-7980XE, I had a look on eBay, and they are still readily available, and they still command "good" money, then you'd have to source a socket 2066 motherboard, which would still be fairly costly, so in the end, you'd probably spend similar or more than a 13th Gen system, and even tho they may have some good tech onboard, they are so far behind in just raw horse power (according to most benchmarks).
Sad but true.. ![]() And I too have several old Supermicro Server systems, with X5680's & 90's, with 32Gb DDR3 ECC Ram, and they used to power along nicely, but now that I have quite the collection of Ryzen's and the 13900, they're just a waste of energy ![]() ![]()
__________________
This can be Very "TeDiouS".. Long term RipBot264 user. Ryzen 9 7950X Intel i9-13900KF Ryzen 9 5950X Ryzen 9 5900X Ryzen 9 3950X Link to RB v1.27.0 Last edited by TDS; 5th March 2023 at 10:29. |
![]() |
![]() |
![]() |
#20 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,000
|
Though i9-7980XE is about the same as currently used TR 1950 - both 4channels of DDR4 and comparable number of cores. So i9-7980 may be a bit faster in some applications (where AVX512 is more or less good used) - https://static.techspot.com/articles...1HandBrake.png and somewhere slower.
Real decreasing of processing time twice is simply adding second TR 1950 system and spreading task to 2 parts. As time going on - the TR1950 will be more and more cheaper because at home systems people typically like to have one a bit more performance system and throw away for a low price second hand old system after upgrade. " if for example with the 1950x it takes 24hrs, would it take around 12hrs using the 7950x?" Very unlikely. 7950x uses 2channels of DDR5 and TR1950x 4channels of DDR4 (if TR is really 'unganged' pack of 2 separate 8cores chips on a single board it even better serve in multithreading). The effective memory bandwidth is about the same. Yes - the 2channels motherboard and CPU socket is cheaper in manufacturing so marketing can make more money from user if use DDR5 2channels instead of DDR4 4channels. The 2x more onboard L3cache is unlikely very helpful with AVS as it like to use very large caches around each filter in a chain and total working set easily run to GBs and 10ths of GBs and can not fit in any MBs cache. The AVX512 support in 7950 also only the beginning and not run at 'full' speed but at the AVX2 rate only. If even yout get 2x performance boost at some few plugins in a chain it will not convert to 2x total performance boost in processing. So to make TR1950x twice faster at real workload it is still more effective to simply add second TR1950x system in parallel and load half of frame processing to second system in a pool. It will have 8channels of memory in 'true/physical unganged' mode of 2 completely independent streams of DDR4 x4. Last edited by DTL; 5th March 2023 at 11:52. |
![]() |
![]() |
![]() |
Tags |
1950x, 7950x, amd, ryzen, threadripper |
Thread Tools | Search this Thread |
Display Modes | |
|
|