Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
26th January 2020, 02:57 | #1 | Link |
Registered User
Join Date: Apr 2019
Posts: 4
|
high core count cpu for x265 encoding good idea?
as i already have x570 motherboard and ram bought like 4 months ago i'm planning to get either 3900x or 3950x from AMD which both are high core count cpu's and wanted to ask how much cores does x265 or hevc encoding can handle? i know that my used program can max out 8 core cpu (tested with friends ryzen 7 3700x) which speeds up video encoding... also another question would be - why do people on other topics say ''Note that quality diminishes as the number of cores/threads used to encode a video file increases. Once that number reaches the double digits, the quality decrease starts to become noticeable. '' why quality should decrease, if encode is being done twice faster because of more cores on same preset and same settings are people suggesting that high core count cpu cannot be used with x265 to produce good encoding results? thus i'm on BIG dilema here also please simplify the answers if possible thanks
|
26th January 2020, 13:33 | #2 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
x265 using default settings won't saturate 3950x while encoding 1920x800 (2.4:1 aspect ratio) video.
However if you use Distributed Encoding in ripbot264 then you can fully utilize even dual AMD Epyc 2 (256 threads total) without any problems.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper Last edited by Atak_Snajpera; 26th January 2020 at 13:45. |
26th January 2020, 16:23 | #3 | Link |
Registered User
Join Date: Apr 2019
Posts: 4
|
But is it really as people say that if core/thread count increases - encoding quality decreases, and if it decreases does it decrease drastically? or for example like 1% from each added core so lets say for 4 core would be 5% loss for 8 would be 10% something like this, because now i was about to get atleast 3900x to make my encoding faster and to have spare resources when encoding is running, for example so i can encode and game at same time... because my 4c 8t cpu lets me encode and use the pc fine no problems (feels a bit less responsive when encoding and cpu is at 99%) and also can play games like lol with some ping or fps drops but its not good experience, AAA games are unplayable when encoding...
|
26th January 2020, 16:51 | #4 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
That issue was totally overblown. x264 has a hard thread limit (24 threads if I remember correctly) to prevent that quality degradation. In x265 default large CTU (64) also works as limiting factor for thread utilization.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper Last edited by Atak_Snajpera; 26th January 2020 at 16:58. |
26th January 2020, 19:04 | #5 | Link | |
Lost my old account :(
Join Date: Jul 2017
Posts: 325
|
Quote:
For 1080p video with preset slow you will see pretty good scaling up to 8c/16t, if you lower the CTU size to 32 or encode 2160p you will probably be good up towards 16C/32T. After that point doing chunk encoding is pretty much a must to continue thread scaling. |
|
26th January 2020, 22:38 | #6 | Link |
Registered User
Join Date: Jan 2015
Posts: 118
|
Hi
I have a TR3960x and I have learned the following: Quality decreases in x265 if you you more frame-threads - auto setting is 6 for high core count CPUs. For a UHD encode I can utilize about 80-90% of CPU usage with pool=64 - making it using 64 threads instead of auto 48. Especially for 1080p it makes sense to either run 2 encodes at the same time or distribute the encoding via RipBot to use all CPU cores. |
27th January 2020, 10:53 | #8 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
Ffmpeg does not support frame accurate seeking. Basicaly you can't tell ffmpeg to seek to specific frame. On other hand lsmash plus avisynth is much more reliable and flexible. Ffmpeg only method would be huge downgrade in every aspect.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
27th January 2020, 23:57 | #9 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
If you let it decode source then it will be frame accurate. Similar to wait for index file. For I frame based codecs -ss before -i is also frame accurate.
You can also seek this way: -ss 8 -i xxxx -ss 4.08 which will seek to 8th second as rough seeking and then 4.08 second from this place as precise seeking. This way you should be able to seek very quickly and precisely in any format if I understand it correctly. avs/vs source filters are its weakest point and not 100% reliable at all. I ha many problems and had to choose correct one base don source format (eg. problem with transport streams or MXF files). They are actually based on ffmpeg and sometimes there are problems even after indexing. Last edited by kolak; 28th January 2020 at 00:21. |
28th January 2020, 00:20 | #10 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
That's the problem. Seeking based on time is not precise enough. Trust me soon or latter you will end up with chunks starting from incorrect frame. I remember that mediacoder was trying this method and failed misserable. Check his forum and posts related with distibutive encoding.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th January 2020, 00:23 | #11 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Only problem I found is sometimes for fractional frames (I assume due to rounding-it would need figuring out how ffmpeg does it). For non-fractional frames never had an issue. Eg. 0.04 is 1 frame for 25p based source. Quite simple.
If you happy to wait for decode time then you have very precise access with select filter. Then you can use frame numbers. Besides, for distributed encoding we either pick "nice/round" numbers or scene changes. There is really no need for places like 00:10:12:11 (if it's not a scene change). You can with ffmpeg jump to rough place, then from this point find 1st clear scene change and get time for it. For 2nd part you jump to same rough place and add time for scene change+ 1 frame (this should actually "force" ffmpeg to "use same math" for both chunks time points). Actually this could work very well. Last edited by kolak; 28th January 2020 at 00:36. |
28th January 2020, 00:32 | #12 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
I'm happy with l-smash (100% frame accurate) + avisynth so going back to ffmpeg only method would be like opening pandora box. Besides. If ain't broken, don't fix IT.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th January 2020, 00:40 | #13 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
If you think about it then it's 5% code change or rather 2nd version. It only changes what you execute, but whole main engine would stay the same.
I'm not programmer, so won't do it, but I realised it can be not so difficult even just with Python (quite easy for just 1 machine). |
28th January 2020, 00:45 | #14 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
But that constant seeking without index would significantly delay encoding for later chunks.
First chunk would start immediatelly, but how about 100th? Also keep in mind that everything is going VIA network and multiple servers would do the same task simultanously.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th January 2020, 00:56 | #16 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
Ha! Try with interlaced vc-1...
Btw. Why are so anti-avisynth?
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th January 2020, 01:03 | #17 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Not anti avs at all
I need as simple and reliable solution as possible. My workflows require 99.99% reliability. I used avs for massive jobs (eg. 3000 hours of very heavy processing) and it did work, but ffmpeg is "easier" and multi platform as well. Interlaced vc-1 - not interested in it at all. Should never exists I deal with broadcast+intermediate formats, so XDCAM MXF, ProRes, DNxHD, AVC-I MXF etc. (mainly MXF and MOV). Last edited by kolak; 28th January 2020 at 01:05. |
28th January 2020, 01:10 | #18 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
But I need to cover all those formats. Another problem. Without avisynth you would not have Access to mdegrain,qtgmc,hdrtosdr tonemspping or multithreaded resizer. I guess you do not also do any filtering like denoising, deinterlacing...
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
28th January 2020, 01:53 | #19 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Deinterlacing for sure. De-noising not really.
I actually use vs for deinterlacing. It only deinterlaces interlaced frames and uses QTGMC. Same can be done with ffmpeg very easily, just less good deinterlacing. Although recent new filter (bwdif= yadif+w3fdif) is good enough for broadcast and of course so much faster than QTGMC. |
28th January 2020, 18:48 | #20 | Link |
結城有紀
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 894
|
For high core count cpu I'd reduce frame threads to 1 and pools to 12 and do chunked encoding (half-half) or parallel encoding (2 at a time).
I believe less threads = less waste on threading for x265, so I always use less threads and run more processes. Besides, I already have the software infrastructure to do chunked encoding, and mux them later. Encoded GOPs can be joined later, so chunked encoding is basically zero cost to me. |
Tags |
3900x, 3950x, hevc, ryzen, x265 |
|
|