Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd March 2021, 17:44   #8041  |  Link
charliebaby
Registered User
 
charliebaby's Avatar
 
Join Date: Jun 2020
Posts: 25
For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)
__________________
I love x265
charliebaby is offline   Reply With Quote
Old 4th March 2021, 02:35   #8042  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
Quote:
Originally Posted by charliebaby View Post
For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)
And if it wasn't clear, this is actually not good advice !

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 4th March 2021, 14:20   #8043  |  Link
_kermit
Registered User
 
Join Date: Apr 2017
Posts: 38
Quote:
Originally Posted by charliebaby View Post
For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)
also made no difference, around 50% util, except that the file got around 20% larger
_kermit is offline   Reply With Quote
Old 4th March 2021, 14:22   #8044  |  Link
_kermit
Registered User
 
Join Date: Apr 2017
Posts: 38
Quote:
Originally Posted by benwaggoner View Post
And if it wasn't clear, this is actually not good advice !

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!
with 50% util. it's fast enough anyway.
UHD does all, so I don't lose that much time there either.
_kermit is offline   Reply With Quote
Old 4th March 2021, 20:40   #8045  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
Quote:
Originally Posted by _kermit View Post
also made no difference, around 50% util, except that the file got around 20% larger
Those are all parameters that reduce compression efficiency to increase parallelism. And there's a reason they're not used in --preset veryfast, as lots of those tradoffs are Not Good Deals.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 5th March 2021, 08:32   #8046  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,383
OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 5th March 2021, 11:11   #8047  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,360
Quote:
Originally Posted by LigH View Post
OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title.
OT2: 2719 days passed ever since the topic was created on the 24th September 2013, with 403 pages and your average of 20 posts per page, not taking into account any outliers, it gives us a total of 8060 replies, which, divided by 2719 gives us an average of 3 posts per day about x265. That's quite alright, I guess.

Quote:
Originally Posted by benwaggoner View Post
And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!
ROTFL

Ok, back on topic:

Quote:
Originally Posted by benwaggoner View Post
Those are all parameters that reduce compression efficiency to increase parallelism.
[...] Lots of those tradoffs are Not Good Deals.
This is absolutely true, no wonder they're not used by default.

@charliebaby... x265 does a pretty good job in using up multicore/multithread CPUs and even multi-socket configurations thanks to the whole Pool/NUMA implementation, especially with UHD contents (a bit less for lower resolutions), so that's generally quite alright. Of course, there is always a trade-off between parallelism and quality and sometimes it's just not worth it, which is why, whenever I have some spare room in my 40c/80th Xeon Platinum, I run more encodes at once rather than being obsessed in getting CPU usage up in one single encode (also 'cause sometimes you see the CPU go up and up without a real improvement in speed or sometimes you do see an improvement in speed at the expense of quality or compression - i.e bigger file).

(for the records, that's one of the server I have a work, I wish I had such a monster at home hehehe... although I would be kinda happy with something like an old Z840 with a 28c/40th Xeon, to be fair, but that's not gonna happen in the near future... )
FranceBB is offline   Reply With Quote
Old 5th March 2021, 15:44   #8048  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,383
@FranceBB: I meant the association with HTTP errors 403 (denied) and 404 (not found), so I get worried about forum software errors...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 6th March 2021, 00:10   #8049  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
I generally run slower, higher quality encodes (--preset slower and up), and on my Xeon 2x18/36 workstation I rarely get much of a boost from using both sockets below 8K. I generally just use --pools "+,-" and "-,+" to run one encode per socket. That automatically reduces the available threads for calculating defaults that change based on available threads, like --frame-threads.

Latency versus throughput are very different tuning targets.

The newer --abr-ladder presumably improves throughput when encoding the same content multiple times, and thus latency when running them all in parallel.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th March 2021, 19:18   #8050  |  Link
ShortKatz
Registered User
 
Join Date: Aug 2018
Location: Germany
Posts: 52
Quote:
Originally Posted by Barough View Post
x265 v3.5_RC1+2-f3f27198 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)
Code:
https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5
Is there some need of macOS binaries? I could provide some. Or are there only Windows users here.
ShortKatz is offline   Reply With Quote
Old 8th March 2021, 20:29   #8051  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
Quote:
Originally Posted by ShortKatz View Post
Is there some need of macOS binaries? I could provide some. Or are there only Windows users here.
x265 is certainly integrated into Mac products like Handbreak, so it is of interest.

I'd be quite interested in seeing a native M1 ARM version for Mac as well. My vague understanding today is that x265 has better perf running in x64 emulation than native ARM on M1 due to how good Apple's emulator is, even with AVX2 code. But it'd be great to be able to test head-to-head, and see how perf evolves as the ARM (including M1 and AWS's Graviton) optimizations get added.

A good justification to expense a new MacBook Pro .
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th March 2021, 17:27   #8052  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 64
If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.
Ritsuka is offline   Reply With Quote
Old 9th March 2021, 20:52   #8053  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
Quote:
Originally Posted by Ritsuka View Post
If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.
Any data on the perf delta from the patch? Does it include M1-specific optimizations?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th March 2021, 10:20   #8054  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 64
~2x than no-asm. There isn't anything specific to M1.
Ritsuka is offline   Reply With Quote
Old 10th March 2021, 19:33   #8055  |  Link
ShortKatz
Registered User
 
Join Date: Aug 2018
Location: Germany
Posts: 52
x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS.zip
sha256 checksum: 60c93733f5d3703f8d7d9e1a9152fc5a84f7d35f4ccb9a1ca55c855c3b9f32a7

x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64e)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS2.zip
sha256 checksum: d46ac744a683db0453f77b856c286a789530ef47caf1f3470fa08279c4d2d956

To install, extract x265_3.5_RC2_macOS.tar.gz. With Terminal cd into the folder "x265_3.5_RC2_macOS" and drag&drop install.command into the Terminal window. Hit Enter.
To check if x265 is properly installed, open Terminal and type:
Code:
x265 --version
It should respond "x265 [info]: HEVC encoder version 3.5_RC2".

I had to include Apple's Neon patch, otherwise arm64(e) did not build properly. arm64e adds Pointer authentication, Nested virtualization and Advanced SIMD complex number support. I don't have a M1, so I can't test if the arm64e binary will work.

Last edited by ShortKatz; 10th March 2021 at 21:13.
ShortKatz is offline   Reply With Quote
Old 17th March 2021, 08:39   #8056  |  Link
ghostshadow
Registered User
 
Join Date: Jan 2020
Posts: 20
Hello, I have a problem with my settings since version 3.4 + 26 of x265 (I am currently in 3.5_RC2, windows 10 64bit, PC Ryzen 3900x).
My concern is in increasing my Avg QP, I am almost always above 24, whereas before with version 3.4 + 20 I managed to have my Avg QP below 20.
here are the parameters used for the3.5_RC2 : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 **** I also tested: --scenecut-aware-qp 1 --masking-strength 300,2,2 *** --scenecut-aware-qp 3 --masking-strength 500,5,6.5,100,-1,-1
for the rest of the parameters : --rd 6 --rskip 2 --deblock -3:-3 --cutree --ctu 64 --min-cu-size 8 --bframes 4 --b-adapt 2 --rc-lookahead 60 --ref 4 --limit-refs 2 --merange 57 --subme 7 --aq-mode 4 --aq-strength 0.70 --me 3 --psy-rd 2.10 --psy-rdoq 1.35 --gop-lookahead 54
I tested by increasing my --bframes and reducing my --gop-lookahead, it is a little better but nothing more.
In 3.4 + 20 I had this in parameters: --scenecut-aware-qp --scenecut-window 500 --qp-delta-ref 6 --qp-delta-nonref 7 and my QP was impeccable
Thank you for your help
ghostshadow is offline   Reply With Quote
Old 17th March 2021, 13:20   #8057  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 367
x265 v3.5+8-57e817329 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)
Code:
https://bitbucket.org/multicoreware/x265_git/commits/branch/master
Barough is offline   Reply With Quote
Old 17th March 2021, 18:20   #8058  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
And the 3.5 release notes are in github. With some minor formatting fixes:


Version 3.5
===========

Release date - 16th March, 2021.

New feature
-----------
1. Real-time VBV for ABR (Average BitRate) encodes in --pass 2 using option '--vbv-live-multi-pass': Improves VBV compliance with no significant impact on coding efficiency.

Enhancements to existing features
---------------------------------
1. Improved hist-based scene cut algorithm: Reduces false positives by leveraging motion and scene transition info.
2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance.
3. Bidirectional scene cut aware Frame Quantizer Selection: Saves bits than forward masking with no noticeable perceptual quality difference.

API changes
-----------
1. Additions to x265_param structure to support the newly added features and encoder enhancements.
2. New x265_param options option '--min-vbv-fullness' and option '--max-vbv-fullness' to control min and max VBV fullness.

Bug fixes
---------
1. Incorrect VBV lookahead in option '--analysis-load' option '--scale-factor'.
2. Encoder hang when VBV is used with slices.
3. QP spikes in the row-level VBV rate-control when WPP enabled.
4. Encoder crash in option '--abr-ladder'.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th March 2021, 18:28   #8059  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,596
I'm curious about "2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance."

Has anyone tested with this? I found some value in --radl with fixed-GOP CBR encoding some years back, but haven't played with it much since. The description doesn't sound any different than what it was originally released in 2.7.

Although I see in ReadtheDocs that its use in variable GOP encoding has been added.
--radl <integer>
Number of RADL pictures allowed infront of IDR. Requires closed gop interval. If enabled for fixed keyframe interval, inserts RADL at every IDR. If enabled for closed gop interval, in case of --hist-scenecut inserts RADL at every hard scenecut whereas for the --scenecut, inserts RADL at every scenecut. Recommended value is 2-3. Default 0 (disabled).

**Range of values: Between 0 and –bframes
I had not theorized that RADL would be of general benefit at hard scene cuts before. Anyone have any experience/data on this front?


P.S. MultiCoreWare, remember to update the Release Notes on ReadTheDocs! It looks like all the new parameters have already been added.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th March 2021, 18:30   #8060  |  Link
vpupkind
Registered User
 
Join Date: Jul 2007
Posts: 39
Quote:
Originally Posted by ghostshadow View Post
Hello, I have a problem with my settings since version 3.4 + 26 of x265 (I am currently in 3.5_RC2, windows 10 64bit, PC Ryzen 3900x).
My concern is in increasing my Avg QP, I am almost always above 24, whereas before with version 3.4 + 20 I managed to have my Avg QP below 20.
here are the parameters used for the3.5_RC2 : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 **** I also tested: --scenecut-aware-qp 1 --masking-strength 300,2,2 *** --scenecut-aware-qp 3 --masking-strength 500,5,6.5,100,-1,-1
for the rest of the parameters : --rd 6 --rskip 2 --deblock -3:-3 --cutree --ctu 64 --min-cu-size 8 --bframes 4 --b-adapt 2 --rc-lookahead 60 --ref 4 --limit-refs 2 --merange 57 --subme 7 --aq-mode 4 --aq-strength 0.70 --me 3 --psy-rd 2.10 --psy-rdoq 1.35 --gop-lookahead 54
I tested by increasing my --bframes and reducing my --gop-lookahead, it is a little better but nothing more.
In 3.4 + 20 I had this in parameters: --scenecut-aware-qp --scenecut-window 500 --qp-delta-ref 6 --qp-delta-nonref 7 and my QP was impeccable
Thank you for your help
Have you tried using just --scenecut-aware-qp 3 --masking-strength 500,-1,-1 ?

Also, what happens to quality?
vpupkind is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.