x265 HEVC Encoder

[LunaRabbit]

Originally Posted by Boulder

If you are on Windows, jpsdr does offer pre-compiled binaries in GitHub.

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs.

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.

Originally Posted by Z2697

Here're 2 test results of "cutree-strength" I just ran.
I leave them as links because they are long images.
Please ignore the speed, I ran them in VM.

https://files.catbox.moe/4qugbj.png
https://files.catbox.moe/1tqlou.png

The curves from "corresponding" qcomp and "cutree-strength" values are almost completely aligned with each other, maybe not easy to see in CRF results since the final bitrate differs quite a lot, so I did a 2-pass test.

(qcomp 0.65 corresponds to cutree-strength 1.75 and qocmp 0.7 corresponds to cutree-strength 1.5)

Also keep in mind this is not a valid quality comparison across different "qcomp and cutree-strength pairs", the matrics usually "performs" poorly when it comes to "bits re-distribution". (my layman's understanding is: aq is bits re-distribution within frame, cutree/mbtree is bits re-distribution across frames)

Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

[Z2697]

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

[Z2697]

Originally Posted by LunaRabbit

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs.

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.



Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As for the metric I think it works OK to evaluate the similarity of the "corresponding qcomp and cutree-strength pairs", it's just not valid when comparing different pairs.
I used my eyes as well, of course. Just the summary report is easier to post.

[LunaRabbit]

Originally Posted by Z2697

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.

Originally Posted by Z2697

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise.

[Z2697]

Originally Posted by LunaRabbit

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.



As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise.

I think it would be easier to cherry-pick the related commits of cutree-strength mod into jpsdr's x265_mod branch, the code behind it is way less.

[jpsdr]

About mbtree and x264.
Disabling mbtree is recommended when you target Blu-Ray, according mp3dom results and tests. But i think it's the only case (or of course, cases using very very similar encode parameters). The issue seems to be the small (1s) keyint value, not working well with mbtree. With standard [250] keyint value, mbtree is doing a proper job (still according mp3dom).

[GeoffreyA]

Originally Posted by jpsdr

About mbtree and x264.
Disabling mbtree is recommended when you targer Blu-Ray, according mp3dom results and tests. But i think it's the only case (or of course, cases using very very similar encode parameters). The issue seems to be the small (1s) keyint value, not working well with mbtree. With standard [250] keyint value, mbtree is doing a proper job (still according mp3dom).

And DS's paper: https://archive.org/download/x264_mb...264_mbtree.pdf

[Z2697]

Hmm, something doesn't feel right.

source/encoder/slicetype.cpp

void Lookahead::cuTree(Lowres **frames, int numframes, bool bIntra)
{
...

    double totalDuration = 0.0;
    for (int j = 0; j <= numframes; j++)
        totalDuration += (double)m_param->fpsDenom / m_param->fpsNum;

    double averageDuration = totalDuration / (numframes + 1);

I'm "investigating" (f* around and find out) the "inconsistency in bitstreams encoded by different compiler target flags produced binaries" which reportedly is associated with cutree.
Me, being a incompetent wannabe programmer, also kind of narrowed things down (or, rather, "confirmed") to cutree after, what, like 4 days? And is now moderately sure the quoted codes (in combination with some compiler / ISA optimization) caused the inconsistency.
(only averageDuration is used in the following codes)
As x265 itself does not have variable framerate awareness, this block code is unnecessary, I think, all these code will just result us back to "(double)m_param->fpsDenom / m_param->fpsNum", theoretically, if not the weirdness of FP math is happening, you know, the .1+.2 != .3 and non-associative stuff.

double averageDuration = (double) m_param->fpsDenom / m_param->fpsNum;

After replacing the loop with just this line, things seemed to be consistent. (But not consistent with "pre-this-modification" version)
Alternatively you can put "#pragma GCC novec" or other compiler's equivalent right before the for loop. (This retains the consistency with "pre-this-modification" "nocona" (default -march of GCC) / "SSE3" version) But this is compiler specific.

I presonally think the former "solution" is more elegant.

The "inconsistency between bitstreams encoded by different compilers produced binaries" still exists after this modification. (e.g. GCC vs Clang)

Some suspicious executables (10bit only)
https://files.catbox.moe/nhla9t.7z

Am I missing something? Please help me. plzzzzzzzz

[rwill]

One way to proceed would be to make a small repro case test application and then objdump -S that, checking what the differences in the generated code are.

[GeoffreyA]

Originally Posted by Z2697

Hmm, something doesn't feel right.

source/encoder/slicetype.cpp

void Lookahead::cuTree(Lowres **frames, int numframes, bool bIntra)
{
...

    double totalDuration = 0.0;
    for (int j = 0; j <= numframes; j++)
        totalDuration += (double)m_param->fpsDenom / m_param->fpsNum;

    double averageDuration = totalDuration / (numframes + 1);

I'm "investigating" (f* around and find out) the "inconsistency in bitstreams encoded by different compiler target flags produced binaries" which reportedly is associated with cutree.
Me, being a incompetent wannabe programmer, also kind of narrowed things down (or, rather, "confirmed") to cutree after, what, like 4 days? And is now moderately sure the quoted codes (in combination with some compiler / ISA optimization) caused the inconsistency.
(only averageDuration is used in the following codes)
As x265 itself does not have variable framerate awareness, this block code is unnecessary, I think, all these code will just result us back to "(double)m_param->fpsDenom / m_param->fpsNum", theoretically, if not the weirdness of FP math is happening, you know, the .1+.2 != .3 and non-associative stuff.

double averageDuration = (double) m_param->fpsDenom / m_param->fpsNum;

After replacing the loop with just this line, things seemed to be consistent. (But not consistent with "pre-this-modification" version)
The "inconsistency between bitstreams encoded by different compilers produced binaries" still exists after this modification. (e.g. GCC vs Clang)

Am I missing something? Please help me. plzzzzzzzz

You're right because all that loop does is multiply (Denom / Numer) by (numframes + 1), using repeated addition, and those values do not change within the loop. I wonder if there is some reason for such a superfluous piece of code. Perhaps some compiler issue back in the day, or the person was half asleep?

[Z2697]

Originally Posted by GeoffreyA

You're right because all that loop does is multiply (Denom / Numer) by (numframes + 1), using repeated addition, and those values do not change within the loop. I wonder if there is some reason for such a superfluous piece of code. Perhaps some compiler issue back in the day, or the person was half asleep?

It looks like "they were planning on VFR support" to me, but eventually that didn't happen.

[GeoffreyA]

Originally Posted by Z2697

It looks like "they were planning on VFR support" to me, but eventually that didn't happen.

Looks like it.

[higher]

Originally Posted by benwaggoner

Yeah, past a certain resolution and frame threads, more cores will start winning over better cores. M4 Max likely wins for 1080p or lower, and possibly 4K if using only 1 frame thread.

Unfortunately the M4 Max is only in MacBook Pro, which is a whole lot more expensive and bigger for headless work. Mac Mini tops out with the M4 Pro currently.

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

[Sagittaire]

Originally Posted by higher

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

Well not really simply because 9950X is more 2X (at least) powerfull than 5900X. 9950X at stock will certainely produce something like ~2 m 30 s to encode this source with x265.

[Ritsuka]

Of course, but the 9950X has a 170W TDP, and the M4 Pro is what, 40 W at max, with 6 less performance cores than the 9950X.

And it seems all the latest arm64 optimizations are again stuck on the x265-devel mailing list.

[higher]

Originally Posted by Sagittaire

Well not really simply because 9950X is more 2X (at least) powerfull than 5900X. 9950X at stock will certainely produce something like ~2 m 30 s to encode this source with x265.

The 9950X is only 50% faster than 5900X at 4K resolution. I guess an M4 Max could almost match an 9950X while consuming a lot less power.

[Sagittaire]

Originally Posted by higher

The 9950X is only 50% faster than 5900X at 4K resolution. I guess an M4 Max could almost match an 9950X while consuming a lot less power.

Attachment 18790

Techpowerup benchmark like many other no codec specialist are not able to test correctly codec: If you want seriousely make codec benchmark, you don't use gui like handbrake and you use codec profil able to saturate correctly 16C/32T CPU.

I create codec benchmark for make that and 9950X at stock has 74% more performance than 5950X for x265. I don't test 5900X but 5950X have theoricaly 20% more perfomance than 5900X. In correct H265 benchmark (all CPU thread saturation) the 9950X will produce 110% more performance than 5900X.

When Techpowerup use correct CPU saturation benchmark like cinebench, blender or stockfish, you evaluate the correct CPU power

Stockfisk:
5900X: 14.52 Mips
9950X: 30.78 Mips (+111%)

Blender:
5900X: 114.9 s
9950X: 56 s (+105%)

V-Ray:
5900X: 21538
9950X: 48899 (+127%)

[ShortKatz]

Originally Posted by higher

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

I would be quite interested how much faster my M4 Max would be compared to the M4 Pro.

[Barough]

x265 v4.1+62-441e1e4
Built on December 31 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://bitbucket.org/multicoreware/.../branch/master

DL :
https://www.mediafire.com/file/6nh9e7dfb72b3pi

[excellentswordfight]

Originally Posted by Sagittaire

Techpowerup benchmark like many other no codec specialist are not able to test correctly codec: If you want seriousely make codec benchmark, you don't use gui like handbrake and you use codec profil able to saturate correctly 16C/32T CPU.

I create codec benchmark for make that and 9950X at stock has 74% more performance than 5950X for x265. I don't test 5900X but 5950X have theoricaly 20% more perfomance than 5900X. In correct H265 benchmark (all CPU thread saturation) the 9950X will produce 110% more performance than 5900X.

When Techpowerup use correct CPU saturation benchmark like cinebench, blender or stockfish, you evaluate the correct CPU power

Stockfisk:
5900X: 14.52 Mips
9950X: 30.78 Mips (+111%)

Blender:
5900X: 114.9 s
9950X: 56 s (+105%)

V-Ray:
5900X: 21538
9950X: 48899 (+127%)

These are two different methodologies, this is not a case of "correct" and "not correct" way of doing it. Single instance encoding is still a thing, and actually the most common case for most users, so benchmarking single instance is still very much relevant.

Most software does not have perfect parallelization scaling, and in most cases were that is the case, i.e. 3d-rendering and simulations etc, those loads usually gain more to be calculated on GPUs anyway. And although I think it makes perfect sense to test both cases here, cause you can just run two encodes at the same time even though you dont wanna start doing chunk-encoding to get "more" out of your CPU. Its not like we have a history of starting to run multiple parallel benchmark of a software cause we dont find the thread-scaling good enough when that the results dont see the full "potential" of the CPU, cause this argument can be made for most of them (audio encoding, compression, compiling etc).