x265 HEVC Encoder

[Boulder]

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

[Z2697]

Originally Posted by Boulder

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.
Note that I used 2-pass bitrate control in the testing because the "cutree-strength" strongly impacted the CRF rate control (iirc).

[Boulder]

Originally Posted by Z2697

I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.

Do you mean increasing qcomp and increasing CRF as well to match bitrate?

[Z2697]

Originally Posted by Boulder

Do you mean increasing qcomp and increasing CRF as well to match bitrate?

I use 2-pass, as I edited in the previous post.
Because without mod, the "cutree-strength" is internally calculated from qcomp, changing the "cutree-strength" without touching qcomp seems to just confuses the CRF rate control so much that iirc, makes them off-chart.
Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

[LunaRabbit]

Originally Posted by Boulder

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks)

Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

What kind of differences are you seeing as opposed to just raising qcomp a bit? I mainly use CRF/1-pass mode. I haven't used 2-pass in a long time (since the xvid days) but if there is an appreciable gain in quality for the same file size I'd be willing to go back. I'll do some testing when I get some time after the holidays are over.

Thanks for the explanation everyone.

Edit: My hope was I could leave qcomp near the default of 0.6 and tweak cutree directly with the goal of maybe shaving off some file size. My CRF is already set so low that I don't want to reduce it any further. Since I try to keep half hour content under 1GB for HD content if possible (lower is obviously better and it varies from episode to episode).

[Leo 69]

Originally Posted by Z2697

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/...it/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

[Boulder]

Originally Posted by Leo 69

You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/...it/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

I think the mailing list is your best bet: x265-devel@videolan.org

It's plain silly that they didn't choose the simple grain table file method which is already used by at least aomenc and SVT-AV1.

[Boulder]

Originally Posted by LunaRabbit

Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

The parameter should still be --aq-auto x. Nothing has changed there, so the same parameters will apply (10 for SDR, 6 for HDR if you want to enable all the tweaks it has).

[Z2697]

So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

[Boulder]

Originally Posted by Z2697

So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

I think someone mentioned that they committed a wrong set of patches under SBRC at first

[LunaRabbit]

Originally Posted by Boulder

The parameter should still be --aq-auto x. Nothing has changed there, so the same parameters will apply (10 for SDR, 6 for HDR if you want to enable all the tweaks it has).

I must have cloned the wrong repo or something. As I couldn't get --aq-auto to work no matter what I did. I'll try again later today thanks.

Do you happen to know anything about the --frame-rc stuff I asked about and if it can be controlled from a qpfile? I'm having the same issue as the person above. The release notes are very sparse about how these new features work. I'll see if I can find an archive of the mailing list in the mean time.

[Boulder]

Originally Posted by LunaRabbit

I must have cloned the wrong repo or something. As I couldn't get --aq-auto to work no matter what I did. I'll try again later today thanks.

Do you happen to know anything about the --frame-rc stuff I asked about and if it can be controlled from a qpfile? I'm having the same issue as the person above. The release notes are very sparse about how these new features work. I'll see if I can find an archive of the mailing list in the mean time.

jpsdr's repo default is 'master' but the mods are in the other branch 'x265_mod'.

Unfortunately I don't know anything about the --frame-rc parameter. Nothing in the online CLI docs either, which is not unexpected as documentation seems to be irrelevant

[LunaRabbit]

Originally Posted by Boulder

jpsdr's repo default is 'master' but the mods are in the other branch 'x265_mod'.

Unfortunately I don't know anything about the --frame-rc parameter. Nothing in the online CLI docs either, which is not unexpected as documentation seems to be irrelevant

Hey thanks for your help again. I was probably just being dumb and probably built the wrong source directory on my local system. I'm sure I'll figure it out when I have more time. I checked again and my last build doesn't have --aq-auto at all. So it's either an unmodded version (I think it's patman's) or I messed up and used the official repo.

If anyone is curious I tried patching the .diff you posting a few pages back against v4.1 from master. As expected, so much changed it can't apply automatically anymore. But I plan to go through the source code later today to see if I can get it working. Here is the output from patch;

patching file README.md
patching file build/MSYS_jpsdr/Build_Win32.sh
patching file build/MSYS_jpsdr/Build_Win32_AVX.sh
patching file build/MSYS_jpsdr/Build_Win32_AVX2.sh
patching file build/MSYS_jpsdr/Build_Win32_Broadwell.sh
patching file build/MSYS_jpsdr/Build_Win32_x86.sh
patching file build/MSYS_jpsdr/Build_mcf_10_Broadwell.sh
patching file build/VS2019_LLVM/Command_Line.txt
patching file build/VS2019_LLVM/Create_Solutions_x64.bat
patching file build/VS2019_LLVM/Create_Solutions_x86.bat
patching file doc/reST/cli.rst
patching file source/CMakeLists.txt
patching file source/abrEncApp.cpp
Hunk #2 NOT MERGED at 1173-1178.
patching file source/avisynth/avisynth.h
patching file source/avisynth/avisynth_c.h
patching file source/avisynth/avs/alignment.h
patching file source/avisynth/avs/arch.h
patching file source/avisynth/avs/capi.h
patching file source/avisynth/avs/config.h
patching file source/avisynth/avs/cpuid.h
patching file source/avisynth/avs/filesystem.h
patching file source/avisynth/avs/minmax.h
patching file source/avisynth/avs/posix.h
patching file source/avisynth/avs/types.h
patching file source/avisynth/avs/version.h
patching file source/avisynth/avs/win.h
patching file source/cmake/Version.cmake
patching file source/common/CMakeLists.txt
patching file source/common/common.cpp
patching file source/common/common.h
patching file source/common/event.cpp
patching file source/common/event.h
patching file source/common/frame.cpp
patching file source/common/frame.h
patching file source/common/lowres.cpp
patching file source/common/lowres.h
patching file source/common/param.cpp
Hunk #1 merged at 154-155.
Hunk #18 NOT MERGED at 1378-1392.
Hunk #27 NOT MERGED at 2449-2455.
Hunk #28 NOT MERGED at 2478-2488.
Hunk #29 NOT MERGED at 2530-2544,2546-2553.
Hunk #30 merged at 2607.
Hunk #31 NOT MERGED at 2638-2646.
Hunk #35 merged at 2907-2908.
Hunk #37 merged at 3003-3005.
patching file source/common/param.h
patching file source/common/threading.h
patching file source/common/x86/asm-primitives.cpp
patching file source/common/x86/loopfilter.asm
Hunk #1 already applied at 3870.
patching file source/encoder/api.cpp
Hunk #1 merged at 120.
patching file source/encoder/encoder.cpp
Hunk #1 merged at 292, NOT MERGED at 298-302.
Hunk #4 NOT MERGED at 2169-2184,2186-2197, merged at 2199-2202, merged at 2205, merged at 2210.
Hunk #5 NOT MERGED at 2536-2542.
Hunk #7 merged at 3748,3750-3751.
Hunk #11 NOT MERGED at 4618-4625, merged at 4627.
patching file source/encoder/frameencoder.cpp
Hunk #1 merged at 467.
patching file source/encoder/ratecontrol.cpp
patching file source/encoder/ratecontrol.h
patching file source/encoder/rdcost.h
patching file source/encoder/search.cpp
patching file source/encoder/slicetype.cpp
patching file source/encoder/slicetype.h
patching file source/input/avs.cpp
patching file source/input/avs.h
patching file source/input/input.cpp
Hunk #2 NOT MERGED at 41-53.
patching file source/input/input.h
patching file source/input/vpy.cpp
patching file source/input/vpy.h
patching file source/input/y4m.cpp
Hunk #1 NOT MERGED at 43-48.
patching file source/input/y4m.h
Hunk #1 NOT MERGED at 58-62.
patching file source/input/yuv.h
patching file source/output/gop.h
patching file source/output/gop_engine.hpp
patching file source/output/output.cpp
patching file source/vapoursynth/VSConstants4.h
patching file source/vapoursynth/VSHelper4.h
patching file source/vapoursynth/VSScript4.h
patching file source/vapoursynth/VapourSynth4.h
patching file source/x265.h
patching file source/x265cli.cpp
Hunk #4 NOT MERGED at 106-122.
Hunk #6 merged at 163-165.
Hunk #7 NOT MERGED at 306-328, merged at 330, merged at 332-336.
Hunk #9 merged at 565-572, NOT MERGED at 574-601, NOT MERGED at 604-611.
Hunk #11 NOT MERGED at 978-989.
Hunk #12 merged at 1112.
Hunk #13 NOT MERGED at 1201-1207.
patching file source/x265cli.h
Hunk #5 merged at 450.
Hunk #6 NOT MERGED at 505-511.
patching file source/x265res.manifest.in

So there aren't many lines that changed. I'm sure I can manually patch in what patch couldn't do automatically.

[Boulder]

Originally Posted by LunaRabbit

Hey thanks for your help again. I was probably just being dumb and probably built the wrong source directory on my local system. I'm sure I'll figure it out when I have more time. I checked again and my last build doesn't have --aq-auto at all. So it's either an unmodded version (I think it's patman's) or I messed up and used the official repo.

If you are on Windows, jpsdr does offer pre-compiled binaries in GitHub.

Fixing the patch file can be a tedious job but just takes time to sit down and work through the errors one by one. The original changes are from so long time ago that the codebase has changed quite a lot since then.

[Z2697]

Here're 2 test results of "cutree-strength" I just ran.
I leave them as links because they are long images.
Please ignore the speed, I ran them in VM.

https://files.catbox.moe/4qugbj.png
https://files.catbox.moe/1tqlou.png

The curves from "corresponding" qcomp and "cutree-strength" values are almost completely aligned with each other, maybe not easy to see in CRF results since the final bitrate differs quite a lot, so I did a 2-pass test.

(qcomp 0.65 corresponds to cutree-strength 1.75 and qocmp 0.7 corresponds to cutree-strength 1.5)

Also keep in mind this is not a valid quality comparison across different "qcomp and cutree-strength pairs", the matrics usually "performs" poorly when it comes to "bits re-distribution". (my layman's understanding is: aq is bits re-distribution within frame, cutree/mbtree is bits re-distribution across frames)

Update:
I ran a CRF test with more data points but only 2 sets of data. Make the interpolated curve more accurate and the "alignment" easier to see.
And switched to a test build I did back in the day.
The points are from CRF 12 to 32.

[LunaRabbit]

Originally Posted by Boulder

If you are on Windows, jpsdr does offer pre-compiled binaries in GitHub.

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs.

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.

Originally Posted by Z2697

Here're 2 test results of "cutree-strength" I just ran.
I leave them as links because they are long images.
Please ignore the speed, I ran them in VM.

https://files.catbox.moe/4qugbj.png
https://files.catbox.moe/1tqlou.png

The curves from "corresponding" qcomp and "cutree-strength" values are almost completely aligned with each other, maybe not easy to see in CRF results since the final bitrate differs quite a lot, so I did a 2-pass test.

(qcomp 0.65 corresponds to cutree-strength 1.75 and qocmp 0.7 corresponds to cutree-strength 1.5)

Also keep in mind this is not a valid quality comparison across different "qcomp and cutree-strength pairs", the matrics usually "performs" poorly when it comes to "bits re-distribution". (my layman's understanding is: aq is bits re-distribution within frame, cutree/mbtree is bits re-distribution across frames)

Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

[Z2697]

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

[Z2697]

Originally Posted by LunaRabbit

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs.

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.



Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As for the metric I think it works OK to evaluate the similarity of the "corresponding qcomp and cutree-strength pairs", it's just not valid when comparing different pairs.
I used my eyes as well, of course. Just the summary report is easier to post.

[LunaRabbit]

Originally Posted by Z2697

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.

Originally Posted by Z2697

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise.

[Z2697]

Originally Posted by LunaRabbit

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.



As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise.

I think it would be easier to cherry-pick the related commits of cutree-strength mod into jpsdr's x265_mod branch, the code behind it is way less.