x265 HEVC Encoder

[excellentswordfight]

Originally Posted by Joey BoomBats

This looks like a very useful feature for bit hungry grainy videos.


Has anyone already tested the efficiency improvements of this feature and the visual implications?


Or does anyone know what the command line parameters are?


I see that Handbrake has now included x265 version 4.1 but unfortunately no specific changes have been made to the UI (yet).

As far as I can see, this is not a complete built-in feature. The parameter added is "--aom-film-grain", so you need to denoise and create a grain parameter file externally, you then point that parameter to that file so it can be injected in to the SEI.

[tormento]

Has anyone tried --frame-dup with Patman or jpsdr builds?

It doesn't work on my AVX only CPU.

I get:

Video encoding returned exit code: -1073741819 (0xC0000005)

It's unclear what this exit code means, in case it's a Windows system error then it possibly means:

# for decimal -1073741819 / hex 0xc0000005
STATUS_ACCESS_VIOLATION ntstatus.h
# The instruction at 0x%p referenced memory at 0x%p. The
# memory could not be %s.
# as an HRESULT: Severity: FAILURE (1), FACILITY_NULL (0x0), Code 0x5
# for decimal 5 / hex 0x5
ERROR_ACCESS_DENIED winerror.h
# Access is denied.

[jpsdr]

Does the issue also happen with a not modified build ?

[Z2697]

Who uses frame-dup anyways?

It's better to give more details such as the whole command line and the "variant" of the build used to produce that error, and the input spec, etc.

[rwill]

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

[Z2697]

Originally Posted by rwill

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

Using container output (ffmpeg libx265, or supported in some mods) can have "slightly more usable" result, but yeah, currently the feature is working, I guess, but no software can recognize it. Even the "slightly more usable" container output is not utilizing timing SEIs.
The container output is using timestamps support of the container (VFR), while the picture timing SEIs don't contain actual timestamp, but only a flag indicates the corresponding frame should be doubled or tripled, if the decoder is able to support it the decoded video should still be CFR.

Even if it's working properly, both encode and decode side, the use case is still extremely limited.

[tormento]

Originally Posted by Z2697

Who uses frame-dup anyways?

Anime. Mostly old ones.

Originally Posted by jpsdr

Does the issue also happen with a not modified build ?

Tried this and it works but I don't know if it honors --frame-dup flag.

[Z2697]

Originally Posted by tormento

Anime. Mostly old ones.

You shouldn't.

To have a better understanding of what this feature means: this feature removes frames based on PSNR thresholding, and signal picture timing SEIs to keep the correct... picture timing... yeah... which no commonly available decoder can recognize.

It also requires VBV and HRD. Since no everyday decoder can recognize the timing SEIs the picture timing will be incorrect if any frame(s) are removed.
If you don't see any incorrect timing,
1) You didn't notice it.
2) You didn't enable the required VBV and HRD.
3) Your video didn't have any frames above the threshold.

Even if it works as it should, the PSNR thresholding is not ideal to begin with, and the bits saved with removing near identical frames are, well, did you know picture timing SEIs cost bits?

[jpsdr]

Does it work on CPU with more than AVX only ?

[Patman]

The reason for the error is the direct support of AVS and VPY scripts (MOD). However, if you use the pipe methods, the encode works. It seems that x265 requires the Y4M or YUV formatfor this case.

[jpsdr]

That's odd, i'm almost sure i used frame-dup on avs scripts with a moded version, during my first attempts of trying to do x265 encodes, before giving up using it because of all the issues i encountered, reasons Z2697 described.
My guess was it could be because of the multiview add, but it was wrong it seems.

[Leo 69]

Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl

For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

[Z2697]

Originally Posted by Leo 69

Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl

For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

[benwaggoner]

Originally Posted by rwill

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

[benwaggoner]

Originally Posted by Z2697

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

[Z2697]

Originally Posted by benwaggoner

Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

I'm not sure "which" duplication you are referring to, the first case is the encoding side:

What it does is actually removing frames and using SEI to tell decoder to duplicate existing frames, which actually should be called de-duplication I think.

Maybe you are not looking at the right place (i.e. you think it's duplicating frames while it actually deletes frames), but assuming you are not able to get frames to trigger the de-duplication during encoding:

There's PSNR thresholding, can be configured via --dup-threshold parameter.
If you use low enough threshold, eventually some frames will be de-duped, but of course this should only be used in experiments, low threshold will just destroy the video.

The second case: the decoding side...

You need a decoder that's able to recognize and utilize the picture timing SEI (which only tells decoder to double or triple the frame, no actual timestamp is stored) and I guess things will... "just work"... yeah, who knows, the most common decoder (avcodec) doesn't support it so I can't test.

Originally Posted by benwaggoner

The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

Maybe I just hate noises in general
But I agree, it (FGS) has potential... just it still has a long way to go.

[LunaRabbit]

Hello Doom9! It's been nearly 20 years since I last posted here although I've been lurking on and off all these years. I recently got back into this old hobby and have greatly appreciated the information ITT. Since I'd been away since the early x264 days and it took several weeks to get up to speed with x265. If it wasn't for the information ITT I would have spent many more weeks aimlessly testing stuff I'm sure. I have a bunch of questions. I hope you don't mind if I ask them.

1) --frame-rc / https://bitbucket.org/multicoreware/...396de62dc618c1

Can someone please explain how to work with this recent commit? Can I just define this stuff in my qpfile? If so, what syntax should I use? There is no documentation about this that I can find and it isn't really explained on the git repo from what I've seen. I've been slowly trying to work my way through ratecontrol.cpp to understand it but I haven't had a chance to throw random stuff in a qpfile and check yet (I will soon when I'm back at home). Has anyone used this feature yet?

I am very interested in this since I already manually create a qpfile anyway to set I-frames at particular frames for various reasons (seeking speed and ensuring frames with overlays get an I-frame when needed). I'm already creating qpfiles by hand for each source I work with. So the ability to manually set CRF/QP/Bitrate would be a much welcomed addition. Up until now I've been having to manually make zones and multiply the bitrate up/down which is less than ideal.

2) Misc. cutree stuff and mods

I did a lot of testing with cutree on/off last year and visually I frankly couldn't see much difference but turning off cutree would sometimes greatly increase bitrates for little gain. The extra bits didn't seem worth what I was getting. I currently "reign in" cutree by tweaking --qcomp typically at something like 0.7 or 0.8 at most. Boulder posted a diff file ITT several pages ago and the repo linked in this post is supposed to allow tweaking cutree directly instead of in the round about way with qcomp. But I've also heard from many people over the last few years that I shouldn't use cutree at all because it's a

Bad implementation of mbtree from x264

I do not believe this because in my experience cutree hasn't degraded visual quality at the CRFs I typically use (15-18 or so). I suspect this is one of those hold over old wives tales from the early days of mbtree in x264 when it wasn't fleshed out yet. I'll be the first to admit I don't fully understand how cutree works in x265. I've been reading the source code in an attempt to better understand it. I've also spent many hours reading various posts here. But there really isn't a lot of good technical discussion about it that I can find. Perhaps it's happening on the mailing list somewhere that I haven't seen yet?

I would greatly appreciate hearing your opinions on cutree and if using the above modifications (along with the hours of testing involved) would be worth my time. I've been building x265 from source for the last year or so. But I only use a handful of modifications and mainly for the AQ modes that aren't in mainline. Which brings me to my next and last question for now;

3) Auto-aq

Earlier this year I built a modded version of x265 from I believe version 3.6 or thereabouts. It included the auto-aq mode which I quickly found to be a huge benefit for the sources I work with. In the version I had it was invoked as so;

--aq-auto 10

Today I got a chance to catch up on things and noticed x265 had developed a lot since I was away. I also quickly found out that this mode no longer worked as it did just a few months ago. Through trail and error and some lurking on the git where it's hosted I found out I can now invoke it like this

--auto-aq

But I didn't see any discussion for why it changed and if you can still supply it with an option (e.g. 6 vs. 10 for HDR or SDR content). Does it automatically determine what kind of source you're working with now? Please advise. I spent a few hours this morning trying to find the exact commit where it was changed and why along with trying to grok posts from here. But there were over 20 pages of new discussion ITT and my searches of the forum's database proved futile.

I do have more questions about how x265 works and some of the settings I've been using for the last 2 years or so. I've gotten really good results at acceptable bitrates so I'm happy with them. But improvements are always welcomed of course. My workstation is getting a bit long in the tooth these days and I'm due for an upgrade. So I'm not able to do as much testing as I'd like. My usual config is already lucky to obtain 1fps. Which means a typical project for me takes several days and sometimes weeks to finish. Although the whole manually creating qpfiles and the type of editing/filtering I do takes up far more time. As I'm usually stepping through things frame-by-frame and splicing together sometimes 4 different sources to restore the content I work with.

This is getting verbose so I'll hold off on asking more questions even though I'm full of them. It's good to be back. I'm happy to see this community is still very active. I prefer using forums to these modern locked down social media networks or God forbid Discord.

Also, thanks again to the kind admin that allowed me to post this today instead of a week from now.

[Z2697]

Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

[LunaRabbit]

Originally Posted by Z2697

Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

Yeah I understand that part of it. In their defense early mbtree wasn't that great from what I remember. But I'm sure it improved some after I stopped paying attention and got busy with life.

My limited understanding of cutree is that it isn't the same as mbtree at all. From what I understand cutree is more about looking ahead and attempting to either insert more I-frames or shift data in a GOP towards I-frames. With the hope that the following P/B frames will benefit and thus be smaller. I might be wrong but I've always understood it as being more like lookahead than what mbtree was. But I don't pretend to understand the inner workings of mbtree either.

It's just frustrating that the only discussion I see about cutree is either "turn it off it'll destroy quality" or "leave it on but I don't really know what it's doing. It just saves bits". I want to have a better understanding of how it works.

I do know for my uses/config it does save bitrate without affecting quality much that I can see. I suppose if I pixel peep I could spot very mild drop in quality here and there. But over the course of an entire episode or movie it seems to save a lot of bitrate and produce something of equal quality if nothing else is changed.

But perhaps that has more to do with the fact that I used closed GOPs and usually VBV. I do understand GOPs in x265 fine. I think I understand VBV as well and I don't think it really hurts quality that much the way I use it. I use it to impose an upper limit on the buffer so my encodes play nicer with some set-top devices and over the network. As I want to stream my stuff over my LAN and sometimes over WAN when I'm away from home. I was going to ask about VBV another time. But I only bring it up because maybe that's why I'm not seeing any improvement with cutree off.

The bitrate savings for me with cutree are very good. In a clip of say 100 frames with cutree on and closed_GOP set to 24 frames I'll produce a video stream that's about 1.5-2.0MB. The --no-cutree version would be 3.5-4MB with no gain in quality that I can see. Although I do "reel it in" with qcomp. But perhaps that's placebo. I'm mainly doing that in an attempt to prevent much variation between keyframes and non-keyframes.

Some guidance concerning cutree, how it relates to qcomp and if these modifications to tweak cu-tree are worth exploring would be very helpful and appreciated. Since I was surprised to find so little discussion about it outside of; "turn it off" or "if you're leaving it on make sure to raise qcomp a bit".

I feel like there are a lot of things like this in x265 that is just advice being repeated for the last 10+ years from people assuming it works just like early x264. Some time way back when everyone just decided to do things a certain way and never bothered to do tests as the underlying code and the codec itself changed.

I do know that people claim cutree can hurt backgrounds and scenes with a lot of movement. But in my experience it doesn't. But again. Perhaps I'm not seeing these issues because 1) I use a low CRF for 720p-1080p content anyway and 2) I regularly boost bitrate through the use of zones.

Do you use cutree?

[GeoffreyA]

As you point out, cutree saves a lot of bitrate, sometimes close to -50% in anime, and the same goes for mbtree. Those savings can go to smaller CRFs for example. So, I do not understand what people gain by disabling these tools. There is a small increase in quality on certain parts of the frame, but you've got to look for it, and I think one would better spend bits by lowering the CRF.

Recently, encoding a set of anime at reference quality, I settled on leaving cutree on and using a low CRF (14). I also set the rc-lookahead to 240 because a longer lookahead is said to be more beneficial for cutree.

I think if one is encoding anime in particular, disabling mbtree or cutree is a massive waste of bits, and a 2-pass encode would likely suffer more for it. As for how it works, my limited understanding is that it increases the quantiser for parts of a frame that are less referenced temporally. Dark Shikari wrote a paper about mbtree back in the day but it was a bit technical.