x265 HEVC Encoder

[Vesdaris]

Originally Posted by Blowis

After several tests I managed to approach the x264.
To do that I use medium profile I change ctu = 16 psy-rd = 1.00 rdoq-level = 2 psy-rdoq = 1.10.

What about CRF?

[LigH]

CRF limits the quality loss. The smaller rate factor, the less loss, the bigger the result. Find your own personal threshold of "visual transparency" (how much loss you find hardly noticable).

[luigizaninoni]

Originally Posted by burfadel

I also found adding --early-skip gave noticeably better performance without a quality or file size hit (or so fractionally slight the speed gain far outweighed any 'loss').

Basically --qg-size 16 is a must, and I believe --aq-mode 3 is as well .

I tried your suggestions of early-skip and aq-mode 3.

I really like early-skip a lot, speed increase is incredible (about + 60%) and quality loss is minimal.

As far as aq-mode 3 is concerned, I don't know ... it increased bitrate by 25%, admittedly on a darkish clip. You probably could just bump down crf a notch or two and obtain similar results.

[Blowis]

Originally Posted by vesdaris

what about crf?

I used CRF 21-22 or 23. For more visual detail is not the largest on the X265.

[Vesdaris]

OK..I've been testing for some time different settings encodign a small sample file. ..
x264 (4380 Kbps) crf18,10bit,very slow

cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / vbv_maxrate=150000 / vbv_bufsize=187500 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=2:1.00

x265 (2933 Kbps) crf 20, 10bit, medium, tune=grain, rdoq-level=2, psy-rdoq=1.00

wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=6 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=0 / limit-modes / weightp / no-weightb / aq-mode=1 / qg-size=32 / aq-strength=0.30 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=0.50 / rdoq-level=2 / psy-rdoq=1.00 / signhide / deblock=-2:-2 / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=20.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.10 / pbratio=1.10

http://screenshotcomparison.com/comparison/153828

It's almost identical. x265 lacks a tiny bit of details here and there but it seems it's really close and encoding speed was nice.

I could prolly increase psy-rd a bit.or maybe change something else...What do you think?

[Blowis]

Originally Posted by Vesdaris

OK..I've been testing for some time different settings encodign a small sample file. ..
x264 (4380 Kbps) crf18,10bit,very slow


x265 (2933 Kbps) crf 20, 10bit, medium, tune=grain, rdoq-level=2, psy-rdoq=1.00



http://screenshotcomparison.com/comparison/153828

It's almost identical. x265 lacks a tiny bit of details here and there but it seems it's really close and encoding speed was nice.

I could prolly increase psy-rd a bit.or maybe change something else...What do you think?

Use ctu = 16 after increasing the psy-rd = 1.00 and = 1.10 psy-rdoq and also try aq-mode = 2. Especially increases psy-rd and psy-rdoq to sharpen the image
Here are but setting after several tries optimal and fast Profile Medium:

wpp / ctu=16 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=2 / psy-rdoq=1.10 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=22.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

On my i7 4774k@4.3Ghz i am between 12.25 to 14.45 fps according to the video.
For a video of 1 hour 40 1080p encoding lasts 3 hours 30.

[Vesdaris]

I'm getting insta crash if I use anything past this (ur settings)

--wpp --ctu=16 --min-cu-size=8 --max-tu-size=16 --tu-intra-depth=3 --tu-inter-depth=3 --me=3 --subme=2 --merange=32 --no-rect --no-amp --max-merge=2 --temporal-mvp --early-skip --rdpenalty=0 --no-tskip --no-tskip-fast --strong-intra-smoothing --no-lossless --no-cu-lossless --no-constrained-intra --no-fast-intra --open-gop --no-temporal-layers --interlace=0 --keyint=240 --min-keyint=23 --scenecut=40 --rc-lookahead=20 --lookahead-slices=5 --bframes=4 --bframe-bias=0 --b-adapt=2 --ref=3 --limit-refs=1

I guess I'm doing something wrong lol

[burfadel]

Originally Posted by luigizaninoni

I tried your suggestions of early-skip and aq-mode 3.

I really like early-skip a lot, speed increase is incredible (about + 60%) and quality loss is minimal.

As far as aq-mode 3 is concerned, I don't know ... it increased bitrate by 25%, admittedly on a darkish clip. You probably could just bump down crf a notch or two and obtain similar results.

Yeah. The quality difference was imperceptible for me, it really only shows if doing a SSIM and PSNR comparison. You can make up and surpass the quality drop by using better settings elsewhere without losing too much of that gain. By using early skip, I also believe the speed penalty of some other settings is minimised, making them viable to use.

Why isn't. --b--intra on by default? What disadvantages are there using --me star seeing as it is so fast?

In any case, when all the settings I use are taken into account, for me at least it is the most suitable outcome speed and performance wise regardless of content. I do use higher b frames, lower aq with animation.

[Ma]

Originally Posted by Vesdaris

I'm getting insta crash if I use anything past this (ur settings)

--wpp --ctu=16 --min-cu-size=8 --max-tu-size=16 --tu-intra-depth=3 --tu-inter-depth=3 --me=3 --subme=2 --merange=32 --no-rect --no-amp --max-merge=2 --temporal-mvp --early-skip --rdpenalty=0 --no-tskip --no-tskip-fast --strong-intra-smoothing --no-lossless --no-cu-lossless --no-constrained-intra --no-fast-intra --open-gop --no-temporal-layers --interlace=0 --keyint=240 --min-keyint=23 --scenecut=40 --rc-lookahead=20 --lookahead-slices=5 --bframes=4 --bframe-bias=0 --b-adapt=2 --ref=3 --limit-refs=1

I guess I'm doing something wrong lol

I tried to reproduce this crash but I wasn't able (console output attached). Can you specify more details?

[Vesdaris]

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

[foxyshadis]

Originally Posted by Vesdaris

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

You can find the actual command-line in Megui's log.

[Ma]

Originally Posted by Vesdaris

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

Thanks for some details. It could be related to outdated x265 version.

Problem with command line that hangs x265 was reported in P.S. part of this message http://forum.doom9.org/showthread.ph...09#post1746309

Fix for this bug is in https://bitbucket.org/multicoreware/...94dde4afadad6b

You can update x265. (I'm only guessing.)

[foxyshadis]

Originally Posted by Blowis

Use ctu = 16

It's very disappointing that x265 is still so bad at mode decision that restricting ctu to 16 is useful, since that eliminates one of the key advantages over AVC. But I tested it, and it really does make the picture sharper even if it slightly increases noise, and retains grain better. Maybe psy-rd also needs to directly correspond to a much stronger version of --rd-penalty, where the higher the psy-rd the higher the threshold for 64x64 or 32x32 ctu becomes, so that larger can still be used when warranted, and the value of psy-rd directly indicates how much grain/fine detail the user is interested in keeping.

[Blowis]

Originally Posted by foxyshadis

It's very disappointing that x265 is still so bad at mode decision that restricting ctu to 16 is useful, since that eliminates one of the key advantages over AVC. But I tested it, and it really does make the picture sharper even if it slightly increases noise, and retains grain better. Maybe psy-rd also needs to directly correspond to a much stronger version of --rd-penalty, where the higher the psy-rd the higher the threshold for 64x64 or 32x32 ctu becomes, so that larger can still be used when warranted, and the value of psy-rd directly indicates how much grain/fine detail the user is interested in keeping.

For me the ctu16 it is best to have the detail with the ctu64 so smooth it was a clear image but less detail.
Here are photos and encodings (for the ctu64 i put RD Penalty: 2 Psy-RD: 1.20 Psy-RDOQ: 1.30, if I put as the encoding of ctu16 smooth it over):

wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.20 / rdoq-level=2 / psy-rdoq=1.30 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Link Image ctu64: http://hpics.li/686abe6

wpp / ctu=16 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=2 / psy-rdoq=1.10 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Link Image ctu16: http://hpics.li/93b5428

You see the differences on the teeth, lips, eyes on the ctu64 was a smoother face so clear but loses detail.
Here is the origin of the image x264: http://hpics.li/28b34fb

After a 80 inch TV i see no difference. For me the best is ctu16 all cases.

[undfeatable]

Originally Posted by benwaggoner

I can't speak to Handbreak specifically (it's more of a heartbreak for me whenever I've tried to us it for anything mildly interesting ).

But there are a bunch of somewhat esoteric parameters that need to be set for a proper HDR-10 bitstream. Looking at a test file I did a few months ago:

--colorprim bt2020
--transfer 16
--colormatrix bt2020nc
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,0)"
--max-cll "1000,274"
--chromaloc 2

And, of course, you'll need to know what your master-display, MaxCLL, and MaxFALL values are for your content.

And you'll need to have a >=10-bit source that's already in the SMPTE 2084 PQ space, not gamma.

Also, I note you're targeting Level 5.1. AFAIK, there aren't any existing HEVC Main10 decoders that support beyond Level 5.1. Which can do 2160p60 !

I have those parameters set already, but really could use some help. Im going to post about it on the actual HDR post though to keep this thread on topic.

Originally Posted by Ma

If the same binary works with SSE4.2 CPU and hangs with AVX2 CPU, you can add '--asm AVX' option or '--asm SSE4.2' option and retest.

Thanks! That solved it. I used --asm SSE4.2 and it now runs!

Question for the x265 crew though. Am I getting the full performance of my computer now or is it being hindered. I have a fully loaded 2015 MacBook pro w/ retina that has a 4.0GHz Intel i7 and 16gb of DDR3 ram. My practice clip is a 8 seconds long and 6GB uncompressed. It seems to take around 5-6 minutes to run through x265 on preset slow. Does that seem about right?

Side question, does everyone have to answer these questions before every post? Its frustrating to post and have to search around to answer a quest before a post!

[LigH]

You probably mean the anti-spam captchas for users with less than 5 posts?

[undfeatable]

Originally Posted by LigH

You probably mean the anti-spam captchas for users with less than 5 posts?

Thats probably it lol. Its the most annoying "captcha" Ive ever encountered. Its just a random question and I never get it right the first 2 or 3 times. Well, this is post 5 so hopefully its gone for good.

[foxyshadis]

Originally Posted by Blowis

For me the ctu16 it is best to have the detail with the ctu64 so smooth it was a clear image but less detail.

After a 80 inch TV i see no difference. For me the best is ctu16 all cases.

There is definitely more annoying gibbs noise in the ctu16, but it's outweighed by the very annoying huge solid blocks with sharp sides that ctu64 has. With psy-rd 1.2 and rd-penalty 2 that should have been avoided as much as possible already.

My post was about how you shouldn't need to disable one of HEVC's greatest strengths just to get an acceptable encode, though. There has to be a way to fix the use of large blocks so that it works in harmony better.

Originally Posted by foxyshadis

... you shouldn't need to disable one of HEVC's greatest strengths just to get an acceptable encode, though. There has to be a way to fix the use of large blocks so that it works in harmony better.

Agree completely. The ability to code 32x32 and 64x64 blocks is perhaps HEVC's greatest asset. When can do this with low visual distortion, you get a big improvement in encoding efficiency. If you have visible distortion with large CUs, it could naturally be much more visible than with smaller CUs. We've got some ideas about how to improve our analysis to select the best PU and TUs, understanding that the distortion measurements used in x265 (and every HEVC encoder) are imperfect. It's just a matter of having the time to experiment and implement these ideas. Of course, as always, suggestions and contributions are welcomed.

[littlepox]

Originally Posted by x265_Project

We've got some ideas about how to improve our analysis to select the best PU and TUs, understanding that the distortion measurements used in x265 (and every HEVC encoder) are imperfect. It's just a matter of having the time to experiment and implement these ideas.

Your efforts are always appreciated; we await the improvements.