Comparisons of x265 vs x264

[sneaker_ger]

Since it has been asked to move comparisons of x265 vs other codecs out of the main x265 thread I'm starting this one.

I used a low quality, grainy, blockbuster source with about 8 Mbit/s as the output bitrate (x265 --crf 21). If you substract some bitrate because of this being a trailer with many scenecuts you come into a region often used for Blu-Ray-rips. This test uses the new psy features of x265 (still experimental). I haven't written down the fps, just believe me when I say it was too slow for encoding complete films.

My subjective conclusion:
x264 is still ahead in this test though both would be transparent under normal viewing conditions. Since they aren't too different I wouldn't be surprised if others were to come to a different conclusion.

Original (dl)
x264 8bit r2452 --preset placebo --tune grain (2pass) (dl)
x264 10bit r2452 --preset placebo --tune grain (2pass) (dl)
x265 1.2.436 10bit --preset placebo --psy-rd 0.5 --psy-rdoq 0.5 --crf 21 (dl)

#64, all B
http://abload.de/img/a_originalvfqkp.png
http://abload.de/img/a_x264_8bit6fq7g.png
http://abload.de/img/a_x264_10bit8eqbr.png
http://abload.de/img/a_x265_10bitapp2t.png

#606, all I
http://abload.de/img/b_original8po9k.png
http://abload.de/img/b_x264_8bit95rif.png
http://abload.de/img/b_x264_10bite3rp5.png
http://abload.de/img/b_x265jho08.png

#1326, all P
http://abload.de/img/c_originaly9rf1.png
http://abload.de/img/c_x2640rpv0.png
http://abload.de/img/c_x264_10bitlxq7h.png
http://abload.de/img/c_x265b6rcy.png

#1472, x264: P, x265: B
http://abload.de/img/d_original3jpl4.png
http://abload.de/img/d_x264vkrw2.png
http://abload.de/img/d_x264_10bitixqwf.png
http://abload.de/img/d_x2658qp01.png

#1610, all B
http://abload.de/img/e_originalhjrxf.png
http://abload.de/img/e_x2647spwg.png
http://abload.de/img/e_x264_10bitwrp38.png
http://abload.de/img/e_x265m2ogi.png

#1831. all B
http://abload.de/img/f_originalbaeaq.png
http://abload.de/img/f_x26458dzb.png
http://abload.de/img/f_x264_10bit8hidb.png
http://abload.de/img/f_x265yqivq.png

#2068, all B
http://abload.de/img/g_originalnzd2x.png
http://abload.de/img/g_x264ssdyg.png
http://abload.de/img/g_x264_10bit6ke1i.png
http://abload.de/img/g_x265b7czq.png

I think for future tests I'll try out higher quality sources and a slightly lower bitrate. The biggest problem I see for x265 is the speed. If it needs the new psy features to compete with x264 it's way too slow (needing at least preset slower to work).

[benwaggoner]

Originally Posted by sneaker_ger

x264 8bit r2452 --preset placebo --tune grain (2pass) (dl)
x264 10bit r2452 --preset placebo --tune grain (2pass) (dl)
x265 1.2.436 10bit --preset placebo --psy-rd 0.5 --psy-rdoq 0.5 --crf 21 (dl)

Why not 2-pass for x265 as well. It should be working now.

[sneaker_ger]

2pass is kinda experimental as well, last I heard. Plus I didn't want to spend even more time encoding.

Originally Posted by sneaker_ger

2pass is kinda experimental as well, last I heard. Plus I didn't want to spend even more time encoding.

2 Pass is working nicely now. The first pass defaults to a turbo mode.

[benwaggoner]

Originally Posted by x265_Project

2 Pass is working nicely now. The first pass defaults to a turbo mode.

Do you see a measurable quality delta between a turbo and a regular first pass, or is it miniscule like with x264.

[pandy]

Are you sure that frame type is exactly the same for x264 and x265? I found unfair to compare I frame vs B frame and it is hardly to believe that encoders decision make this exactly repeatable but... this is my question - are we compare apples to apples?

[sneaker_ger]

They always match except for frame #1472/D. I have added the types into the post.

[Daemon404]

Originally Posted by pandy

Are you sure that frame type is exactly the same for x264 and x265? I found unfair to compare I frame vs B frame and it is hardly to believe that encoders decision make this exactly repeatable but... this is my question - are we compare apples to apples?

It's fundamentally flawed to compare static screenshots anyway, because rate control is a thing.

[Sagittaire]

Originally Posted by pandy

Are you sure that frame type is exactly the same for x264 and x265? I found unfair to compare I frame vs B frame and it is hardly to believe that encoders decision make this exactly repeatable but... this is my question - are we compare apples to apples?

x264 and x265 are certainely comparable IFrame decision. The problem is more to compare PFrame vs BFrame vs bFrame.

x264 is still ahead in this test though both would be transparent under normal viewing conditions. Since they aren't too different I wouldn't be surprised if others were to come to a different conclusion.

Really difficult here for me to find the best. Quality level at crf 21 is certainely to high for that. x264 and x265 produce really comparable output here.

I think for future tests I'll try out higher quality sources and a slightly lower bitrate. The biggest problem I see for x265 is the speed. If it needs the new psy features to compete with x264 it's way too slow (needing at least preset slower to work).

IMO here the best compromise for quality/time:

--preset slower --me hex --no-rect --no-amp --rd 4 --aq-mode 2 --aq-strength 0.5 --psy-rd 1.0 --psy-rdoq 0.2 --bframes 3 --min-keyint 1

[foxyshadis]

Originally Posted by Daemon404

It's fundamentally flawed to compare static screenshots anyway, because rate control is a thing.

Sure, but that's why the streams are provided for anyone who cares enough to look deeper. Although it's obvious which is which (all frames of x265 are slightly softer), it's no longer easy to prefer one over the other, especially in motion. x264 doesn't "pop" that much more like it did just a couple months ago, and I find both equally pleasant.

Of course, equal is a low bar, given the speed penalty, but at least it's not subjectively worse any longer.

[benwaggoner]

Originally Posted by Sagittaire

x264 and x265 are certainely comparable IFrame decision. The problem is more to compare PFrame vs BFrame vs bFrame.

The right frame type decision may be different with different codecs, and I think each encoder should be allowed to make its own optimal decisions. I expect that we'll see more divergence in optimal frame type selection as x265 matures.

Really difficult here for me to find the best. Quality level at crf 21 is certainely to high for that. x264 and x265 produce really comparable output here.

I don't know that we should expect the CRF values to be that comparable, particularly with psychovisual optimizations. I think fixed bitrate comparisons at a low enough bitrate that both codecs show visible artifacts are going to be the most interesting way to test right now. with --bitrate, --vbv-bufsize, and --vbv-maxrate all set the same.

IMO here the best compromise for quality/time:

I think comparing at the same encoding time, and comparing without encoding time limit, are both interesting at this point. Full use of HEVC features is always going to be slower than full use of H.264 features, although HEVC has more potential for parallel and GPU acceleration which may eventually close the gap.

Note that none of the presets turn on --weightb at this point, presumably because the feature was added after the presets were finished. That's probably useful to add to the comparison, at least when comparable speed isn't a testing goal. I'm not sure what the speed impact is.

Also, I'm not sure how comparable the motion search ranges are between x264 and x265

[sneaker_ger]

Originally Posted by benwaggoner

I don't know that we should expect the CRF values to be that comparable, particularly with psychovisual optimizations. I think fixed bitrate comparisons at a low enough bitrate that both codecs show visible artifacts are going to be the most interesting way to test right now. with --bitrate, --vbv-bufsize, and --vbv-maxrate all set the same.

The bitrates in this test are roughly the same. I started with x265 crf and matched the output with x264 2pass.

[Sagittaire]

I don't know that we should expect the CRF values to be that comparable, particularly with psychovisual optimizations. I think fixed bitrate comparisons at a low enough bitrate that both codecs show visible artifacts are going to be the most interesting way to test right now. with --bitrate, --vbv-bufsize, and --vbv-maxrate all set the same.

crf 25 is in 1000-2000 kbps interval for 1080p real source movie. With this quality level you can expect to have 1080p movie on 1.4 Go with really goog quality. If you want show the real potential for HEVC, it's the way.

Note that none of the presets turn on --weightb at this point, presumably because the feature was added after the presets were finished. That's probably useful to add to the comparison, at least when comparable speed isn't a testing goal. I'm not sure what the speed impact is.

weightb is not really important if you have weightp. Don't expect to have really higher efficeincy with that.

[sneaker_ger]

A new test with the same source. This time lower bitrate and more concentrated on viable speeds.

x264 8bit r2452 64 bit (8 bit for hardware compatibility)
x265 10bit 1.2.474 64 bit
2209 frames
Core i7-860
all 2pass --bitrate 4000

Original

x264 (default) (dl)
pass 1: 60,52 fps
pass 2: 23,33 fps

x264 --preset veryslow --tune grain (dl)
pass 1: 23,72 fps
pass 2: 4,32 fps

x265 (default) (dl)
pass 1: 445,05s, 4,96 fps
pass 2: 541,47s, 4,08 fps

x265 --preset slow (dl)
pass 1: 538,56s, 4,10 fps
pass 2: 1799,03, 1,23 fps

x265 --preset slower --psy-rd 0.5 --psy-rdoq 0.5 (dl)
pass 1: 2158,99s, 1,02 fps
pass 2: 6530,43s, 0,34 fps

x265 "Sagittaire" --preset slower --me hex --no-rect --no-amp --rd 4 --aq-mode 2 --aq-strength 0.5 --psy-rd 1.0 --psy-rdoq 0.2 --bframes 3 --min-keyint 1 (dl)
pass 1: 756,51s, 2,92 fps
pass 2: 899,66s, 2,46 fps

#64, all B
http://abload.de/img/a_originalvfqkp.png
http://abload.de//img/a_x264_mediumglusb.png
http://abload.de//img/a_x264_veryslowiquk1.png
http://abload.de//img/a_x265_mediumbiuiz.png
http://abload.de//img/a_x265_slowqeuz4.png
http://abload.de//img/a_x265_sloweru2u0i.png (Ugly artifact here. Not sure if x265 or ffmpeg decoder error.)
http://abload.de//img/a_x265_sagittaireewuwb.png

#606, all I
http://abload.de/img/b_original8po9k.png
http://abload.de//img/b_x264_mediumzfuew.png
http://abload.de//img/b_x264_veryslowyduki.png
http://abload.de//img/b_x265_mediumxju99.png
http://abload.de//img/b_x265_slowzpu1p.png
http://abload.de//img/b_x265_slowerf1ucf.png
http://abload.de//img/b_x265_sagittairebju2t.png

#1326, all P
http://abload.de/img/c_originaly9rf1.png
http://abload.de//img/c_x264_mediumsluha.png
http://abload.de//img/c_x264_veryslowr1u46.png
http://abload.de//img/c_x265_mediumeuuyw.png
http://abload.de//img/c_x265_slowxdun2.png
http://abload.de//img/c_x265_slowerkzu2n.png
http://abload.de//img/c_x265_sagittairefzu10.png

#1610, all B
http://abload.de/img/e_originalhjrxf.png
http://abload.de//img/e_x264_mediumrslu2.png
http://abload.de//img/e_x264_veryslowvxb83.png
http://abload.de//img/e_x265_mediumu7b53.png
http://abload.de//img/e_x265_slow2wbhb.png
http://abload.de//img/e_x265_slowercoym8.png
http://abload.de//img/e_x265_sagittairegoa2t.png

#1831, all B
http://abload.de/img/f_originalbaeaq.png
http://abload.de//img/f_x264_medium85ldo.png
http://abload.de//img/f_x264_veryslow5ub14.png
http://abload.de//img/f_x265_mediumjclox.png
http://abload.de//img/f_x265_slow3sajv.png
http://abload.de//img/f_x265_slower92bsw.png
http://abload.de//img/f_x265_sagittairepbahw.png

#2068, all B
http://abload.de/img/g_originalnzd2x.png
http://abload.de//img/g_x264_mediumyparn.png
http://abload.de//img/g_x264_veryslow2yz2q.png
http://abload.de//img/g_x265_medium9sabc.png
http://abload.de//img/g_x265_slowpjawy.png
http://abload.de//img/g_x265_slowerq0y5q.png
http://abload.de//img/g_x265_sagittaireimac0.png

[Sparktank]

Rather interesting. I've been curious as to how x265 is coming along.

I will say that for x265 being in early development, it's still not too shabby.
I find the -slower preset to be visually pleasing more than the others, but those encoding speeds...
Sagittaire's comes very close to the slower preset without modifying anything.

Upon a second watch between Sagittaire and Slower, there's virtually little difference.
A reasonable compromise to get a speed boost for encoding.

It's nice to see so many updates for x265 in recent months.

Thanks for the testing.

[Sagittaire]

Originally Posted by sneaker_ger

A new test with the same source. This time lower bitrate and more concentrated on viable speeds.

x264 8bit r2452 64 bit (8 bit for hardware compatibility)
x265 10bit 1.2.474 64 bit
2209 frames
Core i7-860
all 2pass --bitrate 4000

Here the setting to have symetrical setting with x264 and x265 (x264 with preset "grain" use low ratio for frametype to have more stable quality on I,P,B,b transition and better stability on grain retention). With high ratio for bframe, you have really higher quantizer on Bframe, but lower on Pframe too.

--preset slower --me hex --no-rect --no-amp --rd 4 --aq-mode 2 --aq-strength 0.5 --psy-rd 1.0 --psy-rdoq 0.2 --bframes 3 --min-keyint 1 --ipratio 1.1 --pbratio 1.1

[benwaggoner]

Originally Posted by sneaker_ger

A new test with the same source. This time lower bitrate and more concentrated on viable speeds.

x264 8bit r2452 64 bit (8 bit for hardware compatibility)
x265 10bit 1.2.474 64 bit

I think both encodes should use the same bit depth to get source and playback dithering out of the comparison.

2209 frames
Core i7-860
all 2pass --bitrate 4000

Is this a low enough bitrate for the content so there are reasonably visible artifacts in the x264 encode? We want to be testing in the range where bitrate has a clear visible impact on the encodes.

Also, you should be specifying the same --vbv-maxrate and --vbv-bufsize.

[Sleepysonic]

I'd really like to see a 2000 bitrate comparison!

[fumoffu]

wait wait wait...
are those comparisons done with the same bitrate? and there isn't really difference in quality and often x264 on default looks better than x265 on slower?
what the hell?!?
I understand that promised 50% reduction isn't always realistic, but come on, what is the point of all this if we can't beat old standard even with much longer encoding and more hardware demanding decoding.
It looks like for now it's better to use 10bit x264, it doesn't have any hardware support either but at least it can give realistic 10% or more bitrate savings.

[LoRd_MuldeR]

Originally Posted by fumoffu

wait wait wait...
are those comparisons done with the same bitrate? and there isn't really difference in quality and often x264 on default looks better than x265 on slower?
what the hell?!?

If they were different bitrates, the whole test would be completely pointless! So I'm pretty sure they are the same

The big question, however, is: Was the bitrate chosen reasonable for what we want to test?

Currently I think there are (at least) two test scenarios:

• Bitrates where x264 still looks good. In this scenario, the question is whether x265 can preserve the same amount of details as x264 does (seemed to be a problem for x265 in the past! not quite sure now).
• Bitrates low enough that x264 (and H.264 in general) starts getting serious problems. Here we expect x265 (HEVC) to really shine...

Originally Posted by fumoffu

I understand that promised 50% reduction isn't always realistic, but come on, what is the point of all this if we can't beat old standard even with much longer encoding and more hardware demanding decoding.

Keep in mind how many years it took until x264 reached the quality and the encoding speed that it provides nowadays. Also, the hardware getting faster over the years certainly helped quite a bit

At the same time HEVC is a relatively new standard and x265 is a relatively young project. So patience!