So I finally have time to play with this plugin, it's very good tool to help find optimal encode settings for my contents.
My test clip is a 1000 frames of 1440x1080 anime with no grain and very little motion. Here are the comparison between x264 and x265 I made with vmaf :
Code:
Aggregate VMAF Percentage
source 98.8374
x264 300 kb/s medium animation 79.9185
x265 300 kb/s medium 85.6454
x265 300 kb/s slower 86.9443
x264 2500 kb/s medium animation 95.1616
x265 2500 kb/s medium 94.9675
x264 CRF 25 medium animation 91.3914
x265 CRF 25 medium 91.1363
x264 CRF 18 medium animation 96.1513
x265 CRF 21 medium 93.6972
x264 CRF 18 medium animation 96.1513
x264 CRF 18 slower animation 96.3279
According to the FAQ, comparing the source clip to itself won't gives 100% score, so 98.8% here is the highest quality for this clip.
For encoding in low bitrate, x265 clearly wins. But for high/transparent encoding, it seems that x264 is still better if encode in same bitrate or same CRF. I don't have good 4k contents to see how that goes.
I have heard people said that x264 crf 18 is equivalent to x265 crf 21 so I did a comparison on those too, and the score shows that there is a difference.
On the last comparison I did it between two different presets. So with high bitrate encoding for my content, using slower preset is just a waste of time with insignificant amount of gains. It only make sense to use it on low bitrate encodes.