Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th June 2019, 17:45   #1  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
Trying to find the equivalent settings from h264 to HEVC

Hi everyone,

This is my first post here and I only started to try and encode my own shows over the last month. I first tried Handbrake and while it was good, I felt it could be better, so here I am. Now on to the the problem I'm having that I hope is solvable.

So I'm primarily working with anime and I found an encode that is as close to perfect as I can see. What's more the settings were documented. The problem is these are for encoding in h264 and I'm hoping to take my raw media I own and convert them to HEVC.

Thankfully I realize that the h264 settings aren't a 1:1 comparison to HEVC. However I am hoping that smarter and more experienced individuals could assist me in understanding what these settings are doing in h264 and locating a way to hopefully duplicate them in h265.

Okay so enough yammering here are the h264 settings:

10-bit depth
Encoding settings:
Code:
cabac=1 / ref=16 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / fade_compensate=0.50 / psy_rd=0.93:0.22 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=18,9 / fast_pskip=0 / chroma_qp_offset=-3 /
[br]threads=4 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / stitchable=1 / constrained_intra=0 / fgo=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=480 /
keyint_min=1 / scenecut=60 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=15.0000 / qcomp=0.80 / qpmin=20:20:20 / qpmax=40:40:40 / qpstep=4 / ip_ratio=1.25 / aq=3:0.84 / aq-sensitivity=10.00 / aq-factor=1.00:1.00:1.00 / aq2=0 / aq3=0
Any suggestions would be incredibly helpful. Thank you again.

Last edited by turinggirl; 12th June 2019 at 18:42.
turinggirl is offline   Reply With Quote
Old 12th June 2019, 19:56   #2  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 361
Quote:
Originally Posted by turinggirl View Post
So I'm primarily working with anime
...
I recommend bframes=16 for cartoon/anime.


Quote:
Animation

--tune animation adjusts encoder settings to optimize the encode quality for animation content without impacting the encode speed. This is done by:

--psy-rd 0.4
--aq-strength 0.4
--deblock 1:1
--bframes Increase by 2
https://x265.readthedocs.io/en/default/presets.html

--deblock 1:1 might be a bit high though, your deblock is OK, the picture get softer with high deblock-value.

ref=6 should be set maximally if you want GPU-decoding support for the GPU that supports H.265/HEVC hardware decoding.

Overall, I'd suggest this x265-line for anime/cartoon,
x265 --crf 18 --preset slower --profile main10 --output-depth 10 --rc-lookahead 60 --aq-mode 1 --aq-strength 0.4 --psy-rd 0.4 --frame-threads 2 --no-sao --deblock -2:-2 --qcomp 0.70 --ref 6 --bframes 16 --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 4

But you set crf=15 in x264, so maybe --crf 17 should be used instead in x265, for more bitrate but higher quality.

Last edited by Forteen88; 12th June 2019 at 20:06.
Forteen88 is offline   Reply With Quote
Old 12th June 2019, 20:09   #3  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
I'll give this a shot on my testing footage and get back to you. Thanks!
turinggirl is offline   Reply With Quote
Old 12th June 2019, 20:55   #4  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
From what I can tell it seems to be working great, the only issue I seem to be having is that the lines, at times look a little soft. I'm not sure if that's just the nature of HEVC and I need to live with it.


I tested it on my dark scene footage and there was a huge amount of blocky colors as well as moire patterns in some places like on the walls. I'm not sure how to correct this.

Last edited by turinggirl; 12th June 2019 at 22:23.
turinggirl is offline   Reply With Quote
Old 12th June 2019, 21:45   #5  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 361
You can also try this x265-build (by Doom9-member MeteorRain) that was made for anime,
https://github.com/msg7086/x265-Yuuki-Asuna

Last edited by Forteen88; 12th June 2019 at 21:48.
Forteen88 is offline   Reply With Quote
Old 12th June 2019, 22:58   #6  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
Quote:
Originally Posted by Forteen88 View Post
You can also try this x265-build (by Doom9-member MeteorRain) that was made for anime,
https://github.com/msg7086/x265-Yuuki-Asuna
haha funny enough that's exactly what I'm using!

Okay so here are the files.

https://drive.google.com/open?id=1I9...nXNjMneAnE_o1S

The largest one is the Raw reference file. and the rest are the different encodes with adjusted settings.

If you pay attention to the walls specifically you can see what I think is banding.

Last edited by turinggirl; 12th June 2019 at 23:32.
turinggirl is offline   Reply With Quote
Old 13th June 2019, 16:56   #7  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
That's my biggest issue right now is trying to figure out the proper setting for animation to deal with low brightness low contrast scenes and the color banding that's occurring. In bright high contrast shows the setting suggested above work fantastic.
turinggirl is offline   Reply With Quote
Old 13th June 2019, 19:11   #8  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,531
You could try aq-mode 3, you probably need to lower aq-strength from the default 1.0 a bit though.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 13th June 2019, 21:01   #9  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 361
Quote:
Originally Posted by turinggirl View Post
That's my biggest issue right now is trying to figure out the proper setting for animation to deal with low brightness low contrast scenes and the color banding that's occurring. In bright high contrast shows the setting suggested above work fantastic.
Many encoding-people uses --zones at dark scenes, and sets higher bitrate there. Maybe you should lower CRF from 17 to 16 or 15 if you don't want to bother with optimizing the settings by looking at what scene startframe and endframe you need to set --zones on.

I hear that higher --aq-strength helps against banding in x264, but I don't know if it does that in x265 too. As 'Boulder' suggested, you should try --aq-mode 3
Maybe you need to deband more in AviSynth+/VapourSynth if the source got a bit banding.

Last edited by Forteen88; 14th June 2019 at 18:44.
Forteen88 is offline   Reply With Quote
Old 13th June 2019, 23:13   #10  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
Quote:
Originally Posted by Boulder View Post
You could try aq-mode 3, you probably need to lower aq-strength from the default 1.0 a bit though.
I'll give that a try tonight, I was wondering if you had any suggestions as to where I should aim for the aq-strength, would it be like .92 or something more like .5? Thanks again for all the help I really appreciate it.
turinggirl is offline   Reply With Quote
Old 13th June 2019, 23:14   #11  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
Quote:
Originally Posted by Forteen88 View Post
Many encoding-people uses --zones at dark scenes, and sets higher bitrate there. Maybe you should lower CRF from 17 to 16 or 15 if you don't want to bother with looking at what scene startframe and endframe you need to set --zones on.

I hear that higher --aq-strength helps against banding in x264, but I don't know if it does that in x265 too. As 'Boulder' suggested, you should try --aq-mode 3
Maybe you need to deband more in AviSynth+/VapourSynth if the source got a bit banding.
I haven't been using Avisynth+/VapourSynth. I assume this is going to be one those 'down the rabbit hole' type past times isn't it?
turinggirl is offline   Reply With Quote
Old 14th June 2019, 00:59   #12  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 361
Quote:
Originally Posted by turinggirl View Post
I haven't been using Avisynth+/VapourSynth. I assume this is going to be one those 'down the rabbit hole' type past times isn't it?
Yeah, it's like a hobby
Forteen88 is offline   Reply With Quote
Old 14th June 2019, 02:03   #13  |  Link
turinggirl
Registered User
 
Join Date: May 2019
Posts: 8
Quote:
Originally Posted by Boulder View Post
You could try aq-mode 3, you probably need to lower aq-strength from the default 1.0 a bit though.
So I changed --aq-mode to 3 and aq-strength to .6 and from what I can tell it seems to have done the trick.

Raw:
https://drive.google.com/file/d/14n7...ew?usp=sharing

Encode: https://drive.google.com/file/d/1mRF...ew?usp=sharing
turinggirl is offline   Reply With Quote
Old 14th June 2019, 14:48   #14  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 129
Quote:
Originally Posted by Boulder View Post
You could try aq-mode 3, you probably need to lower aq-strength from the default 1.0 a bit though.
would u recommend aq mode 1 + aq strength 1 over aq 3 st 0.80? how do they actually work and whats the difference? i recently tested aq mode 1 along with f3kdb and got good results in low bitrates but i would like to know what experts think and how it works
Natty is offline   Reply With Quote
Old 14th June 2019, 16:20   #15  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,531
I haven't done any extensive testing myself, I trust a few users who are quite experienced in dealing with different material. Aq-mode 2 is the default mode nowadays, and the only difference between that and mode 3 is that mode 3 has a bigger bias towards dark areas. That's why in theory, it should be better for SDR material and especially anime etc. where banding is easily seen. The downside is that bitrate will be much higher so a lower aq-strength should compensate that.

Currently I use aq-mode 2 and aq-strength 0.6-0.8 depending on the source. 0.6 for noisy and grainy and higher for cleaner material.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th June 2019, 22:37   #16  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 129
Quote:
Originally Posted by Boulder View Post
Currently I use aq-mode 2 and aq-strength 0.6-0.8 depending on the source. 0.6 for noisy and grainy and higher for cleaner material.
Natty is offline   Reply With Quote
Old 15th June 2019, 08:45   #17  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 361
Quote:
Originally Posted by turinggirl View Post
...
Okay so enough yammering here are the h264 settings:
Code:
keyint=480
...
You might also wanna set --keyint 480
in x265. The compression will be slightly better, but the seeking will not be that good then.

Last edited by Forteen88; 19th June 2019 at 12:18.
Forteen88 is offline   Reply With Quote
Old 19th June 2019, 05:21   #18  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 57
So what is your final command line? I only ask because there are several great posts about anime on here, but each of them are from different versions of x265 with different effects? from those versions. ie the new Lamda tables, the new speed changes, etc. It would be great so we can compare how different settings have changed from older versions of x265. I believe the last set I am using is from Meteor Rain, or maybe Wolfberry, as the results were fantastic. I can't understand all the intricate commands anymore :P Might be time to upgrade haha.
RanmaCanada is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.