Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th October 2019, 15:02   #21  |  Link
Nico8583
Registered User
 
Join Date: Jan 2010
Location: France
Posts: 752
Thank you redbtn for your command lines
And I'm also interested to get a Zopti script.
Nico8583 is offline   Reply With Quote
Old 15th October 2019, 18:26   #22  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
Quote:
Originally Posted by redbtn View Post
Thank you for advice! Can you share your script for Zopti comparison? I just looked Zopti topic, but it's a little complicated for me.
Code:
from vapoursynth import core
from zoptilib import Zopti

orig = core.ffms2.Source(source=r'ridgemont.avi') # lossless intermediate file containing 80-100 single frames throughout the whole movie (see below how I do it)
   
zopti = Zopti(r'results.txt', metrics=['mdsi', 'time'], matrix='709')

b = -110/100.0					# optimize b = _n_/100.0 | -175..-50 | b
c = -10/100.0					# optimize c = _n_/100.0 | -30..70 | c

alternate = core.resize.Bicubic(orig, width=1280, height=688, filter_param_a=b, filter_param_b=c) # this one downsizes the lossless clip to my final size
alternate = core.resize.Bicubic(alternate, width=3840, height=2064, filter_param_a=0, filter_param_b=0.5) # this one upsizes the downsized clip to the full 4K resolution as I have a 4K TV
orig = core.resize.Bicubic(orig, width=3840, height=2064, filter_param_a=0, filter_param_b=0.5) # the lossless clip (having the original resolution minus black borders) is upsized to 4K
zopti.run(orig, alternate) # we compare the upsized original to the downsized-upsized one to get as close as possible
To get the lossless clip, I simply load the source, add the denoising part etc. and then add these ones:

clp = core.std.SelectEvery(clp, 700, 1)
clp = core.std.Trim(clp, 2, 101)

So in this case it would pick one frame in every 700 frames, then trim the result to 100 frames. I do this to make sure the comparison uses various kind of frames throughout the movie. I usually adjust the first line so that the clip length is first something like 105-110 frames, then uncomment the Trim line.
Then I use ffmpeg to encode the lossless clip (you can use FFV1 as the codec for example), like vspipe.exe --y4m "yourscript.vpy" - | c:\utils\ffmpeg\bin\ffmpeg.exe -i - -c:v ffv1 "c:\zopti\yourlosslessclip.avi"

The idea of the Zopti script is to simulate encoding the video in a lower resolution and then your TV or media player will upsize it to 4K when you watch it. We are trying to find optimal parameters in Bicubic to make sure the distortion in the final result is as small as possible. The other metric to try is gmsd, it's faster than mdsi. Usually ends up in a slightly less sharp final result.

This is the Zopti command line I use:
zopti test.vpy -alg mutation -iters dyn -dyniters 15 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script

Then from the log file, get the best result for b and c (filter_param_a and filter_param_b in Vapoursynth Bicubic). Place them in your script that you use to encode.
zopti -mode evaluate -log "nameofthezoptilogfilehere.log"

It may look a bit complicated but once you do it a few times, you get the hang of it. I'm quite happy with the results, much better than what I could have done manually.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 15th October 2019, 19:40   #23  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Boulder Thank you very much for explaining! Just a few questions.

1) If I'm encoding HDR, should I use:

Quote:
zopti = Zopti(r'results.txt', metrics=['mdsi', 'time'], matrix='bt2020nc')
or matrix='709' but first tonemap HDR to SDR video using like tonemap.Hable ?

2) If i don't use denoising (because i don't know how), can i skip encode the lossless clip and just add to the beginning of the script?

Quote:
clp = core.std.SelectEvery(clp, 700, 1)
clp = core.std.Trim(clp, 2, 101)
After a few time, it's my script. Is it right? I don't have good VapourSynth skills, i hope it's right.
Quote:
import vapoursynth as vs
from zoptilib import Zopti
core = vs.get_core()
core.max_cache_size = 8192

# Open video
clip = core.lsmas.LWLibavSource(source="video.mkv", format="YUV420P10", cache=1)

# SelectEvery and Trim
clip = core.std.SelectEvery(clip, 1500, 1)
clip = core.std.Trim(clip, 6, 105)

# Crop the video
clip = core.std.CropRel(clip=clip, left=0, right=0, top=280, bottom=280)

# color adjustment using ToneMap
clip = core.resize.Bilinear(clip=clip, format=vs.RGBS, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=125)
clip = core.tonemap.Hable(clip=clip, exposure=2.000)
clip = core.resize.Bilinear(clip=clip, format=vs.YUV420P8, matrix_s="709", primaries_s="709",range_s="limited", transfer_s="709", transfer_in_s="linear", range_in_s="limited", dither_type="ordered")

# Zopti Starts
zopti = Zopti(r'results.txt', metrics=['mdsi', 'time'], matrix='709')

b = -110/100.0 # optimize b = _n_/100.0 | -175..-50 | b
c = -10/100.0 # optimize c = _n_/100.0 | -30..70 | c

# This one downsizes the lossless clip to FHD
alternate = core.resize.Bicubic(clip, width=1920, height=800, filter_param_a=b, filter_param_b=c)

# This one upsizes the downsized clip to the UHD
alternate = core.resize.Bicubic(alternate, width=3840, height=1600, filter_param_a=0, filter_param_b=0.5)

# My 4k cropped video (I removed upscaling because my original video already 4k)
orig = clip

# We compare the original to the downsized-upsized one to get as close as possible
zopti.run(orig, alternate)

Last edited by redbtn; 15th October 2019 at 20:35.
redbtn is offline   Reply With Quote
Old 15th October 2019, 20:37   #24  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
If you are going the HDR way, you can use bt2020nc as the matrix and skip the tone mapping part. You can also add those two lines selecting the frames there, but it will probably be slower than using a lossless clip with the frames already picked.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 15th October 2019, 20:43   #25  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
I got an error in VS Editor using matrix='bt2020nc'
Quote:
Failed to evaluate the script:
Python exception: Resize error: bad value: matrix_in_s
Anyway, thank you very much!

Last edited by redbtn; 15th October 2019 at 20:46.
redbtn is offline   Reply With Quote
Old 15th October 2019, 20:46   #26  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
http://www.vapoursynth.com/doc/functions/resize.html

It's probably 2020ncl.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 15th October 2019, 20:51   #27  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by Boulder View Post
Yes, you are right! After finish the current job I'll try with 2020ncl without tone mapping and compare results. Thx!!
I hope in the end I'll get better result vs spline64
redbtn is offline   Reply With Quote
Old 15th October 2019, 21:41   #28  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,088
Quote:
Originally Posted by redbtn View Post
Yes, you are right! After finish the current job I'll try with 2020ncl without tone mapping and compare results. Thx!!
I hope in the end I'll get better result vs spline64
Unless you're scaling in the linear light domain, you probably aren't going to get great results. I don't know of any way with most open source pipelines to get linear light transforms. I generally wind up doing it with the Adobe Creative Cloud products.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th October 2019, 21:46   #29  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
It seems weird, sometimes it writes (no samples)

UPD: Oh, it seems like i found the reason. I use script below and it shows 100 frames, but the duration is still 1 hour 44mins (FPS: 16/1001 = 0.01598), like whole movie. I try to do losless video, and i get *.avi with 1 hour 44mins duration. It's very weird. I don't know what I'm doing wrong.

UPD 2: Problem is SelectEvery function. Without it fps is ok (FPS: 24000/1001 = 23.976). I don't know why it's happening.
UPD 3: Solution is very simple, Just set AssumeFPS after SelectEvery. Now it's ok. No, it's not. Fps is fine but it still writes "no samples". Maybe this is not errors. The script works and gives the result.
Quote:
import vapoursynth as vs
core = vs.get_core()

clip = core.lsmas.LWLibavSource(source="video.mkv", format="YUV420P10", cache=1)
clip = core.std.AssumeFPS(clip, fpsnum=24000, fpsden=1001)
clip = core.std.CropRel(clip=clip, left=0, right=0, top=280, bottom=280)

# SelectEvery and Trim
clip = core.std.SelectEvery(clip, 1500, 1)
clip = core.std.Trim(clip, 6, 105)

clip.set_output()
Attached Images
 

Last edited by redbtn; 16th October 2019 at 01:21.
redbtn is offline   Reply With Quote
Old 16th October 2019, 04:42   #30  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
Quote:
Originally Posted by benwaggoner View Post
Unless you're scaling in the linear light domain, you probably aren't going to get great results. I don't know of any way with most open source pipelines to get linear light transforms. I generally wind up doing it with the Adobe Creative Cloud products.
I use resamplehq myself, that one should work fine.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th October 2019, 04:47   #31  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
Quote:
Originally Posted by redbtn View Post
It seems weird, sometimes it writes (no samples)
Don't worry about any of those things, it's working fine

And when you get the results, don't forget to divide them by 100 to get the value to feed in the resizer. As benwaggoner mentioned, you probably need something like resamplehq with HDR sources (in Zopti and final encode).
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 16th October 2019 at 05:57.
Boulder is offline   Reply With Quote
Old 16th October 2019, 18:46   #32  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by Boulder View Post
Don't worry about any of those things, it's working fine

And when you get the results, don't forget to divide them by 100 to get the value to feed in the resizer. As benwaggoner mentioned, you probably need something like resamplehq with HDR sources (in Zopti and final encode).
Thank you! I found the optimal values.
And I have a question, if I wanna spend like 18mb/s of bitrate to all my encodes (If they can use less bitrate for a quality that suits me, I don't care. And if they need more, I don't want to spend more), what the best way, adjust CRF to target bitrate or use ABR?
I'm asking because I did test and noticed that CRF encode in slow scenes much worse than ABR. I didn't compare high motions scenes yet, but I'm a little confused.
redbtn is offline   Reply With Quote
Old 16th October 2019, 19:05   #33  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 221
Quote:
Originally Posted by Tadanobu View Post
PHP Code:
--limit-refs 1 --tu-intra-depth 3 --tu-inter-depth 3 --limit-tu 4 
how these switches work?
Natty is offline   Reply With Quote
Old 16th October 2019, 19:36   #34  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
Quote:
Originally Posted by redbtn View Post
Thank you! I found the optimal values.
And I have a question, if I wanna spend like 18mb/s of bitrate to all my encodes (If they can use less bitrate for a quality that suits me, I don't care. And if they need more, I don't want to spend more), what the best way, adjust CRF to target bitrate or use ABR?
I'm asking because I did test and noticed that CRF encode in slow scenes much worse than ABR. I didn't compare high motions scenes yet, but I'm a little confused.
I always use CRF. I've figured out a level which will generally produce transparent results when watching from about 2 metres, then reduced the figure a bit more to gain some headroom. This is simply because the material varies so much. Some requires very few bits, but some black and white movies with a lot of grain will eat bits for breakfast. CRF 18 is the one I use for my 720p encodes, 1080p could probably handle CRF 19 but 18 is on the safe side.

If I'm not entirely mistaken, a 2-pass encode at 18 Mbps and CRF resulting in the same average bitrate should produce equal quality. The rate control mechanism is most likely the same, at least that's how it is with x264.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th October 2019, 19:51   #35  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by Boulder View Post


If I'm not entirely mistaken, a 2-pass encode at 18 Mbps and CRF resulting in the same average bitrate should produce equal quality. The rate control mechanism is most likely the same, at least that's how it is with x264.
2-pass is taking x2 time for encode. I'm comparing CRF vs 1-pass ABR at the same bitrate. I wonder in what areas or scenes CRF should be better to find the differences.
Do you use --cutree?

Last edited by redbtn; 16th October 2019 at 20:19.
redbtn is offline   Reply With Quote
Old 16th October 2019, 20:20   #36  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by Natty View Post
how these switches work?
This is a very General question. It is unclear what answer you want to hear.
redbtn is offline   Reply With Quote
Old 16th October 2019, 20:52   #37  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,682
Quote:
Originally Posted by redbtn View Post
2-pass is taking x2 time for encode. I'm comparing CRF vs 1-pass ABR at the same bitrate. I wonder in what areas or scenes CRF should be better to find the differences.
Do you use --cutree?
Yes, cutree is enabled by default. ABR is not suitable for your regular encodes, I'd say it's more for streaming or just some quick tests. Better find the CRF level you are happy with..
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 18th October 2019, 17:23   #38  |  Link
Nico8583
Registered User
 
Join Date: Jan 2010
Location: France
Posts: 752
I think I'll go with these settings :
Quote:
HD :
Preset : Slow
CRF : 14 to 16
Sao : 0
Deblock : -1;-1
Colorprim : bt709
Transfer : bt709
Colormatrix : bt709

UHD :
Preset : Slow
CRF : 14 to 16
Sao : 0
Deblock : -1;-1
Colorprim : bt2020
Transfer : smtpe2084
Colormatrix : bt2020nc
HDR : 1
HDR-Opt : 1
Chromaloc : 2 (variable)
Repeat-Headers : 1
AUD : 1
HDR : 1
Range : Limited
Master Display : G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,20) (variable)
Max CLL : 717,578 (variable)

UHD HDR10+ :
Same than UHD +
DHDR10-Info : 1
DHDR10-Opt : <JSON>
My first tests seems to be OK, few parameters but sufficient, no ?

Last edited by Nico8583; 19th October 2019 at 10:45.
Nico8583 is offline   Reply With Quote
Old 18th October 2019, 19:00   #39  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
I'm not sure about Dolby Vision, but overall it looks ok. Just add min-luma 0 and max-luma 1023 to HDR settings.
redbtn is offline   Reply With Quote
Old 18th October 2019, 19:34   #40  |  Link
Nico8583
Registered User
 
Join Date: Jan 2010
Location: France
Posts: 752
I don't know these parameters, hdr-opt and/or colormatrix and/or range don't already define these values ?
And I'm doubting about SAO, with a high bitrate I allways disable it but since last x265 versions perhaps it's not a good idea ?
Nico8583 is offline   Reply With Quote
Reply

Tags
parameters, presets, settings, x265

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.