Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st October 2018, 19:57   #1  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Benwaggoner HEVC encoding challenge

With all the talk about VP9 and AV1 and VVC and different encoder implementations for all of them, I thought it might be fun to set up an open challenge for folks to deliver the best possible quality for each.

To that end, in a semi-inebriated late-night conversation at IBC, we defined a scenario relevant to some important real-world scenarios that reasonably stress current encoders.
  • Tears of Steel
  • 1.0, 1.5, and 2.0 Mbps ABR
  • 4 Mbps peak bitrate
  • 12 Mbps VBV
  • Max 5 sec (120 frame) GOP
  • No preprocessing

Here's the source I used (.y4m.7z): https://1drv.ms/u/s!AlvIQZWsyeO-kKpoG5f3PHAFhWcGig

And my current best HEVC x265 encodes:
1.0: https://1drv.ms/v/s!AlvIQZWsyeO-kKpmDQlxoY-00rk3AQ
1.5: https://1drv.ms/v/s!AlvIQZWsyeO-kKplp2EQ8-Q4bCNVZw
2.0: https://1drv.ms/v/s!AlvIQZWsyeO-kKpZkpw-WQKF1dSF_g

I didn't do any content specific encoding in these. I just did the slowest, highest quality encode 2-pass I had the patience for. Basically
  • --preset placebo
  • --cu-lossless
  • --tskip
  • -F 1
  • --ref 6
  • --bframes 16
  • --aq-mode 3
  • --rd-refine

If I had infinite patience I'd add --me sea and --subme 7 for PlusUltraPlacebo.

Update 6/3/2021
To evaluate anime/line art encoding I'm adding Netflix's Sol Levante as a second source for testing. It's an interesting compression challenge with a lot of titles/credits that can be shrunk down enormously, and some very complex hybrid line art/CGI sequences that x265 can't make look great at even 12 Mbps. It should be very interesting for rate control!

Sol Levante's 4K HDR source is available under Creative Commons, from which I derived an 8-bit SDR 1080p .y4m, as 8-bit SDR playback and testing is still a lot more available.

I've kicked off a 1000 Kbps test encode with x265 for reference that I should post tomorrow. The challenge follows the same constraints for bitrate, VBV, etcetera.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book

Last edited by benwaggoner; 3rd June 2021 at 22:28. Reason: Added Sol Levante as second source to test
benwaggoner is offline   Reply With Quote
Old 2nd October 2018, 07:43   #2  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 556
Thanks. But why not set this? --profile main10
10-bits might help a little with banding maybe.
Is there any reason not to set --profile main10 in x265? Most of latest years GPU:s supports 10-bit H.265-decoding partially or fully.

EDIT: OK, thanks Blue_MiSfit, although watching video at this high resolution on an Android phone (small screen) is just wasteful.

Last edited by Forteen88; 2nd October 2018 at 09:58. Reason: thanking for answer
Forteen88 is offline   Reply With Quote
Old 2nd October 2018, 09:09   #3  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Older devices with early HEVC decoders?

I know some FireTV devices and Android phones are 8 bit only
Blue_MiSfit is offline   Reply With Quote
Old 2nd October 2018, 10:50   #4  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
I like the chalenge, but the source is something i don't want to watch, any other possible candidates?
__________________
certain other member
smok3 is offline   Reply With Quote
Old 2nd October 2018, 16:37   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by smok3 View Post
I like the chalenge, but the source is something i don't want to watch, any other possible candidates?
Sure. Anything that's publicly available and has an appropriate license.

I like Tears of Steel as something that's reasonably short that has a variety of appropriate characteristics. Live action, CGI, film grain, motion graphics in credits.

Any test is going to be somewhat arbitrary and focused on a particular scenario. The one here is very much about encoder features at least as much as bitstreams. It stresses psychovisual tuning and rate control, and ignores encoding performance.

An obvious followup test would be 1-pass CBR with fixed 2 second Closed GOP.

Something incorporating quality @ perf would be interesting, but it's way harder to compare and verify performance.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd October 2018, 16:43   #6  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Forteen88 View Post
Thanks. But why not set this? --profile main10
10-bits might help a little with banding maybe.
Is there any reason not to set --profile main10 in x265? Most of latest years GPU:s supports 10-bit H.265-decoding partially or fully.
One big reason for me is that comparing 10-bit encodes is a lot harder, since different setups can use very different conversion methods. Something that gets a nice dither on one platform might get banding due to truncation on another. Sticking to 8-bit reduces

Also, I wanted to be able to compare 8-bit only codecs.

I'd love to do a HDR-10 test as a followup. I've not found good HDR test content with an appropriate license yet, though.

Quote:
EDIT: OK, thanks Blue_MiSfit, although watching video at this high resolution on an Android phone (small screen) is just wasteful.
Yeah, this is really meant for bigger screens. A 250-500 Kbps SD version would an interesting followup test. Among other things, it would make getting an AV1 encode a lot more feasible.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 8th October 2018, 20:30   #7  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by benwaggoner View Post
I'd love to do a HDR-10 test as a followup. I've not found good HDR test content with an appropriate license yet, though.
You can get the source files (which include 32 bit exr files) from the Tears of Steel project.
It's all open source...

https://media.xiph.org/tearsofsteel/
K.i.N.G is offline   Reply With Quote
Old 8th October 2018, 20:58   #8  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 940
Quote:
Originally Posted by K.i.N.G View Post
You can get the source files (which include 32 bit exr files) from the Tears of Steel project.
It's all open source...

https://media.xiph.org/tearsofsteel/
AFAIK, TOS hasn't been upgraded for HDR.
It's all SDR.

You can find custom edits on youtube where users graded it themselves.
__________________
Win10 (x64) build 19041
NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4)
NTSC | DVD: R1 | BD: A
AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
Sparktank is offline   Reply With Quote
Old 8th October 2018, 21:02   #9  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by Sparktank View Post
AFAIK, TOS hasn't been upgraded for HDR.
It's all SDR.

Thats why I mention the exr files and not the actual exported movie...
The source files are exr which are 32bit (or 16bit at worst, but standard exr is 32bit) thus you can safely grade/master them to 10 HDR, even 12bit if you want. There's plenty of dynamic range/headroom.

I'm not sure in which format the actual filmed/video parts are available though... I'm quite busy so i haven't checked.
I suppose they are in raw format with sufficient dynamic range aswell... but that would be a guess

Last edited by K.i.N.G; 8th October 2018 at 21:05.
K.i.N.G is offline   Reply With Quote
Old 10th October 2018, 04:13   #10  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by K.i.N.G View Post
Thats why I mention the exr files and not the actual exported movie...
The source files are exr which are 32bit (or 16bit at worst, but standard exr is 32bit) thus you can safely grade/master them to 10 HDR, even 12bit if you want. There's plenty of dynamic range/headroom.
I looked into this a few years back. While we have lots of precision in the source, it was all graded and rendered for 709 SDR. One could probably use those assets to remaster it to HDR, but that would require regrading, rerendering the CGI stuff, and recompositing. Feasible, certainly, and a lot less work than doing a project from scratch. But still a sizable effort to make it look like "Real" HDR, and that's even assuming they did the RAW acquisition preserving the highlights and such.

I've been involved in a number of HDR remastering projects, and doing it well is quite a lot of effort. Particularly for something as CGI-centric as Tears of Steel.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th October 2018, 10:12   #11  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 90
Quote:
Originally Posted by benwaggoner View Post
I looked into this a few years back. While we have lots of precision in the source, it was all graded and rendered for 709 SDR. One could probably use those assets to remaster it to HDR, but that would require regrading, rerendering the CGI stuff, and recompositing. Feasible, certainly, and a lot less work than doing a project from scratch. But still a sizable effort to make it look like "Real" HDR, and that's even assuming they did the RAW acquisition preserving the highlights and such.


The 32bit exr's that are available seem to have the 709 grading baked in indeed (which is weird) but its not that big of a deal since they kept the dynamic range and full exposure info in there (they aren't clipped). So, no problems there.
Those exr's seem to be saved in 'only' 1080p though...

Quote:
Originally Posted by benwaggoner View Post
I've been involved in a number of HDR remastering projects, and doing it well is quite a lot of effort. Particularly for something as CGI-centric as Tears of Steel.
CGI data is pretty much ideal. It doesn't get any better than that.
Of course you always have those people who render everything clipped and bake weird color corrections right into the renders, that's another story. But that also happens with video footage, so no difference in that regard.
CGI with the rec.709 profile baked in is still miles easier to work with than video footage with the camera's color response + some custom curve baked in though.

Last edited by K.i.N.G; 15th October 2018 at 10:18.
K.i.N.G is offline   Reply With Quote
Old 15th October 2018, 19:24   #12  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
[QUOTE=K.i.N.G;1854914]The 32bit exr's that are available seem to have the 709 grading baked in indeed (which is weird) but its not that big of a deal since they kept the dynamic range and full exposure info in there (they aren't clipped). So, no problems there.
Those exr's seem to be saved in 'only' 1080p though...

I am sure there are UHD ones as well. I think I downloaded them once some years ago. Maybe off an FTP?

My .y4m was generated from the 16-bit PNG sequence.

Quote:
CGI data is pretty much ideal. It doesn't get any better than that.
The final has film grain added to the CGI, but if the EXR is clean plates, awesome!

Quote:
Of course you always have those people who render everything clipped and bake weird color corrections right into the renders, that's another story. But that also happens with video footage, so no difference in that regard.
CGI with the rec.709 profile baked in is still miles easier to work with than video footage with the camera's color response + some custom curve baked in though.
My understanding is they didn't do anything specific to prevent >>709 things from happening. But they did all the work in 709, so wouldn't have known or worried about if anything in particular was clipped or whatever.

It would be awesome if someone put in the time to do a HDR master! From what you've found so far, it might not even be that big a project. Maybe a day or two for someone with the right skills, software, and hardware. Two days to remastered an hour show to HDR with decent sources is pretty luxurious, and this is only 11 minutes.

But anyway, I'd love to see anyone else's best encodes with the SDR clip, in x265, different HEVC, or even other codecs. In particular, I'd love to see a VP9 expert's best VP9 effort.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 24th October 2018, 08:06   #13  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 56
Hi Ben, great challenge that you have arranged!

Here are encodes with the xvc codec at three different rates:

There is no strict rate control in xvc yet so the margins to the target rates are quite large.

851 kbps: https://drive.google.com/file/d/1xrM...ew?usp=sharing (15% under target rate)
1361 kbps: https://drive.google.com/file/d/1cRs...ew?usp=sharing (9% under target rate)
1924 kbps: https://drive.google.com/file/d/1K46...ew?usp=sharing (4% under target rate)

To play the files you need a player with xvc support such as this build of ffplay: https://drive.google.com/file/d/1gyj...ew?usp=sharing

The zip file also contains the build of ffmpeg that was used to encode the files using the following command:
ffmpeg.exe -i d:/ToS_1920x800_xdither.y4m -c:v libxvc -speed-mode 2 -qp X -max-keypic-distance 120 -threads 16 enc.mp4
with X being 28, 25 and 23, respectively.

It can be noted that these encodes are single pass, with no lookahead (beyond the 16 picture sub-GOP structure), and that reference pictures are stored with 10-bit precision internally (there is no 8 bit profile of xvc).

I would be happy to hear what people think of it! Thanks!
__________________
Jonatan Samuelsson
Co-founder and CEO at Divideon

www.divideon.com | xvc.io
jonatans is offline   Reply With Quote
Old 24th October 2018, 08:34   #14  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
jonatans; Tryed with the latest build of mpv (Debian) and I guess that is not yet supported out of the box?

Code:
mpv xvc_tos_851kbps.mp4 
Playing: xvc_tos_851kbps.mp4
 (+) Video --vid=1 (*) ( 1920x800 24.000fps)
Failed to initialize a decoder for codec ''.
Video: no video
No video or audio streams selected.


Exiting... (Errors when loading file)

mpv --version
mpv 0.29.0-75-gda1073c247 Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on Wed Oct 24 09:29:52 CEST 2018
ffmpeg library versions:
   libavutil       56.19.101
   libavcodec      58.33.102
   libavformat     58.19.102
   libswscale      5.2.100
   libavfilter     7.37.100
   libswresample   3.2.100
ffmpeg version: N-92246-gc2ac3b8e6
__________________
certain other member
smok3 is offline   Reply With Quote
Old 24th October 2018, 11:18   #15  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 56
Quote:
Originally Posted by smok3 View Post
jonatans; Tryed with the latest build of mpv (Debian) and I guess that is not yet supported out of the box?
Correct.

In order to play the xvc files you need to use a player with xvc support such as the version of ffplay provided in the zip file I uploaded (https://drive.google.com/file/d/1gyj...ew?usp=sharing)
__________________
Jonatan Samuelsson
Co-founder and CEO at Divideon

www.divideon.com | xvc.io
jonatans is offline   Reply With Quote
Old 25th October 2018, 21:52   #16  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
WOW.

That is quite impressive at 851 Kbps. Like, dramatically better than HEVC.
Blue_MiSfit is offline   Reply With Quote
Old 26th October 2018, 07:50   #17  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 56
Quote:
Originally Posted by Blue_MiSfit View Post
WOW.

That is quite impressive at 851 Kbps. Like, dramatically better than HEVC.
Thanks!
__________________
Jonatan Samuelsson
Co-founder and CEO at Divideon

www.divideon.com | xvc.io
jonatans is offline   Reply With Quote
Old 4th November 2018, 19:51   #18  |  Link
ShortKatz
Registered User
 
Join Date: Aug 2018
Location: Germany
Posts: 118
Quote:
Originally Posted by jonatans View Post
The zip file also contains the build of ffmpeg that was used to encode the files using the following command:
ffmpeg.exe -i d:/ToS_1920x800_xdither.y4m -c:v libxvc -speed-mode 2 -qp X -max-keypic-distance 120 -threads 16 enc.mp4
Hi jonatans, how can I build a version of ffmpeg that supports xvc? I would need it for Mac and would like to include this ffmpeg in my version of HandBrake than. Thanks.
ShortKatz is offline   Reply With Quote
Old 5th November 2018, 10:14   #19  |  Link
abaxas
Registered User
 
Join Date: May 2018
Posts: 9
Increase me-range (or whatever it's called in x265) for a ultra small increase in quality for a massive increase in encoding time.

Not sure if it applies to x265 but there is an ultra small quality bump for single thread encoding. Noting 4 encodes running on 1 core each will complete much faster than 4x single encodes running on all 4 cores.
abaxas is offline   Reply With Quote
Old 7th November 2018, 00:41   #20  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,531
Quote:
Originally Posted by Blue_MiSfit
WOW.
That is quite impressive at 851 Kbps. Like, dramatically better than HEVC.
Second that. That 851kbit/s file blew my socks off. 0.024bit/pixel and perfectly watchable.
Single Hair visible and stays through motion, a little mosquito around titles
the girl's moving robotic arm pulls only minimum morphing artefacts through the leaves.
Giving bitrate of 1921 even that arm's morphing wake is almost gone, titles mosquito improve,
not missing any detail from a good blu-ray encode...
How was encoding time if I may ask ?

BTW, back in 2009 DS had test-encoded Big Buck Bunny with x264 Build 999+1 at around 0.03bpp,
yielding good results, but BBB is not as challenging as TOS motionwise, good parts of the picture stay static in BBB..

https://forum.doom9.org/showthread.p...ow#post1247874
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."

Last edited by Emulgator; 7th November 2018 at 01:20.
Emulgator is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:31.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.