Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th December 2019, 09:10   #2021  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,706
I didn't mean to imply that encoding 8 bit content in 10 bit AV1 would be the right thing to do.

I just meant that given premium content almost always being > = 10 bit 4:2:2 it makes sense to preserve 10 bit, especially when picky studios are highly critical of how subtle gradients are handled in their content.

Last edited by Blue_MiSfit; 5th December 2019 at 09:14.
Blue_MiSfit is offline   Reply With Quote
Old 5th December 2019, 12:58   #2022  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 154
Quote:
Originally Posted by Blue_MiSfit View Post
I also hope that the dav1d team focuses more on 10 bit soon. I think social media / user generated content is a huge use case for AV1, and most of that content is 8 bit for now.
It's a shame Google pursued gAV1 rather than shunting engineer time to dav1d, they could work much faster on all fronts with some serious money behind them.

I get the whole competition is good angle, but it doesn't seem to have made much of a difference, other than prompting dav1d to shore up their ARM32 priorities - from which it seems dav1d are soundly ahead on all fronts again.
soresu is offline   Reply With Quote
Old 5th December 2019, 13:00   #2023  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 154
Quote:
Originally Posted by utack View Post
Beating a dead horse at this point maybe, but dav1d having a milestone for better PPC support and no word about adding basic 10bit support seems extremely odd
Netflix has signalled they are only interested in 10bit content, Youtube started encoding 10bit for their new higher resolution videos as well.
It should clearly be a priority over armv7 and PPC assembly, and imho all the "make it fast" milestones are not reached yet.
They have basic 10 bit support I think, just very little SIMD asm to accelerate it beyond C code.
soresu is offline   Reply With Quote
Old 6th December 2019, 02:03   #2024  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,706
Regarding Google working on their own AV1 decoder, I'd imagine this has to do with being the masters of their own destiny so they can use the decoder however they want on Android / anywhere else, free from any potential licensing incompatibility with dav1d.
Blue_MiSfit is offline   Reply With Quote
Old 6th December 2019, 09:23   #2025  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,953
Quote:
Originally Posted by Blue_MiSfit View Post
Regarding Google working on their own AV1 decoder, I'd imagine this has to do with being the masters of their own destiny so they can use the decoder however they want on Android / anywhere else, free from any potential licensing incompatibility with dav1d.
dav1d has one of the most liberal licenses anywhere (2-clause BSD), Android already uses libraries under far stricter licenses.

Nevermind that Chrome already uses dav1d as well, so its not like Google somehow doesn't know about it, or something like that.

The reason is probably some BS politics, and has no logical backing.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 7th December 2019, 02:31   #2026  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,706
Hmm. That's unfortunate, I wonder if we'll ever know the real story here
Blue_MiSfit is offline   Reply With Quote
Old 9th December 2019, 23:23   #2027  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,276
Quote:
Originally Posted by Blue_MiSfit View Post
I didn't mean to imply that encoding 8 bit content in 10 bit AV1 would be the right thing to do.

I just meant that given premium content almost always being > = 10 bit 4:2:2 it makes sense to preserve 10 bit, especially when picky studios are highly critical of how subtle gradients are handled in their content.
Premium content is NOT >=10-bit 4:2:2. 10-bit 4:2:0 is all that anyone is distributing to consumers via streaming and disc. The masters are, certainly, but it's not like those look visually better than distribution codecs at a sufficient bitrate.

Sent from my SM-T837V using Tapatalk
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th December 2019, 23:25   #2028  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,276
Quote:
Originally Posted by huhn View Post
isn't 10 bit HEVC still generally better then 8 bit but the difference is very very small?



and the whole story of 8bpp and 16bpp x265. wasn't that an very important part why x264 10 bit was much better thanks to 10 bit bpp instead of 8.

is the 8 bpp x265 even alive?



how does AV1 handle this is this even comparable?



isn't a 8 bit quality pipe absolutely impossible anyway? just the level/RGB conversation pretty much proves it or even the chroma upscale.
There is a significant but shrinking number of devices, mainly handheld, that support 8-bit HEVC but not 10-bit.

More common is devices with 10-bit decoders that then feed into 8-bit compositors, GPUs, or display controllers.

Sent from my SM-T837V using Tapatalk
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th December 2019, 00:52   #2029  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,507
Quote:
Originally Posted by benwaggoner View Post
Do we have any evidence that 8-bit sources encode better in 10-bit than 8-bit in AV1?
If nothing else, 8b RGB to 10b YUV to 8b RGB is substantially higher quality than 8b YUV to 8b RGB (especially near white and black), and 10b YUV can decode to higher depth RGB wherever the panel supports it. It sure beats having to deband afterward, or drop in a shedload of noise (and bits) to hide banding. Those with 6b RGB panels will continue to wail, because hardware companies refuse to incorporate simple tricks like dithering on budget hardware until they're promoted as a requirement, but that's can't be helped.

It's true that AV1 and HEVC fixed AVC's rather staggering difference in quality between 8- and 10-bit encoding, but there's still some utility if the toolchain wants to keep quality at the forefront.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 10th December 2019, 03:07   #2030  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,706
Quote:
Originally Posted by benwaggoner View Post
Premium content is NOT >=10-bit 4:2:2. 10-bit 4:2:0 is all that anyone is distributing to consumers via streaming and disc.
Of course. That's what I said (specifically referencing masters being >= 10 bit)

Last edited by Blue_MiSfit; 10th December 2019 at 03:09.
Blue_MiSfit is offline   Reply With Quote
Old 10th December 2019, 20:29   #2031  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,276
Quote:
Originally Posted by Blue_MiSfit View Post
Of course. That's what I said (specifically referencing masters being >= 10 bit)
Ah, my apologies for the confusion, then.

Sent from my SM-T837V using Tapatalk
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th December 2019, 22:44   #2032  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 6,447
Quote:
Originally Posted by benwaggoner View Post
There is a significant but shrinking number of devices, mainly handheld, that support 8-bit HEVC but not 10-bit.

More common is devices with 10-bit decoders that then feed into 8-bit compositors, GPUs, or display controllers.
i don't see anything that diminished the benefit of using more bit deep in encoding here.
Quote:
Originally Posted by foxyshadis View Post
If nothing else, 8b RGB to 10b YUV to 8b RGB is substantially higher quality than 8b YUV to 8b RGB (especially near white and black), and 10b YUV can decode to higher depth RGB wherever the panel supports it. It sure beats having to deband afterward, or drop in a shedload of noise (and bits) to hide banding. Those with 6b RGB panels will continue to wail, because hardware companies refuse to incorporate simple tricks like dithering on budget hardware until they're promoted as a requirement, but that's can't be helped.
the fact that AMD supports native 6 bit output with there own dithering which helps a lot of sub optimal screens show the benefit of doing this correctly.
the best panel i have ever tested in term of banding is a 6 bit FRC panel the processing just doesn't add banding on such a "bad" device.

and i totally agree using more bits in encoding to delay the dithering until the end device or at least the presentation of a device is a clear benefit.

just reading about 4:2:2 master is letting me wonder if this is just a bad habit that simple didn't stop instead of 16 RGB or higher even the aged BBB has this as a master and what reason could be there not to do that.
huhn is offline   Reply With Quote
Old 11th December 2019, 01:15   #2033  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,706
I always figured the 10 bit 4:2:2 thing most likely goes back to SDI / HD-SDI which was traditionally 10 bit 4:2:2. Capturing the full bandwidth of this signal into a file based format in a way that's perceptually lossless was long since considered "good enough" and widely used mezzanine file formats like ProRes do a great job of this - especially relative to the old days of 50-80 Mbps 1080p MPEG-2 service masters.

Given the context of final distribution always being 8 bit 4:2:0 until recently, perceptually lossless 10 bit 4:2:2 was indeed always good enough.

Now that we're doing HDR, 10 bit is mandatory, and 12 or even 16 bit (ProRes 4444 / 4444 XQ or JPEG 2000) is common for source material when encoding for Dolby Vision. Sure, this gets shaped into a 10 bit IPT file but allegedly more of the input can be recovered after Dolby magic.

Studio archival masters are generally 16 bit full range RGB TIFFs or the OpenEXR equivalent (not sure about the specifics of that), and probably DPX for older stuff. Maybe 16 bit lossless JPEG 2000 if the studio is IMF native.

Last edited by Blue_MiSfit; 13th December 2019 at 04:21.
Blue_MiSfit is offline   Reply With Quote
Old 15th December 2019, 16:29   #2034  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 962
New rav1e build released with 20% speed improvement: https://github.com/xiph/rav1e/releases/tag/p20191215
hajj_3 is online now   Reply With Quote
Old 19th December 2019, 10:25   #2035  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 962
Rav1e 0.2.0 released, 40-70% faster than than v0.1.0 depending on the encoding settings.

changelog:
Optional serialization/deserialization of the encoding parameters through the feature serialize
Optional cli advanced commands to use it.
The builds are now using the dwarf debug format for the targets that support it, before it was a mixture of dwarf and stabs due to the nasm defaults.
Added a --benchmark hidden flag for the cli for MacOS and Linux.
documentation is now available on docs.rs.
Changes
Segmentation support is now a tunable SpeedSetting and currently it is default off since it can produce desyncs, this does cause a 3% decrease in quality.
Fixes
#1903 - edge-of-frame miscomputation
#1858 - desync on speed 0 and 1 when certain quantizers are selected
Known issues
#1930 - segmentation encoding may cause desync
hajj_3 is online now   Reply With Quote
Old 21st December 2019, 02:01   #2036  |  Link
Mr_Khyron
Member
 
Mr_Khyron's Avatar
 
Join Date: Nov 2002
Posts: 140
SVT-AV1 0.8 is out
https://github.com/OpenVisualCloud/S...ses/tag/v0.8.0
Quote:
x Preset Optimizations
x Single-core execution memory optimization [-lp 1 -lad 0]
x Rate estimation update enhancements
x On / off flags for feature run-time switching
x Auto-max partitioning algorithm support
x Multi-pass partitioning depth support
x Remove deprecated RC mode 1 and shifter RC mode 2 and mode 3 to mode 1 and mode 2 respectively
x Update cost calculation for CDEF Filtering
x Intra-Inter compound for 10-bit
x Eigth-pel optimization
x AVX512 Optimizations
x AVX2 Optimizations
Mr_Khyron is offline   Reply With Quote
Old 24th December 2019, 18:25   #2037  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 103
Status report!
"Padoru padoru!" edition

1st edition: https://forum.doom9.org/showthread.p...49#post1852449
2nd edition: https://forum.doom9.org/showthread.p...87#post1857587
3rd edition: https://forum.doom9.org/showthread.p...75#post1860475
4th edition: https://forum.doom9.org/showthread.p...39#post1871939
5th edition: https://www.reddit.com/r/AV1/comment..._half_edition/
Whatever paragraph I don't repeat here can be assumed to be the same as in the aforementioned post

First of all: graphs!
Click to enlarge

Y axis: chosen metric
X axis: bits per pixel

720p:


1080p:


BD rates for 720p:
Code:
Codecs ladder:              | x264 relative:
x264 -> rav1e               | x264 -> rav1e
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -23.2215 1.01158    |  MSSSIM -23.2215 1.01158
PSNRHVS -24.4172 1.36951    | PSNRHVS -24.4172 1.36951
  HVMAF -15.8889 1.38478    |   HVMAF -15.8889 1.38478
----------------------------|----------------------------
rav1e -> svtav1             | x264 -> svtav1
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -5.56998 0.20639    |  MSSSIM -28.4361 1.23111
PSNRHVS -6.41119 0.290801   | PSNRHVS -30.1248 1.65609
  HVMAF -15.6008 1.20241    |   HVMAF -29.5127 2.45437
----------------------------|----------------------------
svtav1 -> vp9               | x264 -> vp9
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM  4.20075 -0.151838  |  MSSSIM -25.2711 1.10719
PSNRHVS  4.84265 -0.215506  | PSNRHVS -26.5998 1.48648
  HVMAF -1.42341 -0.19208   |   HVMAF -29.8597 2.33372
----------------------------|----------------------------
vp9 -> x265                 | x264 -> x265
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -1.31186 0.0508852  |  MSSSIM -26.3349 1.18403
PSNRHVS -5.36837 0.2586     | PSNRHVS -30.5296 1.7606
  HVMAF -1.84033 0.341593   |   HVMAF -30.9904 2.59114
----------------------------|----------------------------
x265 -> av1                 | x264 -> av1
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -23.2948 0.961133   |  MSSSIM -43.3907 2.06939
PSNRHVS -18.5938 0.914589   | PSNRHVS -43.484 2.58526
  HVMAF -19.3808 1.25801    |   HVMAF -44.4219 3.59302
BD rates for 1080p:
Code:
Codecs ladder:              | x264 relative:
x264 -> rav1e               | x264 -> rav1e
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -32.2826 1.19743    |  MSSSIM -32.2826 1.19743
PSNRHVS -30.8004 1.43869    | PSNRHVS -30.8004 1.43869
  HVMAF -24.0161 1.68074    |   HVMAF -24.0161 1.68074
----------------------------|----------------------------
rav1e -> svtav1             | x264 -> svtav1
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -7.09315 0.212708   |  MSSSIM -37.5209 1.44372
PSNRHVS -6.91838 0.253513   | PSNRHVS -36.0074 1.70477
  HVMAF -14.4798 1.05647    |   HVMAF -34.513 2.60239
----------------------------|----------------------------
svtav1 -> vp9               | x264 -> vp9
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM  1.75731 -0.0570674 |  MSSSIM -36.6065 1.39502
PSNRHVS  0.61474 -0.0275689 | PSNRHVS -35.7951 1.68578
  HVMAF -5.87037  0.0512821 |   HVMAF -38.5344 2.64173
----------------------------|----------------------------
vp9 -> x265                 | x264 -> x265
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM 9.95155  -0.292103  |  MSSSIM -30.4864 1.15663
PSNRHVS 4.66281  -0.165867  | PSNRHVS -32.9136 1.56443
  HVMAF 5.65708  -0.054123  |   HVMAF -34.5119 2.53192
----------------------------|----------------------------
x265 -> av1                 | x264 -> av1
        RATE (%) DSNR (dB)  |         RATE (%) DSNR (dB)
 MSSSIM -31.9542 1.15077    |  MSSSIM -52.2509 2.19044
PSNRHVS -26.5316 1.11388    | PSNRHVS -50.3241 2.54395
  HVMAF -22.5904 1.34136    |   HVMAF -49.1759 3.58479
Encoders:
x264 158-2984-3759fcb
x265 3.2-7-37648fca915b
libvpx-vp9 1.8.2-23-g50d1a4aa7
rav1e 0.2.0-32-g350ac84
SVT-AV1 0.8.0-5303-0952998d
libaom 1.0.0-60-g3e421b069

Cmdlines:
Code:
echo -n "16 19 22 26 29 32" | xargs -d " " -n 1 -P 1 -I {} x264 --preset veryslow --tune ssim --crf {} -o test.x264.crf{}.264 orig.i420.y4m
echo -n "16 20 23 27 30 34" | xargs -d " " -n 1 -P 1 -I {} x265 --preset veryslow --tune ssim --crf {} -o test.x265.crf{}.hevc orig.i420.y4m
echo -n "12 21 30 38 47 56" | xargs -d " " -n 1 -P 6 -I {} vpxenc --codec=vp9 --frame-parallel=0 --tile-columns=0 --auto-alt-ref=6 --good --cpu-used=0 --tune=psnr --passes=2 --threads=1 --end-usage=q --cq-level={} --test-decode=fatal --ivf -o test.vp9.cq{}.ivf orig.i420.y4m
echo -n "030 056 082 108 134 160" | xargs -d " " -n1 -P6 -I {} rav1e -o test.rav1e.cq{}.ivf --quantizer {} -s 5 --tune Psychovisual orig.i420.y4m
echo -n "13 21 29 37 45 53" | xargs -d " " -n1 -P1 -I{} SvtAv1EncApp.exe -i orig.i420.yuv -b test.svtav1.cq{}.ivf -w 1280 -h 720 -q {} -scm 0 -enc-mode 8 -fps-num 24000 -fps-denom 1001 -intra-period 47 -output-stat-file fp_stats{}.stat -enc-mode-2p 3
echo -n "13 21 29 37 45 53" | xargs -d " " -n1 -P1 -I{} SvtAv1EncApp.exe -i orig.i420.yuv -b test.svtav1.cq{}.ivf -w 1280 -h 720 -q {} -scm 0 -enc-mode 3 -fps-num 24000 -fps-denom 1001 -intra-period 47 -input-stat-file fp_stats{}.stat
echo -n "12 21 30 38 47 56" | xargs -d " " -n 1 -P 3 -I {} aomenc --frame-parallel=0 --tile-columns=0 --auto-alt-ref=1 --cpu-used=3 --passes=2 --threads=2 --row-mt=1 --end-usage=q --cq-level={} -o test.av1.cq{}.webm orig.i420.y4m
VMAF: model used: vmaf_b_v0.6.3, pooling: harmonic_mean, bagging score (arithmetic mean of 21 models' scores)
Start-to-end times:
x264: 0 hours, 34 minutes and 58 seconds
x265: 4 hours, 13 minutes and 57 seconds
libvpx-vp9: 3 hours, 17 minutes and 52
rav1e: 9 hours, 8 minutes and 25 seconds
SVT-AV1: 5 hours, 6 minutes and 13 seconds
libaom: 7 hours, 26 minutes and 5 seconds

This concludes this report.
As always, I'm open to any kind of feedback to improve my comparisons and my encodes.
SmilingWolf is offline   Reply With Quote
Old 24th December 2019, 22:24   #2038  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,507
Good to see that VMAF-wise, SVT-AV1 has reached parity with vpxenc and x265, at least for 8-bit, and is only a little slower than x265. That's a pretty big milestone, and the codebase is still in a lot of flux, so there's probably some headroom.

BTW, SVT has supported y4m for a long time now, they just haven't updated their readme.The encoder guide in general is pretty outdated, but I want to see where the holiday cleanup goes before I send pull requests.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 25th December 2019, 05:56   #2039  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 444
Phoronix benchmarked SVT-AV1 v0.8 on a variety of CPUs (i7, i9, Xeon, Ryzen, Threadripper, Epyc):

https://www.phoronix.com/scan.php?pa...0.8-Benchmarks
__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.6GHz Pentium G3258 (2c/2t Haswell)
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       

Win7 x64
Nintendo Maniac 64 is offline   Reply With Quote
Old 27th December 2019, 09:55   #2040  |  Link
ts1
Registered User
 
Join Date: Jan 2015
Posts: 71
SmilingWolf, have you made any tweaks to PSNRHVS? AFAIK by default cweight is set to 1, that's too high, set it to 0.125.
Or just use this already tweaked and improved version from libaom (only with this version first video must be original and distorted is second).
ts1 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.