Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th September 2018, 16:21   #981  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Got a few additional questions about aomenc, since I couldn't find any doc about it.
  • Does anyone know where I can lookup the default values for all the options where the help doesn't provide a default value?
  • What are the allowed values for '--usage' and '--profile'?
  • What is the difference between 'Usage' and 'Bitstream' profile?
  • What are the 'forced_max_frame_width' and 'forced_max_frame_height' values for? Does the force to encoder to internally not downscale width/height?
  • What is the recommend value for '--timebase' and when should one change this value?
  • What is the 'resize-mode'-option for and what are valid values for it?
  • What are the 'superres' options for?
  • What are S-frames and what are the sframe-modes?
  • What does the 'input-chroma-subsampling' options for and what are valid values for them?

Thanks for anyone who help with some answers.

Cu Selur
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 19th September 2018 at 16:57.
Selur is offline   Reply With Quote
Old 19th September 2018, 17:19   #982  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
Quote:
Originally Posted by Selur View Post
Got a few additional questions about aomenc, since I couldn't find any doc about it.
  • Does anyone know where I can lookup the default values for all the options where the help doesn't provide a default value?
You can check some defaults here: https://aomedia.googlesource.com/aom...eed_features.c
Not the most user friendly way but that's all there is for now unfortunately

As for the other questions, my best guess is that they are described one way or the other in the spec.
Or in the sources... somewhere. Yeah not quite the best for end users.

Last edited by SmilingWolf; 19th September 2018 at 17:22.
SmilingWolf is offline   Reply With Quote
Old 19th September 2018, 17:30   #983  |  Link
easyfab
Registered User
 
Join Date: Jan 2002
Posts: 332
encoding default :
https://aomedia.googlesource.com/aom...cx_iface.c#107
easyfab is offline   Reply With Quote
Old 19th September 2018, 17:31   #984  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Thanks!
about s-frames (switching frames): https://www.youtube.com/watch?v=o5sJX6VA34o seems to be only interesting for abr low latency live streaming, so not interesting when you don't have multiple streams of different bitrates available.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 19th September 2018 at 17:48.
Selur is offline   Reply With Quote
Old 19th September 2018, 22:51   #985  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Very cool. I should go to Demuxed this year!
Blue_MiSfit is offline   Reply With Quote
Old 20th September 2018, 01:19   #986  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 293
Found this comment in libaom code:
Code:
// TODO(hkuang): Fix the pthread_cond_broadcast in windows wrapper.
Apparently it was made in this commit for vp9. I wonder is it still true?

(Also, libaom phtreads wrapper has a lot of code for pre-Vista Windows. Why? Nobody bothered to clean it up? )
lvqcl is offline   Reply With Quote
Old 20th September 2018, 07:34   #987  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
The last issue (unaligned memory access in GCC 8.2.0) was marginally related to cleaning up VP9 remains...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 21st September 2018, 08:03   #988  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 80
I have diferent Coding path when encoding with differents aom builds with the same y4m.

Coding path: LBD
Coding path: HBD

Can't you explain why and how to change this?
marcomsousa is offline   Reply With Quote
Old 21st September 2018, 08:47   #989  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
My builds come with the CONFIG_LOWBITDEPTH build option enabled.
It enables 8bit content optimized codepaths, which work roughly 2x (in theory) - 1.75x (in practice) faster than the high (10-12) bit depth codepaths because you can stuff 8 bits in just 8 bits of memory, while to use 10-12bits you need to use 16 bits of memory, halving the throughput.
The default of that build options is 0, which means the 8bits codepaths are never used, and a lot of builds out there just use default settings (MABS ones in primis).

Lotsa yadda yadda on my part, issues 2062 and 2147 probably explain better what this means for end users

Side note: today's build are complete, few minutes they'll be up on MEGA.
There's a little feature I've been keen to try for a long time now: loop filter bitmask for decoding, which promises a 6% decoding performance increase in single thread

Last edited by SmilingWolf; 21st September 2018 at 08:54.
SmilingWolf is offline   Reply With Quote
Old 21st September 2018, 08:51   #990  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
I will mention this in the MABS project, maybe wiiaboo agrees to enable it (or even make a choice to build either or both, like for x264 and x265, unless that are able to provide a multilib solution as well).
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 21st September 2018, 09:09   #991  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
I think there's no need for CONFIG_LOWBITDEPTH=0 builds, with CONFIG_LOWBITDEPTH=1 the appropriate codepaths are selected automatically based on input and output bit depths.

Today's build:
av1-1.0.0-629-g7b9ddd4bf: https://mega.nz/#!cg5njRjA!WZkuEg2g7...vEcCgQL5_vRMY8

Cautionary note: the issues tracker has been populating quite fast the last couple of days, so there's a chance this is not the most stable build to work with.
I have just finished a series of encodes that took all week , so I'll be able to try it and report back if anything nasty happens.
Also, some coding features have been temporarily disabled for row-mt to accomodate some corner cases

Silver lining: the code is being modified to enable easier debugging of threading related issues, so maybe we'll see some newfound stability on that front in the coming days/weeks

Last edited by SmilingWolf; 21st September 2018 at 15:09.
SmilingWolf is offline   Reply With Quote
Old 21st September 2018, 13:14   #992  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 80
I'm building (yet another version) AOM with Visual Studio 2017 from sources.

They are automatically builds

AV1 executable builds: here
I also open source the build scripts at Github: here

I note that with VS2017 generates a lot of *.exe compare with gcc
  • aomdec.exe
  • decode_with_drops.exe
  • decode_to_md5.exe
  • twopass_encoder.exe
  • dump_obu.exe
  • lightfield_decoder.exe
  • lightfield_bitstream_parsing.exe
  • lightfield_encoder.exe
  • aom_cx_set_ref.exe
  • lightfield_tile_list_decoder.exe
  • aomenc.exe
  • lossless_encoder.exe
  • noise_model.exe
  • scalable_decoder.exe
  • resize_util.exe
  • scalable_encoder.exe
  • set_maps.exe
  • simple_encoder.exe
  • simple_decoder.exe

At this moment I'm only saving aomenc.exe and aomdec.exe.
You can do pull requests to improve this script.

Last edited by marcomsousa; 21st September 2018 at 13:25.
marcomsousa is offline   Reply With Quote
Old 21st September 2018, 13:31   #993  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
All of that EXEs are generated under MSYS2 too, they are simply not installed by "make install" because they are examples or test tools, not meant for the end user.
Code:
# find . -name \*.exe -type f
./aomdec.exe
./aomenc.exe
./CMakeFiles/3.12.1/CompilerIdC/a.exe
./CMakeFiles/3.12.1/CompilerIdCXX/a.exe
./examples/aom_cx_set_ref.exe
./examples/decode_to_md5.exe
./examples/decode_with_drops.exe
./examples/lightfield_bitstream_parsing.exe
./examples/lightfield_decoder.exe
./examples/lightfield_encoder.exe
./examples/lightfield_tile_list_decoder.exe
./examples/lossless_encoder.exe
./examples/noise_model.exe
./examples/scalable_decoder.exe
./examples/scalable_encoder.exe
./examples/set_maps.exe
./examples/simple_decoder.exe
./examples/simple_encoder.exe
./examples/twopass_encoder.exe
./resize_util.exe
./tools/dump_obu.exe
Love the AppVeyor integration! Thanks!

Last edited by SmilingWolf; 21st September 2018 at 13:40.
SmilingWolf is offline   Reply With Quote
Old 21st September 2018, 13:54   #994  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Just chatted a bit in IRC ... gnafu believes that CONFIG_LOWBITDEPTH=1 is a) still necessary to be set at compile time, b) in general useful, because it enables an optimized code path for 8 bit precision only, but does not alter the behaviour of higher bit depths.

I hope he is right.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 21st September 2018, 14:04   #995  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 80
Quote:
Originally Posted by LigH View Post
Just chatted a bit in IRC ... gnafu believes that CONFIG_LOWBITDEPTH=1 is a) still necessary to be set at compile time, b) in general useful, because it enables an optimized code path for 8 bit precision only, but does not alter the behaviour of higher bit depths.

I hope he is right.
I tested with CONFIG_LOWBITDEPTH=1 and it's faster. But it was disable because something...

Code:
Turn off CONFIG_LOWBITDEPTH by default

CodecWG agreed to have this off for default "C" model.
Commit
__________________
AV1 win64 VS2019 builds
Last build here

Last edited by marcomsousa; 21st September 2018 at 15:28.
marcomsousa is offline   Reply With Quote
Old 21st September 2018, 16:50   #996  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 56
Who is gnafu?
alex1399 is offline   Reply With Quote
Old 21st September 2018, 16:59   #997  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
Quote:
Originally Posted by alex1399 View Post
Who is gnafu?
https://freenode.logbot.info/aomedia/20180921

If your question is "who's him in the project", I think he's just an enthusiast like most of the people here on doom9
SmilingWolf is offline   Reply With Quote
Old 21st September 2018, 17:48   #998  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
Graphs time! Click to enlarge
Y axis: chosen metric
X axis: bits per pixel

720p:


1080p:


BD rates for 720p:
Code:
rav1e -> x264
        RATE (%) DSNR (dB)
 MSSSIM -18.3758 0.910887
PSNRHVS -17.3064 1.14428

x264 -> x265
        RATE (%) DSNR (dB)
 MSSSIM -25.6195 1.21115
PSNRHVS -29.8289 1.83058

x265 -> av1
        RATE (%) DSNR (dB)
 MSSSIM -16.6292 0.680221
PSNRHVS -13.0642 0.641722
BD rates for 1080p:
Code:
rav1e -> x264
        RATE (%) DSNR (dB)
 MSSSIM -19.4204 0.869977
PSNRHVS -16.9114 0.949348

x264 -> x265
        RATE (%) DSNR (dB)
 MSSSIM -28.9956 1.18206
PSNRHVS -31.474 1.63676

x265 -> av1
        RATE (%) DSNR (dB)
 MSSSIM -24.1474 0.840846
PSNRHVS -19.4281 0.796909

Clips used for 720p: F.Y.C, KimiNoNa720, PresageFlowerFight720, PresageFlowerWalk720, TearsOfSteel720, TheFifthElement, ThisIsHalloween
Clips used for 1080p: KimiNoNa, PresageFlowerFight, PresageFlowerWalk, TearsOfSteel

Encoders:
x264 157-2932-303c484
x265 2.8-68-fa57fa584898
rav1e 0.1.0-315-1fa32bbd
av1 1.0.0-577-g8ae39302e

VQM Tools:
dump_msssim and dump_psnrhvs from the daala git repo: https://github.com/xiph/daala
Everything glued together with some bash and python2 scripts
The metrics used are from the Total field of the tools' output, so they take into account both luma and chroma. In dump_msssim this just looks like an arithmetic mean between the three measurements, while dump_psnrhvs applies some weighting.
The final metrics-per-resolution have been obtained taking the measurement for each clip and weighting it with the number of pixels.

Cmdlines:
x264 --preset veryslow --tune ssim --crf 16 -o test.x264.crf16.264 orig.i420.y4m
x265 --preset veryslow --tune ssim --crf 16 -o test.x265.crf16.hevc orig.i420.y4m
rav1e -o test.rav1e.cq80.webm --quantizer 80 -s 3 --tune psnr orig.i420.y4m
aomenc --frame-parallel=0 --tile-columns=6 --auto-alt-ref=1 --cpu-used=4 --tune=psnr --passes=2 --threads=4 --end-usage=q --cq-level=20 --test-decode=fatal -o test.av1.cq20.webm orig.i420.y4m

Quality settings:
x264: CRF 16-24 step 1, and 24-34 step 2
x265: CRF 16-24 step 1, and 24-34 step 2
rav1e: CQ 80-160 step 16
aomenc: CQ 20-40 step 4

Clips used:
F.Y.C: https://amvnews.ru/index.php?go=Files&in=view&id=6801, second link, the 250.78Mb one. Clip cut with ffmpeg -ss 00:00:10 -t 20, 1280x720, 480 frames
KimiNoNa and KimiNoNa720: scene from the Kimi no Namae Wa, BD source. Clip cut with ffmpeg -ss 00:01:13 -t 17, 1920x1080, 410 frames
PresageFlowerFight and PresageFlowerFight720: scene from Fate/stay night: Heaven's Feel - Presage Flower, BD source. Clip cut with ffmpeg -ss 00:43:50 -t 15, 1920x1080, 360 frames
PresageFlowerWalk and PresageFlowerWalk720: scene from Fate/stay night: Heaven's Feel - Presage Flower, BD source. Clip cut with ffmpeg -ss 00:15:14 -t 13, 1920x1080, 312 frames
TearsOfSteel and TearsOfSteel720: https://media.xiph.org/tearsofsteel/...l-1080bis-png/ frames 13500 to 13900, 1920x800, 400 frames
TheFifthElement: clip found on the #aomedia channel: https://freenode.logbot.info/aomedia/20180726 -> themayhaks.com/~gideon/tfe.mkv, 1280x534, 240 frames
ThisIsHalloween: https://www.animemusicvideos.org/mem...o.php?v=196998, second link, the LOCAL/123 MiB one. Clip cut with ffmpeg -ss 00:01:50 -t 15, 1280x720, 450 frames

Clips notes/misc details:
All of them have been losslessly cut and stored in FFv1 in an MKV container to serve as master files, decoded to y4m for encoding. All the master files use 8bits 4:2:0 subsampling, and so do the encodes.
The *720 clips actually have been resized to a 1280 width and the heigth has been adjusted accordingly, so either 720 or 534 as required to keep the aspect ratio.
Resizing done with ffmpeg using the z.img library/zscale video filter, using a 4 taps lanczos kernel.
F.Y.C: mixed and superimposed anime/real life content, lots of effects, rain in a couple of scenes, fast motion, flashes, many scenecuts. AMV encoder unknown, final bitrate around 9.4Mb/s
KimiNoNa and KimiNoNa720: anime, detailed backgrounds, brief scene with moving croud, cut to moving trains, cut to moving background with static foreground, few scenecuts
PresageFlowerFight and PresageFlowerFight720: anime, fight between Rider and Saber for whoever knows what I'm talking about, lots of particles at some point, reasonably fast motion, few scenecuts
PresageFlowerWalk and PresageFlowerWalk720: anime, MC just walking around in the school, no really detailed backgrounds, mostly static and slow, very few scenecuts
TearsOfSteel and TearsOfSteel720: real life with synthetic/rendered elements, smoke, rapid camera movement and motion, very few scenecuts
TheFifthElement: real life, lots of film grain, static, very few scenecuts. The original uploader sourced the clip from the BD and resized it himself
ThisIsHalloween: anime, no effects, some flames in a scene, mostly static but brief scenes, many scenecuts. Scenes used come from the BD box of the anime (Soul Eater), the final encode of the AMV has been made with x264, CRF 15 and --preset veryslow --tune animation, final bitrate around 4.3Mb/s

As you have already noticed, my testbed mostly covers my personal interests: AMVs (F.Y.C and ThisIsHalloween) and anime in general, with some real life content thrown in the mix
Natural consequence, since I tailored this to my needs, is that I used only encoding settings I felt confortable with, especially with aomenc:
- maximum number of tiles allowed per clip for smooth playback and (whenever it didn't bug out) multithreaded encoding
- cpu-used no lower than 4 or I would have never seen the end of it

Alright, wall of text over, I'm open to any kind of feedback since I'm posting this to get corrections or suggestions

Last edited by SmilingWolf; 21st September 2018 at 18:21.
SmilingWolf is offline   Reply With Quote
Old 21st September 2018, 18:31   #999  |  Link
NikosD
Registered User
 
Join Date: Aug 2010
Location: Athens, Greece
Posts: 2,901
Quote:
Originally Posted by SmilingWolf View Post
Alright, wall of text over, I'm open to any kind of feedback since I'm posting this to get corrections or suggestions
Oh man!

I'm speechless of your work and of your post!

Good to see you around.
__________________
Win 10 x64 (19042.572) - Core i5-2400 - Radeon RX 470 (20.10.1)
HEVC decoding benchmarks
H.264 DXVA Benchmarks for all
NikosD is offline   Reply With Quote
Old 21st September 2018, 21:26   #1000  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 80
Quote:
Originally Posted by SmilingWolf View Post
...
Excelent post
__________________
AV1 win64 VS2019 builds
Last build here
marcomsousa is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.