Alliance for Open Media codecs

[foxyshadis]

Originally Posted by Mr_Khyron

Youtube has started to encode videos to AV1
https://www.youtube.com/watch?v=F1B9Fk_SgI0

You must have youtube-dl (2018-09-10)

Holy cow, as soon as the extensions get support I'm going to be downloading everything both ways and testing. Hopefully AV1 broad release lives up to its promise.

[SmilingWolf]

Hi y'all!

So, since I keep finding more and more alignment-related bugs that only show up on GCC 8.2, but not on 7.3 (compiler's fault? aomedia's fault? Who knows <rant>they have barely gotten any attention despite the community's effort and the fact they involve common configurations and major compiler releases</rant>) I decided to just go for it and build a GCC 7.3 cross compilation VM

Builds specs:
GCC 7.3, 64bits only, statically linked, secure api enabled, so no Win XP support
lbd binaries have been compiled with -DCONFIG_LOWBITDEPTH=1
hbd binaries have been compiled with -DCONFIG_LOWBITDEPTH=0
Use the former for all your 8bits needs since they are considerably faster, the latter for 10-12bits

av1-1.0.0-541-g7d447f5b0: https://mega.nz/#!0wxmgCqS!vBuViCEA7...v1MubYI0lJnluI

[Pushman]

It is in HD now:

[LigH]

@SmilingWolf: I'll keep an eye on it, as soon as MABS passes in both branches again. But for now, there is a greater evil breaking the building of ffmpeg. If the API for AV1 was ever "frozen", it seems to melt already.

And yes, I shared one of the issues of GCC 8.2.0 creating unaligned RAM access when vector optimization is enabled, several weeks ago, and apart from adding more CC's, not much happened about it.

[SmilingWolf]

Originally Posted by LigH

@SmilingWolf: I'll keep an eye on it, as soon as MABS passes in both branches again. But for now, there is a greater evil breaking the building of ffmpeg. If the API for AV1 was ever "frozen", it seems to melt already.

Well TBF they froze the bitstream and released a "stable" version 1.0, such issues are to be expected when using an old and moving codebase that needs some cleaning up after many different iterations.
Shouldn't take much more than removing lines 741-744 from libavcodec/libaomenc.c
This one's on FFMPEG, and will probably be fixed in a day or so

What bothers me much more is the same as the second part of your post, the inertia over months-long periods of time in the AOM tracker

Originally Posted by LigH

And yes, I shared one of the issues of GCC 8.2.0 creating unaligned RAM access when vector optimization is enabled, several weeks ago, and apart from adding more CC's, not much happened about it.

Regarding this, they are evaluating using the patch I suggested in comment 8 as a short term fix. BUT now it can't be tested with GCC 8.2 on master, because there is another couple of access violations of various nature happening at different optimization levels because of (as far as I've been able to test) some unholy interaction between GCC 8.2 and AVX code

Side note: unaligned memory access is fine AS LONG AS the ASM instructions emitted take into account memory could be unaligned. In this instance GCC 8.2 -O3 emits instructions that expect memory to be aligned to a 16-byte boundary (movdqa and movaps), while GCC 8.2 -O3 -fno-tree-slp-vectorize emits instructions fit for unaligned memory access (movdqu and movups)

My advice is to stay on GCC 7.3 for AV1 for the time being. I found the laptop I was using back in june to build A/V stuff with an old toolchain, if you want I can zip it up and upload

[LigH]

And I believe MABS even only selects -O2.

MSYS2 can't include GCC 8.2.0 in the 32-bit branch because it doesn't build itself without errors. I like how MABS keeps MSYS2 up-to-date before updating and compiling the projects. I don't want to miss this feature completely due to very rare compiler issues limiting the speed of the generated code; ultimately it has to be fixed in GCC, and when this happens, MABS will update MSYS2 to it.

I could dig in my archives and find an older MSYS with GCC 7.x too, but I don't want to stuff my drive for a rather little reason. I hope I am not the only provider of binaries...

[SmilingWolf]

Nope, the only custom GCC flags MABS uses are the ones that won't cause UMA (because typing unaligned memory access all the time is tiring): https://github.com/jb-alvarado/media...ompile.sh#L837
Every other compiler flag is left to the default, which implies -O3

Anyway, I think I have almost worked out what's wrong with issue 2145 and might be able to propose a temporay fix and an explanation shortly. So that would be one out of everybody's hair

As for binaries, as I said I can compile 64bit stuff on my VM now (the first ones are right above, in my first post), if there's need for something particular I guess I can tweak my setup. E.g. I might be able to stuff an extra 32bit toolchain on the VM. Just, no WinXP support, please

[easyfab]

with MABS, personnaly I create a custom_build_options file ( see L. 63-66 media-suite_compile.sh) with

export CFLAGS="-march=native -O3 -pipe"
export CXXFLAGS="${CFLAGS}"

[TD-Linux]

Originally Posted by mandarinka

1) there is a general lack of configurable settings and I heard some rumor that this might actually be somewhat a policy (like the nonexistent user-facing configurabiltiy in Theora?) and not just effect of being early in development.

I plan to add debugging options exactly for the purpose of getting feedback from users - the first obvious thing to add is the internal cdef-dist parameters to tune them. That said, they will remain debug options - for example, we may change how cdef-dist works internally when we find a better way to do it, and that will break the options. If you find a combination of debug options that looks better, I'd rather you report it as a bug and we use that information to change the defaults.

BTW we are getting segmentation soon - once we get it I'll let you know, it'll make the effect of cdef-dist far stronger.

Originally Posted by mandarinka

2) a smaller thing: there is very little feedback given by the commandline encoder. A FPS counter probably isn't completely important, but there is one thing that really would help with testing, and that is a achieved bitrate being reported at the end. I was rather suprised Rav1e didn't report this, because I wanted to encode a comparison clip with 2pass with another encoder and this is rather complicating. I have to figure it out based on filesize but the output's already in a container that has some unknown overhead... I used bitrate calculator like in 2006 but note that those generally only have 1 second precision, which is not very exact for short testing samples.

I would happily take patches for this. That said, the top priority is actually to get it integrated with ffmpeg, which will give you a printout of the framerate and bitrate, plus allow muxing and many more input formats.

[SmilingWolf]

av1-1.0.0-552-gbb82e05fb: https://mega.nz/#!99ASgKAJ!l7Lc4n9KE...9FgrYxXAEaoByw

As promised, the explanation for BUG=aomedia:2145 has dropped. Does anybody know if one of x264/x86inc.asm maintainers could take a look?
There are some contacts in the header of the file, I suppose my best bet is e-mailing Henrik Gramner, who seems responsible for most of the recent commits, yes?

[nevcairiel]

If a register is being used has to be flagged by the ASM function in question (in its cglobal line), x86inc.asm cannot detect that automatically, the code has to tell it. If its not pushing regs that are in fact being used, then the cglobal line may be wrong. From the code it looks like the macro is being passed a "7" for numbers of register used, while RSI would be the 9th in their counting (ie. r8). I don't see it directly being used in quantize_b, but it maybe used behind a symbolic name to load a parameter.

[SmilingWolf]

Cr*p and I just sent a mail to Henrik.

Yes, RSI is used here for dequantq

Ok let me test that real quick.
Thanks for the input!

[nevcairiel]

If dequant is the highest parameter being used, changing the 7 to a 9 at the end of the file might be all that is needed.

[SmilingWolf]

YES! Fix confirmed!

EDIT: and patch sent: https://bugs.chromium.org/p/aomedia/...ail?id=2145#c7

[GTPVHD]

http://www.streamingmedia.com/Articl...ok-127133.aspx

As you can see, AV1 was the clear leader, H264 the clear laggard, and x265 and VP9 neck and neck in the middle.

[Aleksoid1978]

Originally Posted by SmilingWolf

av1-1.0.0-552-gbb82e05fb: https://mega.nz/#!99ASgKAJ!l7Lc4n9KE...9FgrYxXAEaoByw

As promised, the explanation for BUG=aomedia:2145 has dropped. Does anybody know if one of x264/x86inc.asm maintainers could take a look?
There are some contacts in the header of the file, I suppose my best bet is e-mailing Henrik Gramner, who seems responsible for most of the recent commits, yes?

Hello. Can you compile static lib for use with any project, like ffmpeg ?? I need x86/x64 AV1 decoder only.

I don't know why - but on my home/work PC i can't compile, cmake error on AVX2(my PC don't support AVX2).

[marcomsousa]

Originally Posted by GTPVHD

http://www.streamingmedia.com/Articl...ok-127133.aspx

I like more the second round testing at lower data rates and this comment:

You should focus your tests on the data rates at which your video will most likely be deployed. At this point, H.264 and any newer codec should produce near perfect quality at 6 Mbps, making that data rate irrelevant for forward-looking testing. HEVC and VP9 take the near perfect quality level down to between 3.5Mbps to 4Mbps, and AV1 and future codecs should bring this down into the 2Mbps to 3.5Mbps range.

[SmilingWolf]

Originally Posted by Aleksoid1978

Hello. Can you compile static lib for use with any project, like ffmpeg ?? I need x86/x64 AV1 decoder only.

I don't know why - but on my home/work PC i can't compile, cmake error on AVX2(my PC don't support AVX2).

You should post a log and some infos on your building env. Missing AVX2 at compile or even configure time looks like an outdated version of GCC rather than an issue with your CPU.
Maybe there is an older version/toolchain in your $PATH that's interfering?
GCC 7/8 can emit AVX2 instructions even when running on an old Pentium 4.

[Aleksoid1978]

Use VS 2017 or GCC 8.2.0 - there is no difference.

[SmilingWolf]

Can you post the full cmake -G "MSYS Makefiles" etc. etc. log on pastebin as it appears on screen?
Or, better/more complete, the contents of <yourBuildDir>/CMakeFiles/CMakeError.log and CMakeOutput.log?