Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th September 2018, 07:33   #901  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,388
Quote:
Originally Posted by Mr_Khyron View Post
Youtube has started to encode videos to AV1
https://www.youtube.com/watch?v=F1B9Fk_SgI0

You must have youtube-dl (2018-09-10)
Holy cow, as soon as the extensions get support I'm going to be downloading everything both ways and testing. Hopefully AV1 broad release lives up to its promise.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 11th September 2018, 07:56   #902  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Hi y'all!

So, since I keep finding more and more alignment-related bugs that only show up on GCC 8.2, but not on 7.3 (compiler's fault? aomedia's fault? Who knows <rant>they have barely gotten any attention despite the community's effort and the fact they involve common configurations and major compiler releases</rant>) I decided to just go for it and build a GCC 7.3 cross compilation VM

Builds specs:
GCC 7.3, 64bits only, statically linked, secure api enabled, so no Win XP support
lbd binaries have been compiled with -DCONFIG_LOWBITDEPTH=1
hbd binaries have been compiled with -DCONFIG_LOWBITDEPTH=0
Use the former for all your 8bits needs since they are considerably faster, the latter for 10-12bits

av1-1.0.0-541-g7d447f5b0: https://mega.nz/#!0wxmgCqS!vBuViCEA7...v1MubYI0lJnluI

Last edited by SmilingWolf; 11th September 2018 at 09:40.
SmilingWolf is offline   Reply With Quote
Old 11th September 2018, 09:58   #903  |  Link
Pushman
Registered User
 
Join Date: Nov 2017
Posts: 5
It is in HD now:

Pushman is offline   Reply With Quote
Old 11th September 2018, 11:08   #904  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,690
@SmilingWolf: I'll keep an eye on it, as soon as MABS passes in both branches again. But for now, there is a greater evil breaking the building of ffmpeg. If the API for AV1 was ever "frozen", it seems to melt already.

And yes, I shared one of the issues of GCC 8.2.0 creating unaligned RAM access when vector optimization is enabled, several weeks ago, and apart from adding more CC's, not much happened about it.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 11th September 2018 at 11:12.
LigH is offline   Reply With Quote
Old 11th September 2018, 11:46   #905  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Quote:
Originally Posted by LigH View Post
@SmilingWolf: I'll keep an eye on it, as soon as MABS passes in both branches again. But for now, there is a greater evil breaking the building of ffmpeg. If the API for AV1 was ever "frozen", it seems to melt already.
Well TBF they froze the bitstream and released a "stable" version 1.0, such issues are to be expected when using an old and moving codebase that needs some cleaning up after many different iterations.
Shouldn't take much more than removing lines 741-744 from libavcodec/libaomenc.c
This one's on FFMPEG, and will probably be fixed in a day or so

What bothers me much more is the same as the second part of your post, the inertia over months-long periods of time in the AOM tracker

Quote:
Originally Posted by LigH View Post
And yes, I shared one of the issues of GCC 8.2.0 creating unaligned RAM access when vector optimization is enabled, several weeks ago, and apart from adding more CC's, not much happened about it.
Regarding this, they are evaluating using the patch I suggested in comment 8 as a short term fix. BUT now it can't be tested with GCC 8.2 on master, because there is another couple of access violations of various nature happening at different optimization levels because of (as far as I've been able to test) some unholy interaction between GCC 8.2 and AVX code

Side note: unaligned memory access is fine AS LONG AS the ASM instructions emitted take into account memory could be unaligned. In this instance GCC 8.2 -O3 emits instructions that expect memory to be aligned to a 16-byte boundary (movdqa and movaps), while GCC 8.2 -O3 -fno-tree-slp-vectorize emits instructions fit for unaligned memory access (movdqu and movups)

My advice is to stay on GCC 7.3 for AV1 for the time being. I found the laptop I was using back in june to build A/V stuff with an old toolchain, if you want I can zip it up and upload

Last edited by SmilingWolf; 11th September 2018 at 11:59.
SmilingWolf is offline   Reply With Quote
Old 11th September 2018, 18:13   #906  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,690
And I believe MABS even only selects -O2.

MSYS2 can't include GCC 8.2.0 in the 32-bit branch because it doesn't build itself without errors. I like how MABS keeps MSYS2 up-to-date before updating and compiling the projects. I don't want to miss this feature completely due to very rare compiler issues limiting the speed of the generated code; ultimately it has to be fixed in GCC, and when this happens, MABS will update MSYS2 to it.

I could dig in my archives and find an older MSYS with GCC 7.x too, but I don't want to stuff my drive for a rather little reason. I hope I am not the only provider of binaries...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 11th September 2018 at 18:21.
LigH is offline   Reply With Quote
Old 11th September 2018, 18:32   #907  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Nope, the only custom GCC flags MABS uses are the ones that won't cause UMA (because typing unaligned memory access all the time is tiring): https://github.com/jb-alvarado/media...ompile.sh#L837
Every other compiler flag is left to the default, which implies -O3

Anyway, I think I have almost worked out what's wrong with issue 2145 and might be able to propose a temporay fix and an explanation shortly. So that would be one out of everybody's hair

As for binaries, as I said I can compile 64bit stuff on my VM now (the first ones are right above, in my first post), if there's need for something particular I guess I can tweak my setup. E.g. I might be able to stuff an extra 32bit toolchain on the VM. Just, no WinXP support, please

Last edited by SmilingWolf; 11th September 2018 at 18:36.
SmilingWolf is offline   Reply With Quote
Old 11th September 2018, 18:54   #908  |  Link
easyfab
Registered User
 
Join Date: Jan 2002
Posts: 327
with MABS, personnaly I create a custom_build_options file ( see L. 63-66 media-suite_compile.sh) with

export CFLAGS="-march=native -O3 -pipe"
export CXXFLAGS="${CFLAGS}"
easyfab is offline   Reply With Quote
Old 12th September 2018, 03:50   #909  |  Link
TD-Linux
Registered User
 
Join Date: Aug 2015
Posts: 22
Quote:
Originally Posted by mandarinka View Post
1) there is a general lack of configurable settings and I heard some rumor that this might actually be somewhat a policy (like the nonexistent user-facing configurabiltiy in Theora?) and not just effect of being early in development.
I plan to add debugging options exactly for the purpose of getting feedback from users - the first obvious thing to add is the internal cdef-dist parameters to tune them. That said, they will remain debug options - for example, we may change how cdef-dist works internally when we find a better way to do it, and that will break the options. If you find a combination of debug options that looks better, I'd rather you report it as a bug and we use that information to change the defaults.

BTW we are getting segmentation soon - once we get it I'll let you know, it'll make the effect of cdef-dist far stronger.

Quote:
Originally Posted by mandarinka View Post
2) a smaller thing: there is very little feedback given by the commandline encoder. A FPS counter probably isn't completely important, but there is one thing that really would help with testing, and that is a achieved bitrate being reported at the end. I was rather suprised Rav1e didn't report this, because I wanted to encode a comparison clip with 2pass with another encoder and this is rather complicating. I have to figure it out based on filesize but the output's already in a container that has some unknown overhead... I used bitrate calculator like in 2006 but note that those generally only have 1 second precision, which is not very exact for short testing samples.
I would happily take patches for this. That said, the top priority is actually to get it integrated with ffmpeg, which will give you a printout of the framerate and bitrate, plus allow muxing and many more input formats.
TD-Linux is offline   Reply With Quote
Old 12th September 2018, 09:19   #910  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
av1-1.0.0-552-gbb82e05fb: https://mega.nz/#!99ASgKAJ!l7Lc4n9KE...9FgrYxXAEaoByw

As promised, the explanation for BUG=aomedia:2145 has dropped. Does anybody know if one of x264/x86inc.asm maintainers could take a look?
There are some contacts in the header of the file, I suppose my best bet is e-mailing Henrik Gramner, who seems responsible for most of the recent commits, yes?
SmilingWolf is offline   Reply With Quote
Old 12th September 2018, 09:40   #911  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,454
If a register is being used has to be flagged by the ASM function in question (in its cglobal line), x86inc.asm cannot detect that automatically, the code has to tell it. If its not pushing regs that are in fact being used, then the cglobal line may be wrong. From the code it looks like the macro is being passed a "7" for numbers of register used, while RSI would be the 9th in their counting (ie. r8). I don't see it directly being used in quantize_b, but it maybe used behind a symbolic name to load a parameter.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is online now   Reply With Quote
Old 12th September 2018, 09:47   #912  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Cr*p and I just sent a mail to Henrik.

Yes, RSI is used here for dequantq

Ok let me test that real quick.
Thanks for the input!
SmilingWolf is offline   Reply With Quote
Old 12th September 2018, 09:58   #913  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,454
If dequant is the highest parameter being used, changing the 7 to a 9 at the end of the file might be all that is needed.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is online now   Reply With Quote
Old 12th September 2018, 10:03   #914  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
YES! Fix confirmed!

EDIT: and patch sent: https://bugs.chromium.org/p/aomedia/...ail?id=2145#c7

Last edited by SmilingWolf; 12th September 2018 at 10:14.
SmilingWolf is offline   Reply With Quote
Old 12th September 2018, 11:28   #915  |  Link
GTPVHD
Registered User
 
Join Date: Mar 2008
Posts: 237
http://www.streamingmedia.com/Articl...ok-127133.aspx



Quote:
As you can see, AV1 was the clear leader, H264 the clear laggard, and x265 and VP9 neck and neck in the middle.
GTPVHD is offline   Reply With Quote
Old 12th September 2018, 12:29   #916  |  Link
Aleksoid1978
Registered User
 
Aleksoid1978's Avatar
 
Join Date: Apr 2008
Location: Russia, Vladivostok
Posts: 2,066
Quote:
Originally Posted by SmilingWolf View Post
av1-1.0.0-552-gbb82e05fb: https://mega.nz/#!99ASgKAJ!l7Lc4n9KE...9FgrYxXAEaoByw

As promised, the explanation for BUG=aomedia:2145 has dropped. Does anybody know if one of x264/x86inc.asm maintainers could take a look?
There are some contacts in the header of the file, I suppose my best bet is e-mailing Henrik Gramner, who seems responsible for most of the recent commits, yes?
Hello. Can you compile static lib for use with any project, like ffmpeg ?? I need x86/x64 AV1 decoder only.

I don't know why - but on my home/work PC i can't compile, cmake error on AVX2(my PC don't support AVX2).
__________________
I7 2600K@4.2 /Asrock P67 Extreme4 Gen 3 /Kingston HyperX 8Gb 1866 (4x2) Kit /OCZ Vertex 3 256Gb /Gigabyte GTX 960 /BenQ EW2430 /LG 47LM620T /Yamaha RX-V471 + NS-555 + NS-C444 + NS-333 + YST-SW215
Aleksoid1978 is offline   Reply With Quote
Old 12th September 2018, 13:11   #917  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 37
Quote:
Originally Posted by GTPVHD View Post
I like more the second round testing at lower data rates and this comment:

Quote:
You should focus your tests on the data rates at which your video will most likely be deployed. At this point, H.264 and any newer codec should produce near perfect quality at 6 Mbps, making that data rate irrelevant for forward-looking testing. HEVC and VP9 take the near perfect quality level down to between 3.5Mbps to 4Mbps, and AV1 and future codecs should bring this down into the 2Mbps to 3.5Mbps range.
marcomsousa is offline   Reply With Quote
Old 12th September 2018, 13:50   #918  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Quote:
Originally Posted by Aleksoid1978 View Post
Hello. Can you compile static lib for use with any project, like ffmpeg ?? I need x86/x64 AV1 decoder only.

I don't know why - but on my home/work PC i can't compile, cmake error on AVX2(my PC don't support AVX2).
You should post a log and some infos on your building env. Missing AVX2 at compile or even configure time looks like an outdated version of GCC rather than an issue with your CPU.
Maybe there is an older version/toolchain in your $PATH that's interfering?
GCC 7/8 can emit AVX2 instructions even when running on an old Pentium 4.

Last edited by SmilingWolf; 12th September 2018 at 13:58.
SmilingWolf is offline   Reply With Quote
Old 12th September 2018, 14:06   #919  |  Link
Aleksoid1978
Registered User
 
Aleksoid1978's Avatar
 
Join Date: Apr 2008
Location: Russia, Vladivostok
Posts: 2,066
Use VS 2017 or GCC 8.2.0 - there is no difference.
__________________
I7 2600K@4.2 /Asrock P67 Extreme4 Gen 3 /Kingston HyperX 8Gb 1866 (4x2) Kit /OCZ Vertex 3 256Gb /Gigabyte GTX 960 /BenQ EW2430 /LG 47LM620T /Yamaha RX-V471 + NS-555 + NS-C444 + NS-333 + YST-SW215
Aleksoid1978 is offline   Reply With Quote
Old 12th September 2018, 14:18   #920  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 68
Can you post the full cmake -G "MSYS Makefiles" etc. etc. log on pastebin as it appears on screen?
Or, better/more complete, the contents of <yourBuildDir>/CMakeFiles/CMakeError.log and CMakeOutput.log?

Last edited by SmilingWolf; 12th September 2018 at 14:21.
SmilingWolf is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.