Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th February 2019, 18:14   #6721  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
is there a way to compile the x64 version without avx512?

it only increases the size of the library and i will not use avx512 anyway. for now i'm sticking to v2.7
thanks

Last edited by poller; 18th February 2019 at 18:19.
poller is offline   Reply With Quote
Old 18th February 2019, 18:42   #6722  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 894
Quote:
Originally Posted by poller View Post
is there a way to compile the x64 version without avx512?
Not sure how the size of the library matters, but anyway.

Open \source\common\x86\asm-primitives.cpp, find X265_CPU_AVX512 and note all the avx512 asms.

Remove them from *.asm files, remove the X265_CPU_AVX512 code block, and then try to compile them.
__________________
Projects
x265 - Yuuki-Asuna-mod Download / GitHub
TS - ADTS AAC Splitter | LATM AAC Splitter | BS4K-ASS
Neo AviSynth+ filters - F3KDB | FFT3D | DFTTest | MiniDeen | Temporal Median
MeteorRain is offline   Reply With Quote
Old 18th February 2019, 21:25   #6723  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by excellentswordfight View Post
I just did a quick test on tears of steal with those settings and compared it an encode at native res with slow preset (+ --deblock -1:-1 --no-sao --no-strong-intra-smoothing)... the 1080p one ended up at 40Mbps (downscaled with spline), and the 2160 one at 25, the one at at native res was sharper and was about 3x faster to encode...

Ofc, if you find the time/compression/quality to be a good tradeoff for you go ahead with that. But when you even have a 4k TV I would jut go with a 4k workflow cause you can get away with 4k re-encodes with that bitrate target. Not gonna tell you what to do, but I would at least play with different preset levels and crf values and look at the actually video to see if you can find a better "sweetspot". Cause it sounds awful to me o spend 2days for an encode, just to get a bloated file that still has a loss of detail/sharpness (cause of downscaling).
Yeah, an important thing about 4K encoding is that the pixels are getting near the minimize visible size in typical viewing environments, especially with non-HDR content. So small artifacts that could be visible in 1080p content become MUCH less so at 4K. I've heard a lot of "but 4K will take 4x the bitrate of 1080p), but the frequency distribution gets softer and latitude for non-perceptible distortion gets greater. Doubling bitrate from 1080p to 2160p is typically sufficient to get value out of the extra pixels without any regressions.

In fact a satisfactory H.264 8-bit 1080p bitrate will generally be a satisfactory HEVC 10-bit 2160p bitrate. The "4K will kill the internets!" panic was badly overblown.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book

Last edited by benwaggoner; 18th February 2019 at 21:26. Reason: specified bit depths
benwaggoner is offline   Reply With Quote
Old 18th February 2019, 22:31   #6724  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
Quote:
Originally Posted by MeteorRain View Post
Not sure how the size of the library matters, but anyway.

Open \source\common\x86\asm-primitives.cpp, find X265_CPU_AVX512 and note all the avx512 asms.

Remove them from *.asm files, remove the X265_CPU_AVX512 code block, and then try to compile them.
that's how i started, but i thought there might be a simpler way.

but it does work, size is only slighter bigger than v2.7 now.

thanks.
poller is offline   Reply With Quote
Old 18th February 2019, 23:00   #6725  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by poller View Post
but it does work, size is only slighter bigger than v2.7 now.
I am curious as to why that is worth this effort.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 19th February 2019, 18:32   #6726  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
Quote:
Originally Posted by benwaggoner View Post
I am curious as to why that is worth this effort.
well, it didn't take really long, maybe 30 minutes.
i need ffmpeg for recording from an emulator, and i don't want to see the ffmpeg.dll being 5 times bigger than the actual emu.
so i'm trying to keep it small.


and now... more questions.

first, v3.0 is slower and produces bigger output than previous versions. i did read the changelog, but fail to see what could be the reason (yes, noob here).
/edit: ok, it seems to be aq-mode=2


second, x86 builds by LigH are about 10% (!) faster than mine. is there some magical compiler flag? i tried -O2 and -Ofast... no luck. -Ofast being the slowest of them actually.
x64 builds are at the same speed.


Last edited by poller; 19th February 2019 at 18:41.
poller is offline   Reply With Quote
Old 19th February 2019, 18:52   #6727  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
No magical flags. Just out of media-autobuild suite (more or less, negligible differences in the handling). Do we use the same compiler?
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 19th February 2019, 18:56   #6728  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
v3.0 changed some presets, so maybe the change in speed comparing to v2.7 comes from there. I don't know if the x265 docs are helpful in checking out what changed (I doubt )
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 19th February 2019, 20:35   #6729  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Boulder View Post
v3.0 changed some presets, so maybe the change in speed comparing to v2.7 comes from there. I don't know if the x265 docs are helpful in checking out what changed (I doubt )
Specifically, slower got a lot slower, and veryslow is even more very slow. Slower was a nice sweet spot quality/perf step; I think it would have made more sense to add a superslow or something.

Try using slow and seeing if that helps.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 19th February 2019, 20:37   #6730  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
Quote:
Originally Posted by LigH View Post
No magical flags. Just out of media-autobuild suite (more or less, negligible differences in the handling). Do we use the same compiler?
strange. i now tested with GCC 7.3 from MSYS2 (it seems this is the one you are using), the executable is a lot bigger and even slightly slower than my other builds (GCC 4.9.3).
i am using the default values of x265.exe for testing.
this is beyond me.


Quote:
Originally Posted by Boulder View Post
v3.0 changed some presets, so maybe the change in speed comparing to v2.7 comes from there. I don't know if the x265 docs are helpful in checking out what changed (I doubt )
as mentioned, if i set --aq-mode=1 speed of v3.0 is pretty much the same as before, also the size of output.
no idea about the quality, there might be some other changes as you say.

Last edited by poller; 19th February 2019 at 20:40.
poller is offline   Reply With Quote
Old 19th February 2019, 21:45   #6731  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by poller View Post
strange. i now tested with GCC 7.3 from MSYS2 (it seems this is the one you are using), the executable is a lot bigger and even slightly slower than my other builds (GCC 4.9.3).
i am using the default values of x265.exe for testing.
this is beyond me.
By "default values" you would be implicitly using --preset medium. No idea if that is what you want to be using for your scenario.

Quote:
as mentioned, if i set --aq-mode=1 speed of v3.0 is pretty much the same as before, also the size of output.
no idea about the quality, there might be some other changes as you say.
--aq-mode 2 is the default in 3.0, but was 1 in previous versions. I wouldn't think it would change performance THAT much.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 19th February 2019, 22:55   #6732  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
it seems i started some confusion here.

i tested another video, about the same results.

Quote:
--aq-mode=1
own builds: 40.5 sec
LigH builds: 35.5 sec
so my builds are even more than 10% slower, no matter what preset i use. so annoying.
GCC 9.01, GCC 8.2... always the same bad speed.

Quote:
--aq-mode=2
own builds: 47.5 sec
LigH builds: 40.5 sec
so yes, for MY short low res test videos mode 2 is slower.
that might be different for other input files.
poller is offline   Reply With Quote
Old 20th February 2019, 09:45   #6733  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 894
Quote:
Originally Posted by poller View Post
second, x86 builds by LigH are about 10% (!) faster than mine. is there some magical compiler flag? i tried -O2 and -Ofast... no luck. -Ofast being the slowest of them actually.
x64 builds are at the same speed.

Could be some profiling magic. I'm unsure if LigH uses any profiling when compiling the binary though.

I used it once, made it slower. And I haven't used it since, but things might have changed.
__________________
Projects
x265 - Yuuki-Asuna-mod Download / GitHub
TS - ADTS AAC Splitter | LATM AAC Splitter | BS4K-ASS
Neo AviSynth+ filters - F3KDB | FFT3D | DFTTest | MiniDeen | Temporal Median
MeteorRain is offline   Reply With Quote
Old 20th February 2019, 10:03   #6734  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Nope, already answered that he simply uses media-autobuild suite (see:https://forum.doom9.org/showthread.p...45#post1866145), so no profiling.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 20th February 2019, 10:31   #6735  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Some build log could be useful, maybe there is something missing. I'd expect that if assembler was not used, the difference would be much bigger though.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 20th February 2019, 21:24   #6736  |  Link
poller
Registered User
 
Join Date: Sep 2018
Posts: 12
i tried hard with GCC again.

seconds. lower is better.

Code:
137.0 no assembly
 47.0 (default)
 45.5 (PGO build) -mtune=ivybridge (default here is -O3 which makes 1st pass PGO .exe crash, thus no better speed i guess)
 44.5 (PGO build) -mtune=ivybridge -O2
 43.9 (PGO build) -mtune=ivybridge -funroll-loops -finline-functions -ftree-loop-vectorize -O2
 39.5 LigH
so i get little improvement with all that fiddling, but still far away from LigH's GCC builds.


giving up here, i have no ideas left.

Last edited by poller; 20th February 2019 at 21:48.
poller is offline   Reply With Quote
Old 21st February 2019, 00:14   #6737  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
OK, I forgot little details I edited a long time ago, while testing some compiling issues with a faulty compiler version. A leftover string is:

export CXXFLAGS="-march=pentium4 -mtune=generic"

for the 32-bit compilation (which is still quite generic, just a sensible minimum). That might bring a little advantage. For the 64-bit compilation, the CXXFLAGS is empty.

Furthermore, for the 32-bit compilation, assembly is disabled for 10 and 12 bit precision cores, but enabled for the 8 bit core.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 21st February 2019 at 00:17.
LigH is offline   Reply With Quote
Old 21st February 2019, 10:39   #6738  |  Link
WhatZit
Registered User
 
Join Date: Aug 2016
Posts: 60
Quote:
Originally Posted by Wolfberry View Post
Supply --svt in the command line to use the SVT-HEVC encoder.
Never expected that one!

From http://x265.org/x265-svt-hevc-house/:

Quote:
With changeset a41325fc854f, the x265 library can invoke the SVT-HEVC library for encoding through the —svt option. We have mapped presets and command-line options supported by the x265 application into the equivalent options of SVT-HEVC, and have added a few specific options that are available only when the SVT-HEVC library is invoked. This page in our documentation describes the steps to build, and invoke the SVT-HEVC library in more detail.

Our reason for this integration was to enable our users to evaluate additional relative trade-offs between performance and compression efficiency while working behind the familiar API of the x265 library. In the long term, we plan to leverage this integration to further improve x265’s ability to handle real-time and low turn-around scenarios in pure software; this is the space that SVT-HEVC was focused on. In parallel, we will continue to innovate on our flagship presets that are used in offline encoding where x265 dominates. You can expect to see these changes in the coming releases of x265, increasing the reach of open-source for video compression!
Am I being cynical to suggest that Multicoreware couldn't achieve such speed optimisations on their own, so they formed this "synergy"?
WhatZit is offline   Reply With Quote
Old 21st February 2019, 10:43   #6739  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,342
Personally I think its stupid to incorporate another encoder into the x265 "frontend". If one wanted to use different encoders, one would use say ffmpeg, or just use them directly. x265 should be x265, and nothing else. But oh well. Probably some business driving over common sense.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 21st February 2019, 11:27   #6740  |  Link
shinchiro
Registered User
 
Join Date: Feb 2012
Posts: 46
Lol, good to know I'm not the only one who thinks x265's decision to include another encoder inside itself is stupid. Well, if there's money involved here, I'm not even surprised
shinchiro is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.