x265 HEVC Encoder

[LigH]

Originally Posted by LigH

How do I set up a different codepage in the MSYS console? Localized tools, e.g. hg, detect German language, but Umlauts are displayed incorrectly.

As the default shell is sh.exe, I assume I would have to edit ~/.inputrc somehow (if I could find a verbose specification for it)?

Or maybe sh.exe does not even support codepage configuration at all, and I should try to prefer mintty instead?

[MeteorRain]

Please excuse me if someone has mentioned this before.

In x264-10bit, the CRF value actually results in average qp near CRF+12 to maintain similar visual quality. For example, CRF 20 on 10 bits encoding actually results CRF "32".

In x265 high bit depth encoding, have we already applied the same mechanism? Or should we manually adjust the CRF value to ~+12 on 10 bits or even more on 12 bits to get a similar visual quality level?

[ARRYMatie]

Originally Posted by foxyshadis

You're doing it wrong if you get banding with x265. Either switch up to 10-bit or add some grain, but I just don't believe you that x264 would look uniform where x265 doesn't at the same bitrate and bit depth.

Been there, tried that. Was using 10bit in the beginning, even tried 12 and 16bit. But it looks like x265 does not process colors/grain as well as x264.

The banding problem could be solved if I added grain manually but then it'll require even more bitrate than the x264 version just to cover the flaws. If that's the case, might as well stick to x264.

x265 also has some discoloration errors here and there that cause color splots where as on x264 it'll be uniform.

[foxyshadis]

Originally Posted by ARRYMatie

Been there, tried that. Was using 10bit in the beginning, even tried 12 and 16bit. But it looks like x265 does not process colors/grain as well as x264.

The banding problem could be solved if I added grain manually but then it'll require even more bitrate than the x264 version just to cover the flaws. If that's the case, might as well stick to x264.

x265 also has some discoloration errors here and there that cause color splots where as on x264 it'll be uniform.

Are you sure this isn't the effects of a crappy 6-bit TN monitor? Because all of my experience with x265 so far is exactly the opposite, and I've never seen a 10-bit encode that's anything but banding-free unless the input already had a ton of banding.

[burfadel]

Originally Posted by foxyshadis

Are you sure this isn't the effects of a crappy 6-bit TN monitor? Because all of my experience with x265 so far is exactly the opposite, and I've never seen a 10-bit encode that's anything but banding-free unless the input already had a ton of banding.

Yes that's true. A TN panel will naturally band, the only reason why it doesn't seem to is because of dithering done by the monitor itself.

The encode can only be ever as good as the input to the encoder. If the input has banding, the encoder will actually encode that banding as information which is why you see it in the output file.

[littlepox]

Originally Posted by foxyshadis

Are you sure this isn't the effects of a crappy 6-bit TN monitor? Because all of my experience with x265 so far is exactly the opposite, and I've never seen a 10-bit encode that's anything but banding-free unless the input already had a ton of banding.

It can also be the case where he is using a poor video player. I've seen such case, where potplayer's default settings actually give you a heavily banding output:

1. The internal ffmpeg decoder decodes 10bit HEVC and then convert it to 8bit output
2. EVR render converts that to RGB under 8bit precision
3. Video card then "enhance" the video once again in poor precision. AMD products enable this by default.

[LigH]

Just compared my last builds with brief benchmarks (best of 3, foreman.y4m in PAL CIF) on a legacy CPU (AMD Phenom-II X4, max. used SIMD = SSE2); for slow preset, I got a relation of ~5.0 fps (GCC 4.9.2) : ~5.5 fps (GCC 5.2.0), that should be significant.

[dipje]

Originally Posted by LigH

BTW, please excuse a little off-topic: How do I set up a different codepage in the MSYS console? Localized tools, e.g. hg, detect German language, but Umlauts are displayed incorrectly.

Just thinking out loud here, but I know that with certain Git-for-windows installs (GitExtensions for one) that is the install-time option to change a Windows registry setting somewhere to enable Unicode and UTF-8 on the console.

So I'm thinking that your shell is outputting the correct codes for Umlauts and stuff, but Windows is just forcing the console window to something ancient (or the font that is used to display the console isn't an unicode-compatible font, stuff like that).

[qyot27]

The way this tends to work on Windows is that you're *not* dealing with the actual Bourne shell (or bash, or dash, etc.), but with (ba|da)sh running under cmd.exe. And even forcing it into UTF-8 with chcp 65001 before launching sh doesn't seem to really do anything, as far as I can recall.

If, however, you're using mintty (MSys2 does this) as the host for the shell, then you can just change the codepage to whatever you want in the Options dialog. Although I don't necessarily know how this reacts on other OS codepage settings, but mintty is set on en.US and UTF-8 and it correctly prints eszett and umlauted characters just fine (and o with stroke, and ñ, and thorn, etc.). Not that I think the en.US part is doing anything, that's simply because I am under the English OS codepage here. I'm sure that it's simply because mintty actually does honor the UTF-8 setting.

[LigH]

I switched to mintty as shell by calling "msys.bat --mintty"; this is a quite advanced shell window application and supports locales and codepages (and Windows CP1252 is the correct one here). No need to try to force sh-in-cmd into something possibly not really supported.

[LoRd_MuldeR]

Getting Unicode (UTF-8) strings to show up properly in a Console window on the Windows platform can be quite challenging

First of all, you have to avoid the the stdio-layer of the C-Runtime mangles your UTF-8 strings. This is going to happen by default! With Visual C++, you have to use the non-standard _setmode() function, with _O_U8TEXT or _O_BINARY flag, so that your strings get passed though undistorted. Not sure about MinGW. But you can always skip the stdio-layer altogether and use Win32 API functions, like WriteConsole(), directly. This avoids one source of string distortion.

But even if you manage that your UTF-8 string actually arrives at the Console undistorted, you still have to ensure the Console will interpret it correctly. You have to call SetConsoleOutputCP() with CP_UTF8. Last but not least, the default Console font on Windows (called "Consolas") is not Unicode-aware! You explicitly have to change the font to "Lucidia Console" to make things works. Not sure the font issue can be solved pragmatically. Maybe with some Registry hack...

[LigH]

So I guess mintty does a conversion here. The required changes are minimal, the result is satisfying, and on top I get some fancy gadgets (e.g. a translucent plain or even the "Aero Glass" blurry background).

[foxyshadis]

Originally Posted by LoRd_MuldeR

Last but not least, the default Console font on Windows (called "Consolas") is not Unicode-aware! You explicitly have to change the font to "Lucidia Console" to make things works. Not sure the font issue can be solved pragmatically. Maybe with some Registry hack...

Consolas is missing CJK, other East Asian characters, and Arabic, but it has all Latin and Cyrillic characters, at least. So it has some Unicode support.

[LigH]

How about "DejaVu Sans Mono"? I believe it contains a large subset of Unicode too.

[x265.cc]

Originally Posted by Ma

My patch works, but is not perfect.

mingw-w64 4.0.4 is out. AFAIK this problem is not fixed. am i right? =(

Originally Posted by LigH

It may be more elaborate, but you may want to build the multilib executables to combine 8 + 10 + 12 bit encoder routines into one EXE. Don't forget to strip, that saves a few MB. And 7-zip will compress such large 3-in-1 files a lot more efficiently than ZIP with its small LZ window (only 32 or 64 KB, I believe).

I'm not able to cross-compile a multilib x265.exe on linux. I really want to but i lack of intelligence.
I dont use 7-zip because its not bundled with windows.


Maybe i will automatically build 10 and 12 bit binaries too soon or will manage to build the multilib pe file.

//edit: 10-bit and 12-bit binaries are now build automatically too (for x86-64)

./libx265_main12.a(level.cpp.obj): could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
CMakeFiles/cli.dir/build.make:360: recipe for target 'x265.exe' failed
make[2]: *** [x265.exe] Error 1
CMakeFiles/Makefile2:99: recipe for target 'CMakeFiles/cli.dir/all' failed
make[1]: *** [CMakeFiles/cli.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

[ 22%] Built target encoder
[ 83%] Built target common
[ 85%] Built target x265-static
[ 86%] Linking CXX executable x265.exe
libx265.a(param.cpp.obj): could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
CMakeFiles/cli.dir/build.make:360: recipe for target 'x265.exe' failed
make[2]: *** [x265.exe] Error 1
CMakeFiles/Makefile2:99: recipe for target 'CMakeFiles/cli.dir/all' failed
make[1]: *** [CMakeFiles/cli.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

[qyot27]

Originally Posted by x265.cc

I'm not able to cross-compile a multilib x265.exe on linux. I really want to but i lack of intelligence.

https://github.com/qyot27/mpv/blob/e...ious.txt#L1131

Dependency: x265
================

# Any options with [] around them are optional

cd ~/mpv-build-deps

# Supporting both 8bit and higher bit depths requires an out-of-tree build:
mkdir -p x265-build/12bit x265-build/10bit x265-build/8bit
cd x265-build
hg clone https://bitbucket.org/multicoreware/x265

# Build 12-bit:
cd 12bit
     cmake ../x265/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-12bit" \
     -DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
     -DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
     -DHIGH_BIT_DEPTH:bool=on -DMAIN12:bool=on -DENABLE_SHARED:bool=off \
     -DENABLE_ASSEMBLY:bool=off -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off [-DWINXP_SUPPORT:bool=on]
ninja
     sudo checkinstall --pkgname=x265-main12-mingw --pkgversion="$(grep X265_VERSION \
     build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
     --rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
     --deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main12.a

# Build 10-bit:
cd 10bit
     cmake ../x265/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-10bit" \
     -DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
     -DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
     -DHIGH_BIT_DEPTH:bool=on -DENABLE_SHARED:bool=off -DENABLE_ASSEMBLY:bool=off \
     -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off [-DWINXP_SUPPORT:bool=on]
ninja
     sudo checkinstall --pkgname=x265-main10-mingw --pkgversion="$(grep X265_VERSION \
     build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
     --rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
     --deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main10.a

# Only the .a files from 12-bit and 10-bit are installed to the system,
# to reduce the chances of conflicts.

# Build 8-bit:
cd ../8bit
     cmake ../x265/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="/usr/i686-w64-mingw32" \
     -DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
     -DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
     -DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
     -DEXTRA_LIB="/usr/i686-w64-mingw32/lib/libx265_main10.a;/usr/i686-w64-mingw32/lib/libx265_main12.a" \
     [-DWINXP_SUPPORT:bool=on]
ninja
sed -i 's/lx265/lx265 -lx265_main10 -lx265_main12/' x265.pc
     sudo checkinstall --pkgname=x265-mingw --pkgversion="$(grep X265_VERSION \
     build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
     --rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
     --deldesc=yes --strip=yes --fstrans=no --default ninja install

# 32-bit builds with HIGH_BIT_DEPTH enabled also require the use of
# -DENABLE_ASSEMBLY:bool=off.

# Enabling Windows XP support now requires using the -DWINXP_SUPPORT:bool=on
# option, because the API default is now Win7.

The sed line in the 8-bit instructions is only so FFmpeg can link against the combined static lib. For only x265.exe itself, that's not necessary.

Also, you'll notice some of that stuff is very specific to my own environment (32-bit, PIII compiler opts, XP support), so adapt as needed.

[Ma]

Originally Posted by x265.cc

mingw-w64 4.0.4 is out. AFAIK this problem is not fixed. am i right? =(

Yes, you're right. It is not fixed even on trunk. The last response from mingw-w64 team was 2015-08-02:

Personally, I think the patch is nearly OK, but the lock/unlock
functions should go into its own files, not something hard to correct.

Kai, anything?

I sent new version of patch 2015-08-03 and there is silence. I was so irritated that I've installed VS 2015.

[Barough]

Originally Posted by Ma

I sent new version of patch 2015-08-03 and there is silence. I was so irritated that I've installed VS 2015.

Any chance we could see and 32-bit VS 2015 complies from you also or?

[Ma]

Originally Posted by Barough

Any chance we could see and 32-bit VS 2015 complies from you also or?

Depends of speed results. When I tested 64-bit versions of 10-bit x265 for AVX-CPU, the result was very close with asm turned on (depends of Windows mood, sometimes vs2015 build was the fastest).

But for this options:

--no-asm --preset slower --crf 17.5 --rdoq-level 1 --deblock -1 -f 200 1920x800-hob.y4m w.hevc

GCC 5.2 linked to msvcr120.dll encoded in 676.90s (0.30 fps) and vs2015 -- in 807.63s (0.25 fps). And 107% of best result is 1.07*676.90s = 724.283s. It means that vs2015 (without asm) result is not within 107% of fastest time.

32-bit versions of 10-bit x265 are without asm, so I assumed that vs2015 builds could be disqualified due to 107% rule. I will make speed test and take a look at real numbers.

[Barough]

Not interested in any 10/12-bit encodes here

The VS2015 x64 compiles sure is faster on my i7 so a x86 complie would be nice see 4 making compares 2 ur normal x86 binaries