Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th August 2016, 09:45   #1  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
[Solved] Issue when including ffms2 in x264

I have (for a change... ) an issue including ffms2 in x264.
The last time i've built it, everything was fine, but when i tried this WE, it failed.
As there was a few update since my last build, i created a branch from the commit i've successfully build the last time (even if looking at the changes it wasn't probably that), but it also failed.
It wasn't ffmpeg because i've for both used the exact same version...
The other thing that change also, is that i'm using msys2, and they have updated gcc from 5.3.0 (the one i've used when i successfully build) to 6.1.0.
I've also wanted, out of curiosity, to try with ffmpeg 3.1.1, but it doesn't compile at all with gcc 6.1.0, when it worked with gcc 5.3.0.
The error log generated from the x264 build is here.
There was no error during the build of ffms2.

I need expert advises. From my point of view, it seems to be a compiler issue.
Is 6.1.0 broken ? Do i have to add an option in the compiler for things to work ?

Any help appreciated...

Last edited by jpsdr; 28th September 2016 at 18:57.
jpsdr is offline   Reply With Quote
Old 8th August 2016, 15:53   #2  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
I am building x264 fine using media-autobuild_suite
compiler is 6.1.0
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 8th August 2016, 16:09   #3  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Don't have issue to build x264, issue is with including ffms2 within x264.
jpsdr is offline   Reply With Quote
Old 8th August 2016, 16:22   #4  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Just checked configure log and it says "ffms:no", sorry.
Anyway I would run autobuild_suite and if it fails report issue to author.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 9th August 2016, 03:26   #5  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
6.1.0 isn't broken, if anything it may be that you're not being adequately threading-aware with how ffms2 and x264 are configured (or 6.1.0 was built in a similarly thread-mismatched way, but that doesn't mean 6.1.0 is broken).

Those errors are all C++11-related, and ffms2 uses C++11 threading, which requires using a build of GCC built with MinGW's winpthreads implementation (*-gcc-posix under MSys2, in this case). I'm pretty sure you *have* to use x264's pthreads support in this case, leaving it on win32threads will probably result in those kinds of detection errors.
qyot27 is offline   Reply With Quote
Old 9th August 2016, 09:09   #6  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Thanks for your tip, it helped. It was a little more complicated than that (i'll update my x264 build tutorial according what i've found).

Last edited by jpsdr; 9th August 2016 at 09:11.
jpsdr is offline   Reply With Quote
Old 9th August 2016, 19:03   #7  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Hmm... Seems to be more tricky than expected.
When i've made a x86 build, ffms2 was included in x264, but not when i've made a x64 build...
Need more tests. One thing for sure, something which used to work is not anymore...
jpsdr is offline   Reply With Quote
Old 9th August 2016, 22:56   #8  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Ok, i've made a lot of tests, and i'm still thinking it's a compiler issue.

If i build for x86, everything work fine, whatever i choose (posix or win32threads).
But, if i build for x64, it allways failled, whatever i choose.

Log file for x86 gives :
Code:
G:/msys32/local-x264_x86/posix/Original/lib\libffms2.a(filehandle.o):filehandle.cpp:(.text$_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_[__ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_]+0x6d): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()'
G:/msys32/local-x264_x86/posix/Original/lib\libffms2.a(filehandle.o):filehandle.cpp:(.eh_frame+0x8b): undefined reference to `__gxx_personality_v0'
G:/msys32/local-x264_x86/posix/Original/lib\libffms2.a(filehandle.o):filehandle.cpp:(.eh_frame$_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_+0x13): undefined reference to `__gxx_personality_v0'
collect2.exe: error: ld returned 1 exit status
--------------------------------------------------
Failed program was:
--------------------------------------------------
#include <ffms.h>
int main (void) { FFMS_DestroyVideoSource(0); return 0; }
--------------------------------------------------
checking for FFMS_DestroyVideoSource(0); in ffms.h... yes
checking whether FFMS_VERSION >= ((2 << 24) | (21 << 16) | (0 << 8) | 0) is true... yes
checking for lsmash.h... yes
with finaly :
Code:
lavf:          yes
ffms:          yes
mp4:           lsmash
Log file for x64 gives :
Code:
G:/msys64/local-x264_x64/posix/t_mod/lib\libffms2.a(filehandle.o):filehandle.cpp:(.text$_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_[_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_]+0x78): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
G:/msys64/local-x264_x64/posix/t_mod/lib\libffms2.a(filehandle.o):filehandle.cpp:(.text$_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_[_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_]+0x86): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()'
G:/msys64/local-x264_x64/posix/t_mod/lib\libffms2.a(filehandle.o):filehandle.cpp:(.xdata$_ZStplIcSt11char_traitsIcESaIcEENSt7__cxx1112basic_stringIT_T0_T1_EEPKS5_RKS8_+0xc): undefined reference to `__gxx_personality_seh0'
collect2.exe: error: ld returned 1 exit status
--------------------------------------------------
Failed program was:
--------------------------------------------------
#include <ffms.h>
int main (void) { FFMS_DestroyVideoSource(0); return 0; }
--------------------------------------------------
checking for FFMS_DestroyVideoSource(0); in ffms.h... no
Failed commandline was:
--------------------------------------------------
gcc conftest.c -m64  -Wall -I. -I$(SRCPATH)  -D_POSIX_C_SOURCE=200112L -std=gnu99 -D_GNU_SOURCE   -I/local-x264_x64/posix/t_mod/include  -L/local-x264_x64/posix/t_mod/lib -lz -lffms2 -lavformat -liconv -lm -llzma -lbz2 -lz -pthread -lpsapi -ladvapi32 -lshell32 -lavcodec -liconv -lm -llzma -lbz2 -lz -pthread -lpsapi -ladvapi32 -lshell32 -lswscale -lm -lavresample -lm -lavutil -lm -lstdc++ -lavresample -L.  -L/local-x264_x64/posix/t_mod/lib -lavformat -liconv -lm -llzma -lbz2 -lz -pthread -lpsapi -ladvapi32 -lshell32 -lavcodec -liconv -lm -llzma -lbz2 -lz -pthread -lpsapi -ladvapi32 -lshell32 -lswscale -lm -lavutil -lm  -lshell32 -Wl,--image-base,0x140000000 -m64  -static  -Wl,--high-entropy-va -Wl,--dynamicbase,--nxcompat,--tsaware -lpthread -o conftest
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z35_txnal_cow_string_C1_for_exceptionsPvPKcS_+0x2c): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_RU1'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z35_txnal_cow_string_C1_for_exceptionsPvPKcS_+0x39): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `transaction clone for operator new[](unsigned long long)'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z35_txnal_cow_string_C1_for_exceptionsPvPKcS_+0x5d): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_memcpyRtWn'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z23_txnal_cow_string_c_strPKv+0x1): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_RU8'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z23_txnal_sso_string_c_strPKv+0x1): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_RU8'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z20_txnal_cow_string_D1Pv+0x5): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_RU8'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_Z20_txnal_cow_string_D1Pv+0x1e): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_addUserCommitAction'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_ZGTtNSt11logic_errorC1EPKc+0x2e): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_memcpyRnWt'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_ZGTtNSt11logic_errorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x2e): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_memcpyRnWt'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_ZGTtNSt11logic_errorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x36): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `_ITM_RU8'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0\libstdc++.a(cow-stdexcept.o):(.text$_ZGTtNSt11logic_errorD0Ev+0x1a): additional relocation overflows omitted from the output
collect2.exe: error: ld returned 1 exit status
--------------------------------------------------
Failed program was:
--------------------------------------------------
#include <ffms.h>
int main (void) { FFMS_DestroyVideoSource(0); return 0; }
--------------------------------------------------
with finaly :
Code:
lavf:          yes
ffms:          no
mp4:           lsmash
jpsdr is offline   Reply With Quote
Old 10th August 2016, 20:10   #9  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
So, expert advise help asked. Is there any command line i have to add in the x64 to make it work again ? Or is it realy finaly a compiler issue, and i have to wait for a next update in msys2 ?
jpsdr is offline   Reply With Quote
Old 11th August 2016, 15:13   #10  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Quote:
Originally Posted by jpsdr View Post
Or is it realy finaly a compiler issue, and i have to wait for a next update in msys2 ?
Assuming the problem is in the compiler (GCC 6.1), then (I think) you don't have to "update" MSYS2, just choose an older compiler (5.3 or 5.4). I've read about some issues with GCC 6.1 in the x265 thread, and exactly because of that, I will stick to GCC 5.3 «until I-don't-know-when».
filler56789 is offline   Reply With Quote
Old 11th August 2016, 16:39   #11  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by filler56789 View Post
Assuming the problem is in the compiler (GCC 6.1), then (I think) you don't have to "update" MSYS2, just choose an older compiler (5.3 or 5.4). I've read about some issues with GCC 6.1 in the x265 thread, and exactly because of that, I will stick to GCC 5.3 «until I-don't-know-when».
It was just the stack boundary alignment, IIRC. And that's been fixed now.
qyot27 is offline   Reply With Quote
Old 11th August 2016, 18:21   #12  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Ok, so i'll wait msys2 update gcc. Don't know how to roll back, and don't want to bother to... Thanks for your informations.
jpsdr is offline   Reply With Quote
Old 31st August 2016, 22:32   #13  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Quote:
Originally Posted by jpsdr View Post
Ok, so i'll wait msys2 update gcc. Don't know how to roll back, and don't want to bother to... Thanks for your informations.
But assuming this post may be useful to other people...

well, it's not difficult to change the MinGW-w64 compilers used by MSYS2,
all one has to do is change the contents of the mingw* directories.
One may even use version "A.B.C" for 32-bits and version "X.Y.Z" for 64-bits
(and vice-versa).

FWIW — I have built and packed some ``older´´ GCC bundles for MSYS2,
they can be downloaded from one of my Mediafire folders:

https://www.mediafire.com/?3ib29yauv5kdw
filler56789 is offline   Reply With Quote
Old 1st September 2016, 08:04   #14  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Thanks. There is older and there is newer.
Should i replace the whole mingw* diretory, or just overwrite in the directory your bundle, to keep the existing files that may not be in the bundle ?

Last edited by jpsdr; 1st September 2016 at 08:11.
jpsdr is offline   Reply With Quote
Old 1st September 2016, 09:39   #15  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
First of all, backup the current contents of the mingw32 and mingw64 directories, then you can safely empty these directories and re-fill them with older or newer stuff.

By the way: when configuring MSYS2 for the very-first time, it's OK to do anything, except download and install GCC through the MSYS2 shell Depending on the circumstances, choosing/accepting the «default» options and settings is not the better choice

FWIW (again...) — I use only GCC builds which were compiled to use «pure» pthreads-w32

(many to both Zeranoe and nevcairiel).
filler56789 is offline   Reply With Quote
Old 1st September 2016, 15:41   #16  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Ok... Woh... You can even choose posix/win32 thread version ! More choice than pacman.
Ok, for now, i'll do everything except gcc with pacman so...
Stupid question. When i build x264, i made a posix and a win32 version. Is the best result compiling the posix version with posix gcc, and win32 with win32 gcc ?

Any specific release advised (sjsj, dwarf,seh) or anyone is good ?

Last edited by jpsdr; 1st September 2016 at 15:49.
jpsdr is offline   Reply With Quote
Old 2nd September 2016, 20:53   #17  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
I've tried to use the gcc version from the links provided by HolyWu
ffmpeg, lsmash and x264 build properly.
With win32 version, even with komisar patch, ffms2 don't compile.
With posix version, ffms2 compile but link with error, tried 5.4.0 and 6.2.0, dwarf x86 and seh x64 (i haven't tried the sjsj), i've this link error :
Code:
G:/msys32/mingw32/bin/../lib/gcc/i686-w64-mingw32/6.2.0/../../../../i686-w64-mingw32/lib/../lib/libiconv.a(localcharset.o):localcharset.c:(.text+0x7): undefined reference to `_imp__GetACP@0'
collect2.exe: error: ld returned 1 exit status
make: *** [Makefile:651: src/index/ffmsindex.exe] Error 1
  CXXLD    src/index/ffmsindex.exe
G:/msys32/mingw32/bin/../lib/gcc/i686-w64-mingw32/6.2.0/../../../../i686-w64-mingw32/lib/../lib/libiconv.a(localcharset.o):localcharset.c:(.text+0x7): undefined reference to `_imp__GetACP@0'
collect2.exe: error: ld returned 1 exit status
make: *** [Makefile:651: src/index/ffmsindex.exe] Error 1
Any idea ? Am i missing something ?
jpsdr is offline   Reply With Quote
Old 3rd September 2016, 10:17   #18  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Ok, i've tried everything, but only when i put gcc in the pacman, i'm able to build ffms2 until the end. If in the pacman i put everything i used to, except just gcc, and get any of the release and copy it in mingw32 (i've tried all of them), ffms2 don't link with the error i've notified in previous post.
So, finaly, unless someone is able to tell me what i'm missing, return to square on, i'll have to wait that msys2 update its gcc version to solve the issue i have with 64 bits.
jpsdr is offline   Reply With Quote
Old 4th September 2016, 09:08   #19  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Quote:
Originally Posted by jpsdr View Post
I've tried to use the gcc version from the links provided by HolyWu
ffmpeg, lsmash and x264 build properly.
With win32 version, even with komisar patch, ffms2 don't compile.
With posix version, ffms2 compile but link with error, tried 5.4.0 and 6.2.0, dwarf x86 and seh x64 (i haven't tried the sjsj), i've this link error :
Code:
 <snip>
Any idea ? Am i missing something ?
AFAIK, those "posix" builds from the MinGW-w64 site use winpthreads, which might explain why they don't solve your problem.

Quote:
Originally Posted by jpsdr View Post
Ok, i've tried everything, but only when i put gcc in the pacman, i'm able to build ffms2 until the end. If in the pacman i put everything i used to, except just gcc, and get any of the release and copy it in mingw32 (i've tried all of them), ffms2 don't link with the error i've notified in previous post.
So, finaly, unless someone is able to tell me what i'm missing, return to square on, i'll have to wait that msys2 update its gcc version to solve the issue i have with 64 bits.
Have you already discussed these issues with the "ffms2 people"?
I can be wrong, but I still think the actual problem is not in GCC or in MSYS2, but in the current ffms2 code.
To make things clear(er): I suggested older versions of GCC as a possible (and acceptable) workaround, not as an actual solution...

— in the same way I had to compile the latest ffmbc with GCC 4.8.5 and 4.9.4, because cc1(.exe) from GCC 5.x insists in treating some warnings as (unacceptable) errors

BTW... and FWIW... notice that komisar himself used older versions of GCC for building his latest full-featured releases of x264:

Code:
[C:\]
=> k264 -V
x264 0.148.2705kMod 3f5ed56
(libswscale 4.1.100)
(libavformat 57.40.101)
(ffmpegsource 2.22.0.1)
built by Komisar on Jun 26 2016, gcc: 4.8.4 (multilib.generic.Komisar)
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later
and
Code:
[C:\]
=> k264x64 -V
x264 0.148.2705kMod 3f5ed56
(libswscale 4.1.100)
(libavformat 57.40.101)
(ffmpegsource 2.22.0.1)
built by Komisar on Jun 26 2016, gcc: 4.9.2 (multilib.generic.Komisar)
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

Last edited by filler56789; 4th September 2016 at 09:10. Reason: typo
filler56789 is offline   Reply With Quote
Old 4th September 2016, 17:24   #20  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Quote:
Originally Posted by filler56789 View Post
AFAIK, those "posix" builds from the MinGW-w64 site use winpthreads, which might explain why they don't solve your problem.
This may explain, indeed.

Quote:
Originally Posted by filler56789 View Post
Have you already discussed these issues with the "ffms2 people"?
I've tried once when i've begun to encounter similar issues, answer was roughly to use a "correct" gcc version.

Quote:
Originally Posted by filler56789 View Post
I can be wrong, but I still think the actual problem is not in GCC or in MSYS2, but in the current ffms2 code.
Maybe, this is out of my field, i'm just trying to keep being able to build my x264 versions.


Quote:
Originally Posted by filler56789 View Post
To make things clear(er): I suggested older versions of GCC as a possible (and acceptable) workaround, not as an actual solution...
Not possible for all. Even if gain is just a little, a little is still interesting when you launch several days of encoding, and i'm making my personnal builds targetting Broadwell CPU, option not avaible on too old versions of gcc.

But guys, for all your advises, informations, links, answers and time spend on them, a big

Last edited by jpsdr; 5th September 2016 at 16:55.
jpsdr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:56.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.