Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Programming and Hacking > Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th July 2019, 17:40   #1  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Issue with GCC 9.1.0 and ffms2 and x264

I don't know exactly where to put the information (here or in h264), so for now i put it here.

I build :
- ffmpeg 4.1.4.
- ffms2 (last commit a9e8f73 of 19/03/2019.)
- lsmash (last git commit 88208b5 of 25/02/2019.)
and after, i build x264.

Under MSYS2, with GCC provided by msys2 (get with "pacman") :
With version 7.4.0 Rev1 (x86) and 8.3.0 Rev2 (x64) ffms2 is included and checked/tested "Ok" by the configure script of x264.

With version 9.1.0 Rev3 (x86/x64), ffms2 is not included, it seems that the check/test by the configure script fails.

The code and the building scripts are of course exactly the same in all cases.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 30th July 2019, 19:04   #2  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Have you looked into config.log why it fails? Or at least pastebin it somewhere so we can look it also.
MasterNobody is offline   Reply With Quote
Old 31st July 2019, 16:32   #3  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Pastebin config.log :
Fail with gcc 9.1.0 Rev3 (x86) here.
Success with gcc 7.4.0 Rev1 (x86) here.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 31st July 2019, 19:20   #4  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Looks like problem is with pkg-config. Can you also pastebin generated ffms2.pc files?
MasterNobody is offline   Reply With Quote
Old 31st July 2019, 20:09   #5  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
ffms2.pc created by :
gcc 9.1.0 (fail) here.
gcc 8.1.0 (success) here.
Except directory wich is different (but it's normal as i use 8.1.0 for the winthread build, and 9.1.0 for posix build), i don't see any difference.
Didn't keep the 7.4.0 posix, but if realy needed, i can do it (it's just that i'll have to redo all the ffmpeg and ffms2 build).
__________________
My github.

Last edited by jpsdr; 31st July 2019 at 20:15.
jpsdr is offline   Reply With Quote
Old 31st July 2019, 20:16   #6  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Hm, looks same. Can you check what
Code:
pkg-config --libs ffms2
and
Code:
pkg-config --cflags ffms2
returns for both of them (gcc 9.1.0/gcc 8.1.0)?
MasterNobody is offline   Reply With Quote
Old 1st August 2019, 17:23   #7  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
gcc 9.1.0 :
-LG:/msys32/local-x264_x86/posix/t_mod/lib -lffms2

-IG:/msys32/local-x264_x86/posix/t_mod/include

gcc 8.1.0 :
-L/local-x264_x86/posix/t_mod/lib -LG:/msys32/mingw32/lib -L/local-x264_x86/posix/t_mod/lib -LG:/msys32/mingw32/lib -L/local-x264_x86/posix/t_mod/lib -lz -lffms2 -lavformat -lm -lbz2 -lz -lavcodec -pthread -liconv -lm -llzma -lz -lswscale -lm -lswresample -lm -lavutil -pthread -lm -luser32 -lbcrypt

-I/local-x264_x86/posix/t_mod/include

gcc 7.4.0 :
-L/local-x264_x86/posix/t_mod/lib -LG:/msys32/mingw32/lib -L/local-x264_x86/posix/t_mod/lib -LG:/msys32/mingw32/lib -L/local-x264_x86/posix/t_mod/lib -lz -lffms2 -lavformat -lm -lbz2 -lz -lavcodec -pthread -liconv -lm -llzma -lz -lswscale -lm -lswresample -lm -lavutil -pthread -lm -luser32 -lbcrypt

-I/local-x264_x86/posix/t_mod/include

For the three test cases, i've configured PKG_CONFIG_PATH with the exact same location.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 1st August 2019, 19:16   #8  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by jpsdr View Post
For the three test cases, i've configured PKG_CONFIG_PATH with the exact same location.
If PKG_CONFIG_PATH was the same than what was changed? pkg-config executable? Than it is the culprit.
MasterNobody is offline   Reply With Quote
Old 1st August 2019, 19:45   #9  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
For now, it seems.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 12th August 2019, 09:45   #10  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Odds ! "pkg-config --version" returns 0.29.2 in all cases, when "pkg-config --libs ffms2" returns different results for the exact same PKG_CONFIG_PATH, and so for also the exact same ffms2.pc file.

But... I've noticed that there is two pkg-config.exe files.
One in "mingw32/bin" and another in "usr/bin".

But, if i go in the directory where there is pkg-config.exe and ask the version :
"usr/bin" returns version 0.29.2.
GCC 9.1.0 "mingw32/bin" returns 0.29.2.
GCC 8.1.0 "mingw32/bin" returns 0.29.2.

Ok, i'm totaly
__________________
My github.
jpsdr is offline   Reply With Quote
Old 12th August 2019, 13:58   #11  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Quote:
Originally Posted by jpsdr View Post
Odds ! "pkg-config --version" returns 0.29.2 in all cases, when "pkg-config --libs ffms2" returns different results for the exact same PKG_CONFIG_PATH, and so for also the exact same ffms2.pc file.

But... I've noticed that there is two pkg-config.exe files.
One in "mingw32/bin" and another in "usr/bin".


But, if i go in the directory where there is pkg-config.exe and ask the version :
"usr/bin" returns version 0.29.2.
GCC 9.1.0 "mingw32/bin" returns 0.29.2.
GCC 8.1.0 "mingw32/bin" returns 0.29.2.

Ok, i'm totaly
It seems the MSYS2 devilopers have skrewed royally their latest MSYS2 releases... this is the reason why I returned to their builds from the year 2016 and decided to NEVER AGAIN invoke the command "pacman -Syuu".

Last edited by filler56789; 12th August 2019 at 14:00.
filler56789 is offline   Reply With Quote
Old 12th August 2019, 14:53   #12  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Quote:
Originally Posted by HolyWu View Post
You're really spreading FUD.

The reason why jpsdr has two pkg-config.exe in different directory is that he installed both mingw-w64-i686-pkg-config and pkg-config packages. They don't conflict with each other. Which one will get executed depends on the shell type. msys_shell -mingw32 will execute /mingw32/bin/pkg-config and msys_shell -msys will execute /usr/bin/pkg-config.
Thanks for the detailed info.
Anyway it won't be easy to make me change my mind.
Last week I was playing around with "the latest and greatest" MSYS2 and found out that the mingw32 and mingw64 directories were full of "strange" and "garbage" stuff — in other words, full of files that were not supposed to be there...

Last edited by filler56789; 12th August 2019 at 14:56. Reason: better wording
filler56789 is offline   Reply With Quote
Old 13th August 2019, 09:16   #13  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Building ffms2 has always been tricky.

To build x264 with ffms2, you need a little modification in ffms.h (thanks Komisar).

About GCC, for posix thread model, i use the one provided with MSYS2 (get with pacman).
For win32 thread model, i use one provided here : https://sourceforge.net/projects/min.../mingw-builds/

When you build ffms2, there is first a lot of tests done, and if they all pass Ok, the build begins.

If i build ffms2 with the MSYS2 posix GCC version, the build goes to the end properly, and finish Ok. (*)
If i try to build ffms2 with a posix GCC version from the link above, the build fails at the end during the link of the EXE file. With this version, the build never ends properly. But, except the exe file, all the others files (libraries) are build properly.

To build ffms2 with a win32 thread model GCC it's lot tricky. You need a not so trivial patch (at least for me), including putting back numthreads files, again, thanks Komisar.
Now, after patched, if you try to build ffms2 with a win32 GCC version from the link above, the build fails at the end during the link of the EXE file. But, except the exe file, all the others files (libraries) are build properly. You have to install the files manualy.

(*) : Well... That was true until around a year ago. Now, even with the MSYS2 posix GCC, the build fails at the end during the link of the exe. For now, personnaly, i don't know a GCC version which can build ffms2... It's not a big deal, as i don't need the EXE files, and all the other stuff i need are made properly (libraries, .pc file, etc...).

Unlike ffmpeg where the is a --disable-programs option, practical when you just want to build/install the libraries and don't need the EXE files, i've not been able to find such an option for ffms2.
It would be very practical, because it would prevent the build to fail at the end, and allow make install to install...

Out of curiosity, i took the GCC provided by filler56789. Big and good surprise, it was a win32 thread model.
So i build ffmpeg, build ffms2... no... This version unfortunately doesn't even begin to build, it fails during the preliminary tests...
Edit :
It fails with the following :
Code:
checking whether the i686-w64-mingw32-g++ linker (g:/msys32/mingw32/i686-w64-mingw32/bin/ld.exe) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking zlib.h usability... no
checking zlib.h presence... no
checking for zlib.h... no
checking for inflateEnd in -lz... no
configure: error: in `/g/x264/ffms2/Build_x264':
configure: error: cannot locate zlib.h and -lz, specify a valid zlib installation using --with-zlib=DIR
See `config.log' for more details
make: *** Pas de cible spécifiée et aucun makefile n'a été trouvé. Arrêt.
make: *** Aucune règle pour fabriquer la cible «*install*». Arrêt.
And if i run pacman -S --needed zlib it says that zlib is allready installed and up to date...
Another ffms2 mystery of why it fails here with this GCC version, and not with the others GCC version.
Tested with 8.3.0 and 9.2.0, same results, fails here on the zlib check...
__________________
My github.

Last edited by jpsdr; 13th August 2019 at 12:41.
jpsdr is offline   Reply With Quote
Old 13th August 2019, 10:47   #14  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Just tested the fresh new 9.2.0 MSYS2 GCC version, still no ffms2 when building x264...

I begin to wonder if it's not a GCC 9.x issue...
__________________
My github.

Last edited by jpsdr; 13th August 2019 at 10:52.
jpsdr is offline   Reply With Quote
Old 13th August 2019, 14:10   #15  |  Link
Wolfberry
Helenium(Easter)
 
Wolfberry's Avatar
 
Join Date: Aug 2017
Location: Hsinchu, Taiwan
Posts: 99
Your issue sounds like a broken environment.

Quote:
And if i run pacman -S --needed zlib it says that zlib is allready installed and up to date...
You want mingw-w64-i686-zlib (for 32bit) or mingw-w64-x86_64-zlib (for 64bit), if you are compiling native binaries for windows.

I build ffms2 using the meson build system from HolyWu: https://github.com/FFMS/ffms2/pull/338, and adds -DFFMS_STATIC to Cflags in ffms2.pc

No need for any patches from Komisar.
__________________
Monochrome Anomaly
Wolfberry is offline   Reply With Quote
Old 13th August 2019, 16:09   #16  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Tested to run pacman -S --needed mingw-w64-i686-zlib but it says allready installed and up to date.
Issue is that swapping gcc version by only renaming directories. Once zlib is installed, it seems it doesn't check in the directory if it exist, but it checks in a database if it's installed.
Finaly i think, when i swap gcc version renaming directories, even if i swap to a directory without zlib, for the environment zlib is installed.
The gcc versions i usualy use are probably provided with zlib allready installed.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 14th August 2019, 07:20   #17  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
Maybe off-topic but surely necessary:

Quote:
pacman -S --needed mingw-w64-i686-zlib
Packages named mingw-w64-* should not even exist to begin with, since the (supposed) goal of MSYS2 is emulate a unix(oid) building-environment on Windows as much as possible.

And especially when they are placed inside the /mingw##/bin/ directories.

</END_OF_RANT>

Last edited by filler56789; 14th August 2019 at 07:24. Reason: remove emoji — old vBulletin suxxx.
filler56789 is offline   Reply With Quote
Old 14th August 2019, 07:38   #18  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Quote:
Originally Posted by jpsdr View Post
Tested to run pacman -S --needed mingw-w64-i686-zlib but it says allready installed and up to date.
Issue is that swapping gcc version by only renaming directories. Once zlib is installed, it seems it doesn't check in the directory if it exist, but it checks in a database if it's installed.
Finaly i think, when i swap gcc version renaming directories, even if i swap to a directory without zlib, for the environment zlib is installed.
The gcc versions i usualy use are probably provided with zlib allready installed.
Clean Pacman cache (pacman -Scc) and reinstall zlib.

I'm preferring this GCC for MSYS2 - https://gcc-mcf.lhmouse.com/ ; https://github.com/lhmouse/mcfgthread/wiki ; https://github.com/lhmouse/mcfgthread/wiki/Benchmarking
StvG is offline   Reply With Quote
Old 14th August 2019, 08:47   #19  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Tested with pacman -S --overwrite --needed mingw-w64-i686-zlib (as --force is deprecated). I've seen the zlib stuff appears in the mingw32 directory of filler56789 GCC version. Unfortunately, ffms2 build with it is still failling on the zlib test.
I'll stop here, it's not like if i haven't allready others GCC versions working.
I'll take a look at StvG links, thanks for these.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 14th August 2019, 10:02   #20  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,308
Tested ffms2 integration in x264 with the 8.3.1 from the link provided by StvG. ffms2 builds, but unfortunately, the integration in x264 fails.
The pkg-config --libs ffms2 returns the same value than 9.1.0 (or 9.2.0) MSYS2 GCC (see previous post).
Can't test a 8.1.x because these GCC versions begin with 8.3.1.

As far as i know, it doesn't exist mcf thread build option for ffmpeg, ffms2 or x264, only posix or win32threads. What is the closets ? My guess would be Win32threads, but...
__________________
My github.

Last edited by jpsdr; 14th August 2019 at 10:06.
jpsdr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.