Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > (HD) DVD & Blu-ray authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th November 2019, 12:58   #81  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
In case anyone is wondering why there haven't been any binaries produced so far it's primarily because how long it has taken for me to produce a build pipeline.

I wanted to use dockcross and osxcross Docker containers but ran into all sorts of issues getting them set up to build the project. After a while I had to scrap the whole effort and start over.

I set things up in a VM to get Linux, Windows and MacOS builds all being produced successfully, I'm now transferring that setup over to a new, clean Docker container. Once that's all completed and tested I'll use GitHub Actions to get builds created automatically on each push.

It's taking a while I know but the upshot will make things a lot easier to manage going forward!
justdan96 is offline   Reply With Quote
Old 8th November 2019, 15:11   #82  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,393
Quote:
Originally Posted by justdan96 View Post
...
I set things up in a VM to get Linux, Windows and MacOS builds all being produced successfully, I'm now transferring that setup over to a new, clean Docker container. Once that's all completed and tested I'll use GitHub Actions to get builds created automatically on each push.
Have you checked that the Windows binaries that you've managed to build DO NOT depend on external libraries?

Namely, on winpthreads, seh and libstdc++.

Because when I use the winpthread-ed GCC from the MinGW-w64 project with MSYS2, the 64-bit .EXE is not linked statically to those libraries; and when I used the "toolchain" provided by pacman, even zlib could not be linked statically.

Last edited by filler56789; 8th November 2019 at 15:12. Reason: clarity
filler56789 is offline   Reply With Quote
Old 9th November 2019, 00:51   #83  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,445
-DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static -pthread" when configuring resolves it with vanilla MSys2 with default repo packages.

Comparison with ldd:
'plain' build without the additional linker flags:
Code:
$ ldd tsmuxer-plain.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7fff09500000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7fff09400000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7fff065f0000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7fff07910000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7fff072f0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7fff07350000)
        zlib1.dll => /mingw64/bin/zlib1.dll (0x62e80000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7fff08d10000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7fff07510000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7fff06490000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7fff07b40000)
        gdiplus.dll => /c/WINDOWS/WinSxS/amd64_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.18362.418_none_17b1a56c6d9fd02b/gdiplus.dll (0x7ffed5a00000)
        libgcc_s_seh-1.dll => /mingw64/bin/libgcc_s_seh-1.dll (0x61440000)
        libwinpthread-1.dll => /mingw64/bin/libwinpthread-1.dll (0x64940000)
        libstdc++-6.dll => /mingw64/bin/libstdc++-6.dll (0x6fc40000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x7fff089d0000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7fff08000000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x7fff06af0000)
With the linker flags given above:
Code:
$ ldd tsmuxer.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7fff09500000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7fff09400000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7fff065f0000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7fff07910000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7fff072f0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7fff07350000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7fff07510000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7fff06490000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7fff07b40000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7fff08d10000)
        gdiplus.dll => /c/WINDOWS/WinSxS/amd64_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.18362.418_none_17b1a56c6d9fd02b/gdiplus.dll (0x7ffed5a00000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x7fff089d0000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7fff08000000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x7fff06af0000)
The actual filesizes also corroborate this; the binary using the above linker flags is a whole 3MB larger (before stripping).
qyot27 is offline   Reply With Quote
Old 9th November 2019, 23:18   #84  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
You should be able to set the CMake flag "TSMUXER_STATIC_BUILD", which should generate a static build automatically.

For the Docker container specifically the Windows builds produced only use static dependencies and don't have any sort of external dependencies - I checked with Dependency Walker.

Last edited by justdan96; 10th November 2019 at 11:44. Reason: forgot to mention that is for Windows builds in Docker container
justdan96 is offline   Reply With Quote
Old 10th November 2019, 10:04   #85  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,445
Quote:
Originally Posted by justdan96 View Post
You should be able to set the CMake flag "TSMUXER_STATIC_BUILD", which should generate a static build automatically.
That'll work too. I didn't remember that CMakeLists for each subdir had been added.
qyot27 is offline   Reply With Quote
Old 10th November 2019, 18:23   #86  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,393
Quote:
Originally Posted by filler56789 View Post
I followed the instructions and managed to build the new tsMuxeR with the Win32-threaded GCC
I've just updated my GitHub repository accordingly:

https://github.com/filler56789/RV-tsMuxeR
filler56789 is offline   Reply With Quote
Old 16th November 2019, 20:38   #87  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
I've started creating the GitHub actions to create the builds for all 3 platforms. I haven't quite finished yet but if people could test the builds produced I'd really appreciate it!
https://github.com/justdan96/tsMuxer/actions

Just check the "Artifacts" on the latest run of "Build for Linux", "Build for Windows" and "Build for Mac".
justdan96 is offline   Reply With Quote
Old 16th November 2019, 21:11   #88  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,393
Quote:
Originally Posted by justdan96 View Post
https://github.com/justdan96/tsMuxer/actions

Just check the "Artifacts" on the latest run of "Build for Linux", "Build for Windows" and "Build for Mac".
Hummm, only the 64-bit .EXE, that's not good for the users of Windows XP

(I know, XP should be dead and buried, but don't forget tsMuxeR had been a 32-bit-only application since its zer0th day...)

P.S.: ¿¿¿ where is the new GUI ???
filler56789 is offline   Reply With Quote
Old 17th November 2019, 14:15   #89  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
Quote:
Originally Posted by filler56789 View Post
Hummm, only the 64-bit .EXE, that's not good for the users of Windows XP
If they are dedicated enough to use an operating system that went out of support over 5 years ago I'd like to think they are dedicated enough to compile it themselves

Quote:
P.S.: ¿¿¿ where is the new GUI ???
I haven't added Qt5 to the Docker container yet. For Windows it would be straightforward, Linux a bit more difficult as I'd need to compile Qt5 as static and Mac even more complicated as I'd have to compile Qt5 as static with osxcross.

Long story short, I'll get it sorted but it could take some time.

Last edited by justdan96; 17th November 2019 at 14:16. Reason: formatting error
justdan96 is offline   Reply With Quote
Old 17th November 2019, 18:27   #90  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,445
Quote:
Originally Posted by justdan96 View Post
If they are dedicated enough to use an operating system that went out of support over 5 years ago I'd like to think they are dedicated enough to compile it themselves
There are modern-ish platforms that use 32-bit Windows 10. Kind of but not really niche things like the Quantum Byte/Byte Plus or other mini-PCs. A lot of them (like the Byte3) have since moved to 64-bit Windows 10 or Ubuntu or leave OS installation to the user, but the first couple generations from ~2015-2017 used 32-bit Windows and 32-bit UEFI.

That means either having a separate i686-w64-mingw32 toolchain that needs to be invoked, or the existing toolchain needs to be multilib-enabled (in which case it's just a matter of throwing in some -m32 flags and telling it where the /lib directory is for i686, sometimes it might require overriding the CMAKE_SYSROOT, but not often).

What would be a showstopper specifically for XP is if MXE builds its toolchain with Secure API support turned on in MinGW-w64, as that will restrict everything it builds to Vista+.
qyot27 is offline   Reply With Quote
Old 17th November 2019, 19:22   #91  |  Link
a5180007
Registered User
 
Join Date: Jun 2014
Location: Marrakech, Morocco
Posts: 253
Bug with aac

Hi all,

Another bug for aac muxing. In brief, the aac frame size is written on 13 bits (bits 31-43 in the frame), however tsMuxeR takes into account only the first 11 bits. So when the aac frame size is above 2048 bytes, tsMuxeR incorrectly reports a bad frame and discards it.

In aac.cpp, replace line 52:
Code:
int AACCodec::getFrameSize(uint8_t* buffer)
{
	return buffer[4]*8 + (buffer[5] >> 5);
}
with
Code:
int AACCodec::getFrameSize(uint8_t* buffer)
{
	return ((buffer[3] & 0x03) << 11) + (buffer[4] << 3) + (buffer[5] >> 5);
}

Last edited by a5180007; 17th November 2019 at 19:49.
a5180007 is offline   Reply With Quote
Old 17th November 2019, 19:57   #92  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,248
Quote:
Originally Posted by a5180007 View Post
Another bug for aac muxing.
LC-AAC or HE-AAC?
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 17th November 2019, 21:13   #93  |  Link
a5180007
Registered User
 
Join Date: Jun 2014
Location: Marrakech, Morocco
Posts: 253
Quote:
Originally Posted by SeeMoreDigital View Post
LC-AAC or HE-AAC?
All AAC types.
Edit: I've just tried and I can confirm, tsMuxeR accepts LC and HE and the above fixes both.

Last edited by a5180007; 17th November 2019 at 21:43.
a5180007 is offline   Reply With Quote
Old 18th November 2019, 00:20   #94  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
Quote:
Originally Posted by qyot27 View Post
That means either having a separate i686-w64-mingw32 toolchain that needs to be invoked, or the existing toolchain needs to be multilib-enabled (in which case it's just a matter of throwing in some -m32 flags and telling it where the /lib directory is for i686, sometimes it might require overriding the CMAKE_SYSROOT, but not often).
I suppose I can look at making a CMake option for it, if the demand is there.

Also @a5180007 thanks for the bug reports, I'll get those created in GitHub as issues so we can take a closer look at them.

Additionally I've taken a close look at tsMuxerGUI and there aren't any actual code dependencies on tsMuxer - so a drop-in replacement could be created. Qt is pretty heavyweight for a simple application. My preference would be an application that can be made cross-platform with ease that doesn't depend on large GUI frameworks. I don't think we can swap out Qt for something like Nuklear that easily within the existing application so it may even be easier to rewrite it (if someone out there can prove me wrong I'd be very thankful!).

I'll create an issue for it so we can track it.

Last edited by justdan96; 18th November 2019 at 17:30. Reason: mention bug reports + GUI
justdan96 is offline   Reply With Quote
Old 22nd November 2019, 17:20   #95  |  Link
a5180007
Registered User
 
Join Date: Jun 2014
Location: Marrakech, Morocco
Posts: 253
Also tsMuxeR cannot demux correctly mp4 aac 5.1 : the problem seems to be linked to a bug from ffmpeg. tsMuxeR takes the channel count from 'mp4a' atom (cf. movDemuxer.cpp line 1252), but as noted here ffmpeg sets the channel count to 2 in 'mp4a' atom even for aac 5.1, and the correct channel count has to be taken from 'esds' atom.

The following should correct the bug, could anybody please test and report:
In movDemuxer.cpp add line 186 as follows:
Code:
			if (isAAC) {
				m_aacRaw.m_channels = m_sc->channels; // added line
				m_aacRaw.buildADTSHeader(dst, frameSize + AAC_HEADER_LEN);
				memcpy(dst + AAC_HEADER_LEN, buff, frameSize);
				dst += frameSize + AAC_HEADER_LEN;
			}
and add line 1454 as follows:
Code:
			if (st->parsed_priv_data) {
				((MovParsedAudioTrackData*)st->parsed_priv_data)->isAAC = true;
				st->parsed_priv_data->setPrivData(st->codec_priv, st->codec_priv_size);
				st->channels = (st->codec_priv[1] >> 3) & 0x0f; // added line: copy channel count from 'esds' AudioSpecificConfig
			}
a5180007 is offline   Reply With Quote
Old 22nd November 2019, 17:34   #96  |  Link
a5180007
Registered User
 
Join Date: Jun 2014
Location: Marrakech, Morocco
Posts: 253
Quote:
Originally Posted by a5180007 View Post
I have corrected the bug on my side by simply changing the line 7 in avPacket.h from:
Code:
const static int MAX_AV_PACKET_SIZE = 32768;
to
Code:
const static int MAX_AV_PACKET_SIZE = 800000;
Ok, so I have tried to get totally rid of the max frame size limitation, as HEVC UHD frames can be several MBs.

The following should do the trick, could anybody please test and report:

In mpegStreamReader.cpp at line 130:
Code:
	//if (m_bufEnd - m_curPos < MAX_AV_PACKET_SIZE) { // remove condition : size of frame is now checked for all frames
		uint8_t* nextNal = NALUnit::findNALWithStartCode(min(m_curPos + 3,m_bufEnd), m_bufEnd, m_longCodesAllowed);
		if (nextNal == m_bufEnd) {
			storeBufferRest();
			return NEED_MORE_DATA;
		}
	//}
and at line 65, remove truncating of last frame:
Code:
			avPacket.size = m_tmpBufferLen; //changed from min(MAX_AV_PACKET_SIZE, m_tmpBufferLen);
        }
		/*if (m_tmpBufferLen > MAX_AV_PACKET_SIZE) {
			LTRACE(LT_ERROR, 2, "Too large last buffer (" << m_tmpBufferLen << " bytes). Truncate buffer to " << 
				   MAX_AV_PACKET_SIZE << " bytes. It does not have to be!");
		} */
With this fix the MAX_AV_PACKET_SIZE can be left = 32768 (although I don't know where this value comes from...)

Last edited by a5180007; 22nd November 2019 at 17:49.
a5180007 is offline   Reply With Quote
Old 24th November 2019, 18:09   #97  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
Thank you so much a5180007, I'll test these fixes myself and if they are resolved I'll publish the test cases as well.
justdan96 is offline   Reply With Quote
Old 24th November 2019, 21:21   #98  |  Link
FilipeAmadeuO
Registered User
 
Join Date: Aug 2002
Location: Portugal
Posts: 340
Quote:
Originally Posted by justdan96 View Post
Thank you so much a5180007, I'll test these fixes myself and if they are resolved I'll publish the test cases as well.
Can you publish new binaries here for testing ?
Any plan to build with GUI ?
FilipeAmadeuO is offline   Reply With Quote
Old 25th November 2019, 17:19   #99  |  Link
Hart2hart
Registered User
 
Join Date: Aug 2016
Posts: 12
To go back a step and sorry if I missed it...

Were binaries produced that would be “like” V2.6.12 so we can compare results to the long exiting binary files. This would provide a great starting point for enhancements to 3D and HEVC 4K images.

Excellent that TSMuxeR has been released to community. I’ll jump in at some point.
Hart2hart is offline   Reply With Quote
Old 26th November 2019, 19:41   #100  |  Link
justdan96
Registered User
 
Join Date: Jun 2019
Location: UK
Posts: 49
Sorry I didn't update fully here, here is the quote from the readme:

Quote:
To download the pre-compiled binaries head over to the following URL (you must be logged in) https://github.com/justdan96/tsMuxer/actions and click on the latest action for your chosen platform (Windows, Mac or Linux). In the Artifacts section in the top-right you can download a ZIP file containing the executable for tsMuxer.

At the moment only the CLI is included, the GUI is not yet available.
We've had quite a few good enhancements and bug fixes go into the repo recently so I think we are close to a release of 2.6.16. I want to get a nice automated workflow for creating the releases in GitHub so it may take a while for me to develop it.

Yes the binaries will be similar to 2.6.12, you can test the CLI now and use the older GUI if you can't (or don't want to) compile it yourself. Once I get the automated builds in GitHub working with tsMuxerGUI as well you can use the official binaries for that as well.

It would be fantastic if people can compare the older versions to the latest, you can post the results to this forum thread or if you encounter a specific issue with what you think is the root cause you can create an issue on GitHub.
justdan96 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.