Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd February 2019, 14:43   #301  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
No. Whenever I have free time and feel motivated to tackle it. Thing is, it isn't a trivial change.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 23rd February 2019, 20:31   #302  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
A warning, as people still seem to be posting directly to threads and may not be aware:

It appears that the forum may have been hacked. There is a suspicious "test" announcement, apparently from tebasuna51 (but probably not), parts of the forum are not working, and there appear to be some malicious javascript files.

I'm not speaking in any official capacity here, but I would recommend, at the very least, NOT entering your password anywhere on Doom9 for the time being.

Please refer to this post: https://forum.doom9.org/showthread.p...wpost&t=176128
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 26th February 2019, 19:52   #303  |  Link
LeMoi
Registered User
 
Join Date: Sep 2004
Location: France
Posts: 367
Sometimes I make a mistake when I encode a video and I forget to crop, I just realize after I mux.
So I do a reencode of the video with the proper crop settings, I remove the muxing job from the queue and re-edit it in MKVToolNix GUI, but it doesn't reload the properties of the files, and if I just remux the files, the ratio of the muxed file is wrong... (ie old 16/9 instead of new 2.35). I have to remove the video track from the list and re-add it to get the new properties.
Is it possible that, when I edit a job and remove it from the queue, MKVToolNix GUI re-checks the properties of the files? while keeping the settings like delay, track name, etc.?
LeMoi is offline   Reply With Quote
Old 26th February 2019, 20:23   #304  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by LeMoi View Post
Is it possible that, when I edit a job and remove it from the queue, MKVToolNix GUI re-checks the properties of the files? while keeping the settings like delay, track name, etc.?
No, that's not possible at the moment, and it would require quite a lot of work to implement. Not something I'm willing to spend time on.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 27th February 2019, 08:49   #305  |  Link
LeMoi
Registered User
 
Join Date: Sep 2004
Location: France
Posts: 367
OK, no problem, you're the programer, I thought it would be easier to reload the files, I'll wait until you change your mind
LeMoi is offline   Reply With Quote
Old 8th March 2019, 06:33   #306  |  Link
Wakaku
MKV and tools user
 
Join Date: Apr 2014
Posts: 43
Thank you for implementing "emoji" support. No more "black diamonds with white question marks". All areas seems covered (eg. input of/output to emoji filenames, Header editor tags, retaining XML tags, covers). Thus such metadata can now be preserved. Output MKV is playable without problems in MPC-HC, MPC-BE, PotPlayer, KMPlayer, VLC.

Last edited by Wakaku; 8th March 2019 at 06:40.
Wakaku is offline   Reply With Quote
Old 12th March 2019, 23:05   #307  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
MKVToolNix v32.0.0 released

Hey,

Here's MKVToolNix v32.0.0, a really small bug fix release, the most important probably being the handling of Unicode code points > U+ffff (e.g. Emojis). For that to work a bug-fixed libEBML is also needed, which is why libEBML v1.3.7 is now required. It was released earlier today.

Other than that nothing has changed for package managers since v31.0.0.

Here are the usual links: the MKVToolNix home page, the Windows installer/portable version & macOS DMG & Linux AppImage and the source code.

The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are still being built and will be available over the course of the next couple of hours.

Here are the NEWS since the previous release:

Quote:
# Version 32.0.0 "Astral Progressions" 2019-03-12

## New features and enhancements

* mkvinfo: when sizes are output the size of the element's data portion is output in addition to the element's total size.
* MKVToolNix GUI: info tool: the element's data portion is shown as an extra column.
* MKVToolNix GUI: multiplexer: added column "Delay" to the track list containing the additional delay to apply during multiplexing. Implements #2506.

## Bug fixes

* all: fixed handling of Unicode code points > U+FFFF. Fixes #2516.
* mkvmerge: Windows: mkvmerge was crashing with an exception when trying to identify certain files that can be used on Blu-rays (such as MPEG transport streams of MPLS play list files) and when the file name was given as a UNC path (e.g. `\\servername\sharename\path\to\file.m2ts`). The GUI emitted errors such as "the JSON output could not be parsed" in that case. Fixes #2507.
* MKVToolNix GUI: the portable mode wasn't detected correctly when the current working directory the GUI was started from wasn't the directory the GUI's executable file was located it. Examples for when this is the case are Windows' "send to" or "open with" functions. Fixes #2501.
* MKVToolNix GUI: multiplexer: using button to change the current destination directory to one of the recently used ones did not update the file name according to the "make file name unique" setting. Part of the fix of #2519.
* MKVToolNix GUI: multiplexer: the function "set destination file name from selected file's name" will now only change the destination file's name but not its path. Part of the fix of #2519.

## Build system changes

* libEBML v1.3.7 and libMatroska 1.5.0 are now required as they fix their handling of Unicode code points > U+FFFF (see #2516).
Have fun
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 12th March 2019, 23:40   #308  |  Link
hubblec4
Matroska find' ich toll
 
Join Date: Apr 2008
Posts: 1,370
Thanks a lot.
hubblec4 is offline   Reply With Quote
Old 20th March 2019, 21:13   #309  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
Slow in command line mode

Hi,

I'm tying to use mkvmerge in command line mode, inside some scripts I wrote to automatize remuxing MKV files originating from Blu-rays.

I'm working with an overclocked 6 core cpu on Windows 7, and on a RAID system that can read and write at around 500 Mbs.

When I use MKVtoolnixG, Processhacker shows that remuxing runs at about 180MB/s, which is much faster than the drive speed, and remuxing a 60GB file is pretty fast.

If I display the command line, and cut and paste it into a DOS windows, and run it manually, I get barely 25MB/s, and muxing if much much slower.

In both cases, it's exactly the same command running, but it's much faster from the gui.

Why, and how can I get back the full speed from the command line?
Attached Images
  
robena is offline   Reply With Quote
Old 20th March 2019, 21:24   #310  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by robena View Post
… remuxing runs at about 180MB/s, which is much faster than the drive speed …
I'm pretty sure you're experiencing the nice effects of caching. If you really want to make speed comparisons, you'll have to re-run the same test multiple times in sequence and discard the first run (e.g. in the GUI add the same job three times and then let the whole queue run to finish; only take the time of the second & third run).

The most important thing is to flush all the caches to disk after completing one test and before moving on to the next. Otherwise the OS will still be busy writing data from the first test to disk while you're already running the second test. That will severely skew the results in favor of the first test, obviously.

In fact, real testing should look ideally look something like this:
  1. Flush all cached data to disk
  2. Run the test and…
  3. Flush all cached data to disk directly afterwards

I cannot tell you how to do 1, but mkvmerge itself can do steps 2 and 3 if you add the command line option "--flush-on-close". In that case the time mkvmerge reports will be the real duration of one round of multiplexing. Of course you must add that option both in the GUI as well as on the command line; both tests must use identical options.

Again, the test is worthless if 1. (flushing before muxing) isn't done.

In the end using the GUI and using mkvmerge directly cannot produce different results as the GUI does nothing more than starting mkvmerge with the command-line parameters you can see via "Multiplexer" → "Show command line". There's no secret option, no additional setup the GUI does.

Benchmarking is real hard.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 20th March 2019, 22:53   #311  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
Quote:
Originally Posted by Mosu View Post
In the end using the GUI and using mkvmerge directly cannot produce different results as the GUI does nothing more than starting mkvmerge with the command-line parameters you can see via "Multiplexer" → "Show command line". There's no secret option, no additional setup the GUI does.

Benchmarking is real hard.
That's a very logical answer, but unfortunately it's not a caching problem.

Muxing with the GUI is always at least 5 times faster, whether I start it first or second.

I did multiple tests like:

1) command line
2) GUI
3) command line
4) GUI

and the result are consistent every time, the GUI is much faster.

Besides, these are UHD very large 60GB files, so memory caching would be filled up very fast.

When I copy a 60GB file from my RAID to a SSD, I see first (sometimes) 1.5Gbs speed, but after a few gigabits, the caching ends and it goes down to the real disk speed.

With the GUI, the whole 60GB run at the same fast speed from beginning to end.

With the command line cust & pasted, the whole 60GB run at the same very slow speed, even if I run it after the GUI run. I just redid the GUI first, command line second second test, and the command line was easily 5 times slower.

Edit: I used the sysinternals "sync -r" command to flush all the drives before each run, same results.

Here is the command line:

Code:
"C:/Program Files/MKVToolNix\mkvmerge.exe" --ui-language en --output ^"B:\prob\test3\Lord of War UHD ^(1^).mkv^" --no-subtitles --language 0:eng --track-name ^"0:MPEG-H HEVC Video / 71269 kbps / 2160p / 23.976 fps / 16:9 / Main 10 @ Level 5.1 @ High / 10 bits / HDR10 / BT.2020^" --default-track 0:yes --language 1:eng --track-name ^"1:Dolby TrueHD/Atmos Audio / 7.1 / 48 kHz / 4750 kbps / 24-bit^" --default-track 1:yes --language 2:eng --track-name 2:AC3 ^"^(^" ^"B:\prob\test3\Lord of War UHD.mkv^" ^"^)^" --language 0:und ^"^(^" ^"B:\prob\test3\Lord of War UHD.sup^" ^"^)^" --language 0:und ^"^(^" ^"B:\prob\test3\Lord of War UHD-grey.sup^" ^"^)^" --title ^"Lord of War ^(2005^)^" --flush-on-close --track-order 0:0,0:1,0:2,1:0,2:0
I am at a complete loss to explain that.

Last edited by robena; 20th March 2019 at 23:21.
robena is offline   Reply With Quote
Old 20th March 2019, 23:23   #312  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Are you writing to the same file in each iteration? If so you're likely incurring a lot of time from simply deleting the existing huge output file. That's a rather slow operation on NTFS.

I can reproduce your results somewhat, but not if each call to mkvmerge writes to a different file that doesn't exist yet.

So… add "remove output file, flush all of Windows' caching (e.g. by rebooting)" to your list of things to do before each run.

Like I said, proper benchmarking is hard.

Edit: just for kicks: the simple act of deleting ~25 GB in two files on my NTFS SDD takes 1m24s (that's deleting via "del" in cmd.exe or via the Explorer — mkvmerge does something similar: it opens the file in write & truncate mode, meaning it will be cut to 0 bytes of size when opening it which practically does the same as deleting it: all blocks allocated for the file are freed at that moment, and that takes a lot of time on NTFS). That's rather significant given that multiplexing _to_ that file took (3m 42s).
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.

Last edited by Mosu; 20th March 2019 at 23:29.
Mosu is offline   Reply With Quote
Old 20th March 2019, 23:39   #313  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
Quote:
Originally Posted by Mosu View Post
Are you writing to the same file in each iteration? If so you're likely incurring a lot of time from simply deleting the existing huge output file. That's a rather slow operation on NTFS.

I can reproduce your results somewhat, but not if each call to mkvmerge writes to a different file that doesn't exist yet.

So… add "remove output file, flush all of Windows' caching (e.g. by rebooting)" to your list of things to do before each run.

Like I said, proper benchmarking is hard.
I did both, either writing the same file after deleting, or letting the GUI increasing the (N) number.

You probably did not see my edit while writing your answer, I found out how to flush all the drives without rebooting with the sync command.

Also, aside these specific tests, I made scores of command line remuxes that were all always extremely slow, and every time I try the GUI, it's 5 time faster. At first I though I was writing poorly my scripts (which I did with the doc, not by looking at the command line in thr GUI), and rewrote them to have the exact same syntax as the GUI. It did not help, and I think that the use of the sync command and of different file names rules out caching.
I also tried to read on one drive and write on another, with exactly the same difference in speed. My RAID is very good at parallel accesses (unlike regular hard drives), I barely notice a difference when using the 2 drives strategy.

Anyway, thanks for spending time on that subject.
robena is offline   Reply With Quote
Old 20th March 2019, 23:56   #314  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
Another thing: this seems to happen only with UHD HEVC video files.
I did not notice my scripts being slow with 50GB regular HD AVC files. Even weirder!
robena is offline   Reply With Quote
Old 22nd March 2019, 07:54   #315  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
So, nobody has an idea why this is happening, and how to correct it?
robena is offline   Reply With Quote
Old 23rd March 2019, 19:20   #316  |  Link
Dulus_No
Registered User
 
Join Date: Sep 2016
Posts: 10
Hi!
I've encountered a strange behavior of SPS/PPS duplication when merging and extracting an avc stream.
Used commands:
Code:
mkvmerge -o ldaQLGOoJW0_1.mkv ldaQLGOoJW0.avc
mkvextract ldaQLGOoJW0_1.mkv tracks 0:ldaQLGOoJW0_1.avc
mkvmerge -o ldaQLGOoJW0_2.mkv ldaQLGOoJW0_1.avc
mkvextract ldaQLGOoJW0_2.mkv tracks 0:ldaQLGOoJW0_2.avc
mkvmerge -o ldaQLGOoJW0_3.mkv ldaQLGOoJW0_2.avc
mkvextract ldaQLGOoJW0_3.mkv tracks 0:ldaQLGOoJW0_3.avc
H.264 analysis: https://imgur.com/a/rPWNZdm
7z archive with source and result files: https://mega.nz/#!J1hASKpA!VLINH3Ie2...8q9fbA3xiOBZGI
Dulus_No is offline   Reply With Quote
Old 23rd March 2019, 19:29   #317  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Yeah, my tools are definitely not trying to deduplicate SPS & PPS (or VPS for HEVC). Instead, they try to achieve best playback compatibility (including seeking to arbitrary places) while not complicating the code too much.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 23rd March 2019, 19:37   #318  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by robena View Post
So, nobody has an idea why this is happening, and how to correct it?
One last idea: when starting mkvmerge manually, do you perchance start it with a lower process priority (e.g. because the process you start it from, such as cmd.exe, is already running with a lower process priority)? Or do you start the GUI (and therefore mkvmerge-started-from-the-GUI) with a higher one? See Windows 10's task manager's "detail" page, right-click on the column header & enable the "base priority" column.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 23rd March 2019, 21:14   #319  |  Link
Perenista
Registered User
 
Join Date: Oct 2013
Posts: 205
It's odd. I tried to add a dubbing to a Matroska video and told MKVToolnix to add + 7000 ms as delay, to sync said dubbing audio (AC3) with the video. This time didn't work. However when I open MPC-HC (and both files at the same time) and add 7000 ms manually the audio matches. Now it seems 7000 ms is too much because the audio is still 0.5 second or 1 second out of sync.

(I know the best way to do this is adding these +7 seconds in the dubbing audio and then adding to the MKV without any delay option enabled in MKVToolnix. The problem is that I might have to do this with subtitle streams, too, so it would be a hassle if this didn't work in all cases.

Maybe this time didn't work because 7 seconds is too much?
Perenista is offline   Reply With Quote
Old 23rd March 2019, 21:19   #320  |  Link
robena
Registered User
 
Join Date: May 2007
Posts: 29
Quote:
Originally Posted by Mosu View Post
One last idea: when starting mkvmerge manually, do you perchance start it with a lower process priority (e.g. because the process you start it from, such as cmd.exe, is already running with a lower process priority)? Or do you start the GUI (and therefore mkvmerge-started-from-the-GUI) with a higher one? See Windows 10's task manager's "detail" page, right-click on the column header & enable the "base priority" column.
Thanks, I already tried that. I also verified the processor affinity.

I start it 2 ways: normally using a REXX interpreter that I wrote (my scripting language of choice, having worked with an IBM mainframe in the 80s) which does a direct CreateProcess() call without using the Windows cmd.exe. My interpreter has a routine allowing to set the priority, and it changes nothing. I integrated the calls into the shell, so I just have to right click on a file and choose the aspect ratio to create sup files with the command line version of tsmuxer correctly placed at the bottom of the video, and remux then automatically. I get details from the processed files with (again) the command line version of mediainfo. It's a one click process, and why I want to use the command line. Without a script to handle every thing, the GUI would be much easier, but very far from a one click process.

And in case it was my interpreter the culprit, by cutting and pasting the command line as is from the GUI into a DOS window, and checking the priority of the mkvmerge process running with the task manager.

So, still a headscratcher.
robena is offline   Reply With Quote
Reply

Tags
matroska

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.