MKVToolNix v24.0.0 released

[ndjamena]

Originally Posted by ndjamena

MakeMKV is complaining bitterly about any VC-1 track that has been remux using MKVMerge, so far the only difference I can see using MKVInfo is that the first frame in the MKVMerge file is 33 bytes longer for some reason. The file was originally created by MakeMKV then remuxed by MKVMerge, so is this added 33 bytes expected?

http://www.makemkv.com/forum2/viewtopic.php?f=8&t=9039

I just remuxed a VC-1 file previously muxed using MKVMerge and checked the first frame and it was 30 bytes longer than the original, then I remuxed THAT file and checked again and the third version was 30 bytes longer than the previous one (60 bytes longer than the first).

Is this new or should I assume all my VC-1 files have these extra 30 bytes?

[Mosu]

As the VC-1 handling code hasn't changed in ages this has probably been present for a long time.

[Thunderbolt8]

Originally Posted by ndjamena

I just remuxed a VC-1 file previously muxed using MKVMerge and checked the first frame and it was 30 bytes longer than the original, then I remuxed THAT file and checked again and the third version was 30 bytes longer than the previous one (60 bytes longer than the first).

Is this new or should I assume all my VC-1 files have these extra 30 bytes?

does this somehow translate into audio/video delay issues? how big of a delay would that be?

[Mosu]

Uhm… I doubt that. 30 bytes sounds more like CodecPrivate being duplicated (think of SPS/PPS info in h.264 tracks being duplicated by extraction).

[ndjamena]

I was thinking that but Codec Private was 74 bytes and MakeMKV is refusing to remux them. They play fine though but that could be error correction or not.

Mike should be looking at a sample by now and hopefully the next version of MakeMKV will be able to read through whatever it is.

It might be nothing, but Batman is VC-1 and I know I've remuxed that movie several times already, and I'm trying to get a final version of The Matrix done at the moment, the audio is being a pain...

I would like to know exactly what it is though.

[Boulder]

Is there a possibility that mkvalidator is not playing nicely or is it possible that a Matroska file corrupts in a few minutes?

First I got:

R:\>mkvalidator --no-warn "The X-Files - S07E01 - The Sixth Extinction.mkv"
................................................................................
mkvalidator 0.5.0: the file appears to be valid
        file created with libebml v1.3.1 + libmatroska v1.4.2 / mkvmerge v7.9.0
('Birds') 64bit

A few minutes later I got:

R:\>mkvalidator --no-warn "The X-Files - S07E01 - The Sixth Extinction.mkv"
ERR066: The SeekPoint at 87 references an unknown Cues at 401390961
        file created with libebml v1.3.1 + libmatroska v1.4.2 / mkvmerge v7.9.0
('Birds') 64bit

I have also other files from the same season which have the same thing happen to them.

I used "for %f in (*.mkv) do mkvalidator --no-warn "%f"" to check the files.

[Mosu]

Hmm such things could happen due to uninitialized variables (= a bug). You should file a report at https://github.com/Matroska-Org/foun...-source/issues

[Boulder]

Is it possible to compare Matroska files? I'd like to make sure that my computer is not to blame, but using an MD5 checksum doesn't work.

[Mosu]

I don't know of an easy-to-use tool for that. One possibility is to run mkvinfo -v on both files, direct the output to text files, use a diff tool and to ignore those elements known to be unique to each file (e.g. muxing date, all the UIDs).

[Boulder]

It seems that I can pinpoint the problem to occur on two hard drives that are in an external USB dock (connected to USB 3.0 via a PCIe card). Copying the same (working) muxed files from a different drive to one with problems doesn't seem to break them. Also other drives attached to the same PCIe card don't have the problem. I also tested a few FLAC files I have there but they are fine.

My question is: does mkvmerge use some special method to write the files? I have had write caching disabled so the drives are in quick removal mode.

[Mosu]

mkvmerge uses the default functions offered by the operating systems; WriteFile() on Windows and fwrite() everywhere else.

[Boulder]

I was able to make all USB drives corrupt data, no matter if they were directly connected to the motherboard or via the PCIe card. Then I enabled write caching (the "Better performance" option) in Device Manager for all my USB drives and the issue is now gone. I have absolutely no clue why the setting would have anything to do with it, in fact I would expect the exact opposite, but no more corruption when muxing I do recall reading that uTorrent and KTorrent had similar issues related to Windows caching some time in the past.

[hello_hello]

Boulder,
I think that problem has been mentioned in the VideoHelp computer sub-forum a few times and from memory it only applies to the 64 bit version of Windows 7. Is that what you're using? Have a look there but I'm not sure if anyone found another solution, aside from not using Windows Explorer to copy and paste but using a third party copying utility instead, although of course that probably doesn't help when using a USB drive as the destination drive when muxing it.

Anyway, I just thought I'd mention it in case you wanted to look at those threads. Maybe it's related, or maybe not.

[Selur]

Here's what I do is:
I analyse my input with mkvinfo

mkvinfo.exe --ui-language en  -s "H:\193.mkv"

which gives me:

...
Track 1: video, codec ID: V_MPEG4/ISO/AVC (h.264 profile: High 10 @L3.0), mkvmerge/mkvextract track ID: 0, default duration: 41.708ms (23.976 frames/fields per second for a video track), language: jpn, pixel width: 698, pixel height: 478, display width: 698, display height: 537
Track 2: audio, codec ID: A_AC3, mkvmerge/mkvextract track ID: 1, default duration: 32.000ms (31.250 frames/fields per second for a video track), language: jpn, sampling freq: 48000, channels: 2
...

and MediaInfo

"G:\MkvCutter\MediaInfo.exe" --Full "H:\193.mkv"

which gives me:

...
Video
Count                                    : 311
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
Unique ID                                : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format/Url                               : http://developers.videolan.org/x264.h
tml
Commercial name                          : AVC
Format profile                           : High 10@L3
Format settings                          : CABAC / 6 Ref Frames
Format settings, CABAC                   : Yes
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 6
Format settings, ReFrames                : 6 frames
Internet media type                      : video/H264
Codec ID                                 : V_MPEG4/ISO/AVC
Codec ID/Url                             : http://ffdshow-tryout.sourceforge.net
/
Codec                                    : V_MPEG4/ISO/AVC
Codec                                    : AVC
Codec/Family                             : AVC
Codec/Info                               : Advanced Video Codec
Codec/Url                                : http://ffdshow-tryout.sourceforge.net
/
Codec profile                            : High 10@L3
Codec settings                           : CABAC / 6 Ref Frames
Codec settings, CABAC                    : Yes
Codec_Settings_RefFrames                 : 6
Duration                                 : 1461712
Duration                                 : 24mn 21s
Duration                                 : 24mn 21s 712ms
Duration                                 : 24mn 21s
Duration                                 : 00:24:21.712
Duration                                 : 00:24:22;02
Duration                                 : 00:24:21.712 (00:24:22;02)
Bit rate                                 : 1044138
Bit rate                                 : 1 044 Kbps
Width                                    : 698
Width                                    : 698 pixels
Height                                   : 478
Height                                   : 478 pixels
Pixel aspect ratio                       : 0.890
Original pixel aspect ratio              : 0.889
Display aspect ratio                     : 1.300
Display aspect ratio                     : 4:3
Original display aspect ratio            : 1.298
Original display aspect ratio            : 1.298
Frame rate mode                          : CFR
Frame rate mode                          : Constant
Frame rate                               : 23.976
Frame rate                               : 23.976 fps
Frame count                              : 35046
Resolution                               : 10
Resolution                               : 10 bits
Colorimetry                              : 4:2:0
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10
Bit depth                                : 10 bits
Scan type                                : Progressive
Scan type                                : Progressive
Interlacement                            : PPF
Interlacement                            : Progressive
Bits/(Pixel*Frame)                       : 0.131
Delay                                    : 0
Delay                                    : 00:00:00.000
Delay, origin                            : Container
Delay, origin                            : Container
Stream size                              : 190778595
Stream size                              : 182 MiB (72%)
Stream size                              : 182 MiB
Stream size                              : 182 MiB
Stream size                              : 182 MiB
Stream size                              : 181.9 MiB
Stream size                              : 182 MiB (72%)
Proportion of this stream                : 0.71656
Writing library                          : x264 - core 142 r2453 ea0ca51
Writing library                          : x264 core 142 r2453 ea0ca51
Encoded_Library_Name                     : x264
Encoded_Library_Version                  : core 142 r2453 ea0ca51
Encoding settings                        : cabac=1 / ref=6 / deblock=1:1:1 / ana
lyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me
_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_p
skip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=
0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / b
frames=9 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=
0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_
lookahead=60 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.70 / qpmin=0 / qpmax=81 /
qpstep=4 / ip_ratio=1.40 / aq=1:0.60
Language                                 : ja
Language                                 : Japanese
Language                                 : Japanese
Language                                 : ja
Language                                 : jpn
Language                                 : ja
Default                                  : Yes
Default                                  : Yes
Forced                                   : No
Forced                                   : No
...

then I split the file, using:

-o
H:\\Temp\\test.mkv
--split
parts-frames:32545-34102,34102-34352
--no-audio
--no-subtitles
--no-buttons
--no-track-tags
--no-chapters
--no-attachments
--no-global-tags
H:\\193.mkv

as H:\Temp\test_mkvOptions.txt
with:

"G:\MkvCutter\mkvmerge.exe" @"H:\Temp\TEST_M~1.TXT"

which creates:

1. H:\Temp\test-001.mkv
2. H:\Temp\test-002.mkv

then I cut the file using

LoadPlugin("G:\MkvCutter\LSMASHSource.dll")
LWLibavVideoSource("H:\Temp\test-002.mkv", format="YUV420P10", cache=false)

Trim(0,length=195)

as: H:\Temp\test-002.avs
and rencode the file using:

"G:\MkvCutter\avs2yuv.exe" -raw "H:\Temp\test-002.avs" -o - | "G:\MkvCutter\x264-10bit.exe" --profile high10 --level 30 --sps-id 0 --cabac --ref 6 --deblock 1:1 --partitions all --me umh --subme 10 --psy-rd 0.40:0.00 --merange 24 --trellis 2 --8x8dct --deadzone-inter 21 --deadzone-intra 11 --qblur -2 --bframes 9 --b-pyramid normal --b-adapt 2 --b-bias 0 --direct auto --weightp 2 --keyint 250 --min-keyint 23 --scenecut 40 --rc-lookahead 60 --qcomp 0.70 --qpmin 0 --qpmax 81 --qpstep 4 --ipratio 1.40 --aq-mode 1 --aq-strength 0.60 --chroma-qp-offset -2 --stitchable --non-deterministic --thread-input --crf 19 --demuxer raw --input-depth 10 --input-res 698x478 --fps 24000/1001 --sar 1000:890 -o "H:\Temp\test-002_reencode.264" -

encoding next file,...

raw [info]: 698x478p 100:89 @ 24000/1001 fps (cfr)
x264 [info]: using SAR=100/89
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
x264 [info]: profile High 10, level 3.0, 4:2:0 10-bit
1 frames: 1.36 fps, 745.56 kb/s
23 frames: 22.16 fps, 147.96 kb/s
46 frames: 34.10 fps, 127.19 kb/s
64 frames: 39.98 fps, 119.78 kb/s
85 frames: 45.14 fps, 116.71 kb/s
107 frames: 49.22 fps, 114.23 kb/s
137 frames: 56.03 fps, 114.47 kb/s
170 frames: 63.08 fps, 117.26 kb/s
x264 [info]: frame I:1     Avg QP:27.84  size:  3887
x264 [info]: frame P:38    Avg QP:30.81  size:  1305
x264 [info]: frame B:156   Avg QP:34.43  size:   409
x264 [info]: consecutive B-frames:  1.0%  0.0%  3.1% 12.3% 33.3% 43.1%  7.2%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 62.2% 36.2%  1.6%
x264 [info]: mb P  I16..4:  2.8%  1.5%  0.0%  P16..4: 41.0%  2.2%  1.9%  0.0%  0.0%    skip:50.5%
x264 [info]: mb B  I16..4:  0.0%  0.0%  0.0%  B16..8: 24.9%  0.9%  0.1%  direct: 0.2%  skip:73.9%  L0:50.1% L1:48.7% BI: 1.2%
x264 [info]: 8x8 transform intra:36.6% inter:88.6%
x264 [info]: direct mvs  spatial:95.5% temporal:4.5%
x264 [info]: coded y,uvDC,uvAC intra: 10.7% 0.0% 0.0% inter: 0.8% 0.0% 0.0%
x264 [info]: i16 v,h,dc,p: 20% 25% 11% 44%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  5%  6% 50%  5%  8%  5%  9%  4% 10%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  6% 20% 13%  4% 10%  3% 33%  2%  9%
x264 [info]: i8c dc,h,v,p: 100%  0%  0%  0%
x264 [info]: Weighted P-Frames: Y:44.7% UV:0.0%
x264 [info]: ref P L0: 37.4% 20.1% 18.0%  9.9%  9.9%  4.7%  0.1%
x264 [info]: ref B L0: 73.9% 11.9%  8.4%  4.1%  1.7%
x264 [info]: ref B L1: 93.0%  7.0%
x264 [info]: kb/s:115.36

encoded 195 frames, 68.06 fps, 115.36 kb/s

Note that is used '--sar 1000:890'.
Now I cut the audio using:

-o
H:\\Temp\\test_AudioCut.mkv
--split
parts:00:22:37.398-00:23:50.471
--no-video
--no-subtitles
H:\\193.mkv

as: H:\Temp\test_AudioCut_mkvOptions.txt
which creates:

1. H:\Temp\test_AudioCut.mkv

Then I join the audio&video files again, using H:\Temp\test-001_mkvOptions.txt:

-o
H:\\Output\\test.mkv
--clusters-in-meta-seek
--engage
no_simpleblocks
--disable-lacing
--engage
no_cue_duration
--engage
no_cue_relative_position
--no-global-tags
--no-chapters
--no-subtitles
--no-track-tags
--no-buttons
--no-audio
--no-attachments
--forced-track
0:no
--default-duration
0:24000/1001p
--fix-bitstream-timing-information
0:1
H:\\Temp\\test-001.264
--no-global-tags
--no-chapters
--no-subtitles
--no-track-tags
--no-buttons
--no-audio
--no-attachments
--forced-track
0:no
--default-duration
0:24000/1001p
--fix-bitstream-timing-information
0:1
+
(
H:\\Temp\\test-002_reencode.264
)
--no-video
--no-global-tags
--no-chapters
--no-subtitles
--no-track-tags
--no-buttons
H:\\Temp\\test_AudioCut.mkv
--append-to
1:0:0:0

and:

"G:\MkvCutter\mkvmerge.exe" @"H:\Temp\test-001_mkvOptions.txt"

so far so good, problem is that when I know analyse the output with MediaInfo, I see:

...
Video
Count                                    : 311
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
Unique ID                                : 12902876771495624352
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format/Url                               : http://developers.videolan.org/x264.h
tml
Commercial name                          : AVC
Format profile                           : High 10@L3
Format settings                          : CABAC / 6 Ref Frames
Format settings, CABAC                   : Yes
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 6
Format settings, ReFrames                : 6 frames
Internet media type                      : video/H264
Codec ID                                 : V_MPEG4/ISO/AVC
Codec ID/Url                             : http://ffdshow-tryout.sourceforge.net
/
Codec                                    : V_MPEG4/ISO/AVC
Codec                                    : AVC
Codec/Family                             : AVC
Codec/Info                               : Advanced Video Codec
Codec/Url                                : http://ffdshow-tryout.sourceforge.net
/
Codec profile                            : High 10@L3
Codec settings                           : CABAC / 6 Ref Frames
Codec settings, CABAC                    : Yes
Codec_Settings_RefFrames                 : 6
Duration                                 : 73073
Duration                                 : 1mn 13s
Duration                                 : 1mn 13s 73ms
Duration                                 : 1mn 13s
Duration                                 : 00:01:13.073
Duration                                 : 00:01:13;02
Duration                                 : 00:01:13.073 (00:01:13;02)
Bit rate                                 : 914620
Bit rate                                 : 915 Kbps
Width                                    : 698
Width                                    : 698 pixels
Height                                   : 478
Height                                   : 478 pixels
Pixel aspect ratio                       : 1.123
Original pixel aspect ratio              : 0.889
Display aspect ratio                     : 1.640
Display aspect ratio                     : 16:10
Original display aspect ratio            : 1.298
Original display aspect ratio            : 1.298
Frame rate mode                          : CFR
Frame rate mode                          : Constant
Frame rate                               : 23.976
Frame rate                               : 23.976 fps
Frame count                              : 1752
Resolution                               : 10
Resolution                               : 10 bits
Colorimetry                              : 4:2:0
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10
Bit depth                                : 10 bits
Scan type                                : Progressive
Scan type                                : Progressive
Interlacement                            : PPF
Interlacement                            : Progressive
Bits/(Pixel*Frame)                       : 0.114
Delay                                    : 0
Delay                                    : 00:00:00.000
Delay, origin                            : Container
Delay, origin                            : Container
Stream size                              : 8354260
Stream size                              : 7.97 MiB (69%)
Stream size                              : 8 MiB
Stream size                              : 8.0 MiB
Stream size                              : 7.97 MiB
Stream size                              : 7.967 MiB
Stream size                              : 7.97 MiB (69%)
Proportion of this stream                : 0.69014
Default                                  : Yes
Default                                  : Yes
Forced                                   : No
Forced                                   : No

...

which indicates 'Pixel aspect ratio : 1.123'
-> Any idea why the Pixel Aspect ratio isn't 0.89?


Cu Selur

[RDF]

Hello, Mosu

Is mkvtoolnix ever going to switch to ISO 639-3 language codes or will stay forever with the standard ISO 639-2?

[Mosu]

RDF: The Matroska file format uses ISO 639-2. Therefore mkvmerge cannot simply use 639-3 codes. So the first step would have to be to extend the Matroska file format with new language elements that use the 639-3 codes, then to have support implemented in muxers like mkvmerge (and the associated GUIs), ffmpeg or gstreamer, and then in the players. A shitload of work, all in all, especially as all tools would have to be able to deal with two language attributes at once suddenly.

In short: it's highly unlikely that this will ever happen unless someone who cares about such a feature does the work. I don't care about that enough, so it won't be me.

[Mosu]

Selur: I really don't understand your question. You're actively telling x264 to use a pixel/sample aspect ratio of 1000/890 which is 1.1235… and then you wonder why MediaInfo outputs exactly that very same pixel/sample aspect ratio? Anyway, I have no clue whatsoever which value(s) MediaInfo choses to output as the various aspect ratios (sample/display/original sample/original display/pixel/whatever…); so may this question would be better off in a MediaInfo thread.

[Selur]

DOH,.. you are right I totally overlooked that I used 1000/890, when I wanted to use 890/1000. Man looked at it for quite some time and totally overlooked it.

[Mosu]

You even pointed it out in your post

[Selur]

I know,.. I really need some more sleep.