Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th October 2019, 12:19   #481  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
Quote:
Originally Posted by Aleksoid1978 View Post

It is interesting - who and why come up with different names for the same types.
What diff font/sfnt & application/font-sfnt ??

P.S. Do you have spec for new font/ mime types ?
to the best of my knowledge:
font/ttf & font/otf = the current standard Media Types for TTF & OTF respectively, defined in 2017 as RFC 8081, assigned and listed by the IANA
font/sfnt = abstract type that includes both font/ttf and font/otf
application/font-sfnt = ex-standard (2013-2017), still valid as a deprecated alias of font/sfnt
Liisachan is offline   Reply With Quote
Old 7th October 2019, 12:45   #482  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by Liisachan View Post
fonts_embedded.mkv (268 KiB, 10 sec), testing four different FileMimeType values:
Test1 font/ttf
Test2 font/sfnt
Test3 application/font-sfnt
Test4 font/otf

Ideally, something like this is shown:

Above = hardsubbed
Below = softsubbed <-- check this
(PNG)
mpv passes all four tests, and it even warns the user about the old MIME type:
Code:
[sub/ass] Loading font attachment 'homework smart.TTF' with MIME type application/font-sfnt. Assuming this is a broken Matroska file, which was muxed without setting a correct font MIME type.
qyot27 is offline   Reply With Quote
Old 8th October 2019, 13:26   #483  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
Since Mosu suggested a comprehensive test about "font/", I'm posting another test clip: fonts_embedded2.mkv (329 KiB, 10 sec)
This one is like the first sample, except the three other subtypes will be tested:
Test5 = TTC attached as "font/collection" <-- mkvtoolnix-gui uses "application/x-truetype-font"
Test6 = WOFF attached as "font/woff"
Test7 = WOFF2 attached as "font/woff2"
Test8 = TTF files with extension .xtf are attached with 3 different FileMimeType values (mkvtoolnix-gui uses "application/x-truetype-font").
Test5 is practically important for a few users (esp. in CJK where TTC fonts are common). Test6/7 are only for completeness, not practical atm. Test8 is something different.

The letters in the upper area are hardsubbed. Ideally, softsubs in the lower area should look similar to the hardsubs, like this:


Quote:
Originally Posted by Mosu View Post
mkvmerge doesn't detect MIME types itself. It uses the widely used "libmagic" from the "file" project. Waaaay back when "libmagic" used to return "application/…" for fonts. Then one day they switched to "font/…". That was sometime in 2011, and it didn't go down well with users (see bug 682[1]) as their programs weren't ready for "font/…". Therefore I implemented a workaround in mkvmerge in v5.2.0 which manually mapped "font/…" back to "application/…".
As RFC 8081 pointed out, media types such as "font/ttf" had been non-existent until they were defined in 2017. If someone had started using it in 2011 in a widely used library, what they did was rather questionable, given that technically no one is allowed to use an arbitrary top-level media type, while anyone is allwed to use a subtype x-something privately. Besides, not "font/" but "application/font-sfnt" became the official media type for TTF/OTF in 2013.
It's only after 2017 that one may officially use "font/" when attaching a font file. So I think that this is a relatively new problem, not directly related to what you did in 2011. As for what you did in 2011, it was a correct move imho - refusing then-illegal media types in favor of legal (though unregisterable) x- subtypes.

@qyot27
While mpv seems to be a fantastic player, that warning message is a bit strange. Does it show a similar warning about "application/x-truetype-font"? As a fact, "application/font-sfnt" was the only standard Media Type for TTF/OTF before 2017 and it's still officially valid, so one could argue it's more correct than "application/x-truetype-font". Was mpv from 2016 happy about "application/font-sfnt" while unhappy about "font/ttf"? If so, in a way I'd respect such a rigorous player, though maybe it'd be more elegant to handle older files transparently, esp. when the file in question is technically not broken.

Last edited by Liisachan; 12th October 2019 at 07:05.
Liisachan is offline   Reply With Quote
Old 8th October 2019, 15:40   #484  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by Liisachan View Post
@qyot27
While mpv seems to be a fantastic player, that warning message is a bit strange. Does it show a similar warning about "application/x-truetype-font"? As a fact, "application/font-sfnt" was the only standard Media Type for TTF/OTF before 2017 and it's still officially valid, so one could argue it's more correct than "application/x-truetype-font". Was mpv from 2016 happy about "application/font-sfnt" while unhappy about "font/ttf"? If so, in a way I'd respect such a rigorous player, though maybe it'd be more elegant to handle older files transparently, esp. when the file in question is technically not broken.
The relevant commit in mpv is f3d2f4c6c2a (from 2017-12-12), which added the font/ top level type to follow RFC8081. The list of font mimetypes it accepts in the subtitle demuxer is (currently, sub/sd_ass.c @ lines 85-92):
Code:
    "application/x-truetype-font",
    "application/vnd.ms-opentype",
    "application/x-font-ttf",
    "application/x-font", // probably incorrect
    "font/collection",
    "font/otf",
    "font/sfnt",
    "font/ttf",
And if it doesn't see the font as one of those, it throws that warning shown before. I'm not sure why it doesn't have "application/font-sfnt" in the list, as there's no indication that it was removed, but because mpv detects fonts based on file extension as well (sub/sd_ass.c @ line 96), Test 3 in the original file worked despite the warning, but the application/font-sfnt .xtf in Test 8 did not. The file extension detection may have simply masked the issue, so there was never any reason to explicitly include application/font-sfnt...maybe?
qyot27 is offline   Reply With Quote
Old 9th October 2019, 13:54   #485  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
LAV Filters now explicitly support the 3 important now-standard Media Types and 1 standard alias (related code changes); the binary (v0.74.1-26) was tested with MPC and a few MPC-based players (by disabling their internal source filter for Matroska), 100% passing Tests 1-4 and 8 (one-upping mpv in Test 8 ). In the LAV Filters thread, I requested the addition of the "font/collection" support.

In retrospect, it would have been convenient if the Matroska specs had, under each AttachedFile, something like "AttachmentType", the value of which could have been 0 (default: unspecified), 1 (cover art), 2 (back cover art), 3 (embedded font), etc. Then, players could simply load everything as font iff AttachmentType=3, and it could reliably show the cover art in MKA even if a lot of images are attached.
Liisachan is offline   Reply With Quote
Old 9th October 2019, 17:52   #486  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Thank you very much for all the investigative work, Liisachan!
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 11th October 2019, 21:30   #487  |  Link
MrVideo
Registered User
 
MrVideo's Avatar
 
Join Date: May 2007
Location: Wisconsin
Posts: 2,130
I found a minor grammatical error in mkvmerge, as seen in the MKVToolnix GUI error output listing:
Code:
However, no headers where found for that track number.
The word "where" describes a physical location. The correct word is: were
MrVideo is offline   Reply With Quote
Old 12th October 2019, 07:03   #488  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
@qyot27
Perhaps it simply shows that application/font-sfnt was unsuccessful (*). Failure to support it is not a big problem as this type is so rarely used, and it's indirectly supported anyway if the extension is .ttf or .otf (I think). It's impressive that mpv's FileMimeType support was near perfect already 2 years ago.

@Mosu
Real thanks should go to Aleksoid1978 and Nevcairiel If ToolNix suddenly starts using the new font types, there may be a lot of confusion. I suggest there be a transitional period, like until the end of 2020 (or whenever you see fit); mkvmerge can output some kind of notices. E.g. if font/ttf is used too early, it could say, "this is a standard type name, but old players may not recognize it", and if application/x-truetype-font is used too late, it could say "you might want to use the newly standardized type names".

(*) meaning, the media type application/font-sfnt, though officially registered, was never commonly used. Most developers, servers, clients, end-users didn't start using it — perhaps they didn't/don't even know it exists — to the point that the same media type had to be re-defined using a more intuitive name, font/sfnt. CSS3 @font-face was still a Working Draft when application/font-woff & application/font-sfnt were registered in early 2013; maybe only a few, experimental and standard-aware parties (like Firefox devs?) were interested in them back then. One may remember that MSIE hadn't recognized application/xhtml+xml for many years even after XHTML 1.1 was standardized.

EDIT:
Now MPC-BE (r4808+) fully supports the 5 mime types: font/sfnt, /ttf, /otf, /collection & application/font-sfnt, passing Tests 1–5 & 8 100%. LAV is also already near-perfect (except font/collection), passing Tests 1–4 & 8 100%.

EDIT2 (2019-10-14):
Now LAV Filters fully support the 5 mime types!!

Last edited by Liisachan; 15th October 2019 at 01:00.
Liisachan is offline   Reply With Quote
Old 14th October 2019, 23:49   #489  |  Link
saracas
Registered User
 
Join Date: Dec 2018
Posts: 10
I cannot for the life of me figure out how to batch edit 300 MKVs using the same steps... I go into MKVtoolnix and copy the command line after setting one up one episode and I get this:

"C:/Program Files/MKVToolNix\mkvmerge.exe" --ui-language en --output ^"W:\SHOWNAME\Season 01\Show.Name.S01E01.DVD ^(1^).mkv^" --audio-tracks 1 --language 0:eng --track-name ^"0:MPEG-2 / 480p / 23.976 fps^" --language 1:jpn --track-name ^"1:Japanese / AC-3 / 2.0 / 448 kbps^" --default-track 1:yes --language 2:eng --track-name 2:English --default-track 2:yes --forced-track 2:yes ^"^(^" ^"W:\SHOWNAME\Season 01\Show.Name.S01E01.DVD.mkv^" ^"^)^" --track-order 0:0,0:1,0:2

How do I make the two directories in there specify all files in those folders? Basically all I am doing is removing english track and setting first subtitle track to forced for all these files.
saracas is offline   Reply With Quote
Old 15th October 2019, 00:47   #490  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
you can try something like this example
test.bat
---
FOR %%f IN (*.mkv) DO (mkvmerge -o "new\%%f" --no-attachments "%%f")
---

EDIT (FIX)
I forgot to quote the last %%f, which doesn't work if a file name has a space, "%%f" should work in such a case too. The example above will transmux every MKV file in the current folder and write new files in the sub-folder "new". --no-attachments is just an example, where attachments will be dropped in newly created MKVs. One can replace this part with what they actually want to do.

Last edited by Liisachan; 18th October 2019 at 21:21.
Liisachan is offline   Reply With Quote
Old 18th October 2019, 01:09   #491  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
Sometimes Job output says "No errors" while status bar says "error"

[GUI] Error status from the previous session persists until explicitly acknowledged, even when no errors occur in the current session. Maybe by design, but could be confusing as follows:

Tested: MKVToolNix v38 on Win7 32-bit

Steps to repro:
1. Start mkvtoolnix-gui.exe, select a source file (e.g. some.avi)
2. Rename "some.avi" to "renamed.avi" behind the back of the GUI
3. On GUI: Start muxing -> the status bar shows [X] 1 error (as the source file is not found)
4. Just close mkvtoolnix-gui, restart it later -> Still [X] 1 error is shown
5. Select "renamed.avi" and mux it into "renamed.mkv" -> Now successful. but...

Expected result:
In most tools (especially command line ones), even if an error occurred in the previous session and an error message was shown, that error will be automatically forgotten if one restarts the tool and do something new. If old errors are remembered, the corresponding old error messages should be shown too, to explain what errors it is talking about.

What happened instead:
GUI still shows [X] 1 error. Job Output inconsistently says "No errors yet".
Liisachan is offline   Reply With Quote
Old 18th October 2019, 08:30   #492  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by Liisachan View Post
In most tools (especially command line ones), even if an error occurred in the previous session and an error message was shown, that error will be automatically forgotten if one restarts the tool and do something new. If old errors are remembered, the corresponding old error messages should be shown too, to explain what errors it is talking about.
And that is the case for MKVToolNix, too. What you failed to realize is that the status bar indicators refer to the jobs known to MKVToolNix, not to what the "job output" tool shows. As soon as that job containing the error is removed, the status bar indicator will go back to 0 errors. In the same spirit, you can right-click on the job & select "View output". A new tab will be opened in the "job output" tool, and that tab will include the error.

On top of all that, the always-visible tab in the "job output" tool is labeled "current job". Of course there aren't any errors in it when you've just opened the GUI as there hasn't been a "current job" yet.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 18th October 2019, 08:36   #493  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by MrVideo View Post
I found a minor grammatical error in mkvmerge, as seen in the MKVToolnix GUI error output listing:
Thanks, I'll fix it.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 18th October 2019, 08:40   #494  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
Quote:
Originally Posted by Liisachan View Post
I suggest there be a transitional period, like until the end of 2020 (or whenever you see fit); mkvmerge can output some kind of notices. E.g. if font/ttf is used too early, it could say, "this is a standard type name, but old players may not recognize it", and if application/x-truetype-font is used too late, it could say "you might want to use the newly standardized type names".
I'm hesitant to add a real warning for such things as it would punish those who use the new types on purpose (punish in the sense that none of their jobs using the new font types would be succeeding anymore, and they'd always have to check if it was only due to the font warning or to something actually important). This means I'd have to make the warning configurable, and that's always a lot of work. A note would be fine, of course. Then again, a simple note would probably never be read by most GUI users.

End of 2020 sounds reasonable; maybe even a bit longer.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 18th October 2019, 09:07   #495  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
Quote:
This means I'd have to make the warning configurable, and that's always a lot of work.
For the GUI aspect there are some ways to reduce the work, run time generated GUI is some work but easier to deal with than older UI toolkits (never worked with QT), generic GUI based on a grid is easier, most effective is just a INI conf file, I've used it this week for my new MediaInfo GUI and it was totally easy to do.

https://postimg.cc/TLkPDKsX
stax76 is offline   Reply With Quote
Old 18th October 2019, 13:40   #496  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,278
The problem isn't adding yet another checkbox to the Qt UI. It's the amount of things to do for each such configurable thing:
  1. Add checkbox in preferences dialog (trivial, it's one line in the corresponding .ui file)
  2. Determine keyboard shortcut that isn't used by any other visible control for English & German
  3. Save that setting in the configuration
  4. Add a corresponding command line option to mkvmerge
  5. Add help output for the new command line option
  6. Add documentation in mkvmerge's man page
  7. Implement both the warning as well as having it turned off in mkvmerge
  8. Implement test cases for all three cases (other MIME type, bad MIME type with warning at defaults, bad MIME type with warning turned off)
  9. Update the program & man page translations
  10. Translate all the new content into German
  11. Add a NEWS entry

Nothing about this is hard. It's just tedious & takes time. Nothing about what you've said would reduce the amount of work I'd have to do.
__________________
Latest MKVToolNix is v83.0

If I ever ask you to upload something, please use my file server.
Mosu is offline   Reply With Quote
Old 18th October 2019, 21:08   #497  |  Link
Liisachan
李姗倩 Lǐ Shān Qiàn
 
Liisachan's Avatar
 
Join Date: Nov 2002
Posts: 1,340
@Mosu
My point is, if MKVToolNix started using "font/" suddenly, there could be a lot of confusion. If you have a plan to migrate to the new types, I suggest you advertise in advance that there will be compatibility-breaking changes, so that subbers can make informed decisions. Warning from mkvmerge (CUI, not GUI) is one of the possible things you could do, but if it's not easy, that's fine. There are other things we could do, such as simply posting a short explanation in subtitle-related forums or making a user-friendly font-attachment FAQ page.

Maybe some subbers don't want to migrate to the new types. Technically, RFCs explicitly guarantee that one can freely use private x-tokens such as application/x-truetype-font as long as both parties (in this case muxer and player) agree on it. As such, it will be perfectly fine and standard-compliant (perhaps even recommended) to keep using x- MIME types privately inside Matroska (note: SSA/ASS itself is a private format, not officially standardized). In fact, when font-embedding was first implemented by Haali (as a patch to Gabest's splitter), it was announced in doom9 that one MUST use application/x-truetype-font.
On the other hand, since Matroska is an open-spec format, where FileMimeType is defined as the MIME type, one may (and perhaps now should) use the registered MIME types. The catch is, doing so will generate MKVs that older players don't understand. In the worst case, subtitles will be unreadable at all. Sometimes the problem will be only stylistic (wrong font faces), though I imagine young typesetters will be upset when that happens (old typesetters know that font-embedding is optional to begin with, e.g. disabled by default in Haali Splitter until like 2007).

Yet another concern is, unless clearly documented (ideally as a footnote to FileMimeType in the official specs on Matroska.org), some of the legacy types might be forgotten quickly and new-generation players in the future won't play old MKVs right. There are actually as many as 10 different MIME Types that may be used for attached fonts. Implementers generally don't know this undocumented, confusing situation. For example, LAV Filters were fixed in v0.74 saying "Fonts embedded in MKVs without a proper mimetype were not being imported" - where it is assumed that the legacy types are the proper mime types. As another example, application/font-sfnt is registered but was/is poorly supported (even by mpv).
End users (including myself) are even more clueless. I only noticed this problem recently, though the top-level type "font" was registered in 2017. Imho, it could be helpful if mkvmerge warns when "problematic" MIME types are specified. But like you said, some of the "problematic" types are actually "good" (officially registered) ones, and there is no fool-safe solution to this dilemma. That said, there are a few end-users who happened to use new types such as application/font-sfnt simply because their tool chains had been updated that way, even though they wouldn't have used such poorly-supported types if warned about them.

I'd say, a warning like "font/ttf may cause problems. This is not an error. It's fine if you know what you're doing" wouldn't be "punishment". Seeing that, 1% of the users would be cool, like "yeah, I know why this warning is shown. This is a difficult problem and I see mkvmerge is trying to be helpful" and 99% of the users would be like "did I do something wrong? I'll have to check about this" - both would be good. In the second case, ideally they will do some search and will be like "I now understand the problem. I tested several players. I'll use font/ttf" - or they'll be like "I now understand the problem. I'll keep using x-types". Both options sound fine to me. Let them make an informed decision about their own files

As for the error status of GUI, it's ok if it's by design. I just wanted to point out that it could be confusing when one uses GUI as a one-time front end, and not as a batch-job front end.

~~PS~~
About the status bar message on GUI, I'd think two minor text changes might be helpful.
1) In the main manu: "Job queue" -> "Job queue/log"
2) In the context menu: "Show job queue" -> "Show job queue/log"

Unfamiliar with this GUI, one may assume the queue is empty in the example I described above (the same status bar says "Jobs to execute = 0, 0, 0", certainly looking like the queue is empty). This may be just me, but intuitively, one has no reason to look into the "Job queue" when an error is reported. But if the context menu item says "Show job queue/log" when "error(s)" is right-clicked, it'll be immediately clear where to look.

Last edited by Liisachan; 19th October 2019 at 05:40.
Liisachan is offline   Reply With Quote
Old 20th October 2019, 20:58   #498  |  Link
mood
Registered User
 
Join Date: May 2012
Posts: 92
In the last continuous build 38.0.0-revision-028, how can I disable the dark theme?? the new GUI is mess up, I'm using windows 7 x86

with same configuration settings, the dark theme GUI size change turn it a mess up


38.0.0-revision-028



38.0.0-revision-019


Last edited by mood; 20th October 2019 at 21:06.
mood is offline   Reply With Quote
Old 21st October 2019, 14:22   #499  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
Sort of an odd request, but when setting mkvtoolnix to split by chapters, would it be possible to have a period as a valid separator along with the comma? Since I use the numpad to enter chapter numbers, it would make it far easier to enter 2.3.5.6 than reach all the way over for the comma. (I generally sit at an angle to my desk with the keyboard almost out of reach when I don't have it in my lap for typing, and just making a stab for the comma will usually end up with me hitting "m" or ".")

Any of /*-+ would also work. i.e. 2+3+5+6
kuchikirukia is offline   Reply With Quote
Old 21st October 2019, 16:13   #500  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,351
I agree with mood, the "dark theme" thing suxxx.
At least there should be an option to turn it off.
filler56789 is offline   Reply With Quote
Reply

Tags
matroska

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.