BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[0xdeadbeef]

@avivahl:
Ok, I had a look at the "dmd-sfchunli" sample. First thing to mention is that there are three languages in the IDX (English, French, Spanish). The default language is English (langidx: 0), so BDSup2Sub ignores all but the English captions.

This simple approach worked for all the multilanguage VobSubs I tried up to now, but unfortunately not for this one. The problem is that the the packets for the three languages are interleaved. I.e. the first English subtitle starts at 0x0000, followed by the first French subtitle at 0x0800, followed by the first Spanish subtitle at 0x1000. The 2nd English subtitle then starts at 0x1800. This still would be no issue, but at subtitle #177 (offset 0x141000 / timestamp: 00:17:53:451), the stream contains interleaved packages for a subpicture spread over multiple packets. So where BDSup2Sub expects the first extended packet of the English subpicture, it reads the start packet of a French subpicture which leads to the error.

The only way to fix this is to actually check the substream ID embedded into each packet (which I ignored up to now) and to skip packets with the wrong substream ID. I kinda fear though that there are some VobSub streams out there with inconsistent substream ID, but yeah well.

So this will be fixed and probably soon, but if you can't wait, I'd recommend to use VobSubs with only one language ID as a workaround for the moment (e.g. load and export in SC).

[0xdeadbeef]

Hm, fixing this was surprisingly easy.
BTW: did I already mention how much it sucks to have only a 56k modem until my DSL connection is fixed?

21.06.2009: 3.9.3 -> 3.9.4

• Changed: VobSub: stream ID is used to skip packets to work around problems with interleaved streams.
• Changed: added tooltips for most buttons, combo boxes etc.

[avivahl]

Wonderful, thanks!
I love how you give such detailed problem analyzation for every bug report.

[avivahl]

btw, I just finished looking over the code you included in the jar file and I must say it looks great.
You have a really good coding style.
Everything seems to be very modular and well commented (functions).

I think it would be great if you:
a) Open source it (completely). The available code looks great and easy to read. If the rest of the code is like this, it's already in good shape.
b) Add srt *import* support. Yes, that probably means adding SRT parsing and text rendering using fonts and such, but I'm sure it will be worth it.

Good job.

EDIT:
http://docstore.mik.ua/orelly/java-ent/jfc/ch04_09.htm

Originally Posted by avivahl

b) Add srt *import* support. Yes, that probably means adding SRT parsing and text rendering using fonts and such, but I'm sure it will be worth it.

I think this is supposed to be a tool for editing bitmap-based subtitle formats and not a text-based format like SRT. Plus most of the options that this program provides would be rather useless since SRT contains no font or things like positional information in the format. It's really up to 0xdeadbeef to want to add such support but in case he doesn't and you want something to use in the meantime you might just want to grab a tool like Aegisub, Subtitle Workshop, etc.

[avivahl]

Well, since it also supports exporting to many formats (including BluRay SUP), I think that SRT importing would be great. Would probably require a single class to act as a bridge between the text and bitmap. The rest of the program wouldn't even know it's a text source. The class would render the text and return a bitmap representation of it. I'm thinking about a dialog with font/style selection and such.

Btw, deadbeef, which GUI toolkit do you use? SWT? AWT? Swing?

Originally Posted by avivahl

Well, since it also supports exporting to many formats (including BluRay SUP), I think that SRT importing would be great.

That's because SUP is a bitmap-based subtitle format and hence would fall under the scope of a program for allows you to "convert and tweak bitmap subtitle streams".

Originally Posted by avivahl

Would probably require a single class to act as a bridge between the text and bitmap.

Sure, but it would again go against the whole point of the program. If you want to edit text-based subs there are already numerous programs to do this.

Originally Posted by avivahl

The rest of the program wouldn't even know it's a text source. The class would render the text and return a bitmap representation of it. I'm thinking about a dialog with font/style selection and such.

Or you can just use this or this to do that already since you now seem to want an SRT to SUP converter.

Originally Posted by avivahl

Btw, deadbeef, which GUI toolkit do you use? SWT? AWT? Swing?

He uses AWT and Swing.

[0xdeadbeef]

I wouldn't rule out the possibility that I might decide to add support for SRT import some day - yet I don't plan to right now. Mainly for the reasons that Gokumon already pointed out: There are other tools for this and BDSup2Sub is focused on bitmap subtitles.

About the source code: I sure plan to release all of it some day. What I released up to now is what I consider to be stable (in the sense of: I probably don't need to change this much more) and (more or less) well-designed. This is not necessarily the case for the rest of the sources

About the GUI: yep, it's plain Swing (aka "Java Foundation Classes") which is based on the AWT. I don't use any AWT classes directly though.

[SquallMX]

Hi, i have a shameless request

By default BDSup2Sub restores "hidden" subtitles in VobSub files (Invisible caption due to zero alpha - used alpha info of last caption.), would you create an option to keep them invisible (or simply exclude the subs from export)?

[0xdeadbeef]

27.06.2009: 3.9.4 -> 3.9.5

• Changed: VobSub+SUP/IFO: fixing completely invisible subtitles is now disabled by default.
  Can be enabled via command line (/fixinv) or via "Settings" menu.
• Changed: CLI: Boolean parameters like "verbatim" can be switched on/off now with trailing +/- (e.g. "/verbatim-").
• Changed: released Bitmap and Palette classes under the Apache License.
• Fixed: GUI: default background color handling missing in "min time" textfield.

[Chetwood]

0xdeadbeef, let me thank you for providing such a great tool for the community. I'm not into HD material just yet but since your tool also processes idx/subs and given your expertise I hope you might be able to find out what's wrong with these sample subs.

I've ripped my original DVDs to HDD with DVD Fab and encoded them to avi with AutoGK. They run fine on my PC and DVD standalones but not on some of the HD media players I'm currently testing. Every other item or so is skipped. Opening them in BDSup2Sub results in this error message "WARNING: Language id en(index:1) inactive -> ignored" which got me confused since english is index 1 and available.

For a test I recreated an idx/sub by loading the AutoGK subs in Subtitle Creator and appending both to a new file. Again, I got the same error message from BDSup2Sub, only this time none of the subs would be displayed on my media player despite both languages being selectable with the remote. So I used VobSubStrip to rip only the german stream and this time there's no error message from BDSup2Sub and the sub is displayed just fine. Would be cool if you could find the time to shed some light on this. Thanks.

[0xdeadbeef]

Well, first of all there are two languages defined in your sample IDX and German is defined as default:

# Language index in use
langidx: 3

...

# English
id: en, index: 1

...

# Deutsch
id: de, index: 3

Since BDSup2Sub can only load one subtitle stream at a time and I was too lazy up to now to add a selection dialog, BDSup2Sub loads the language marked as default and outputs a warning that the other language(s) (in this case: English) was/were ignored. So selecting the German track is valid and BDSup2Sub behaves as described in the manual:

Note that BDSup2Sub ignores all captions but those after the id,index header that fits the language index defined via the "langidx" keyword.

So if you want to show/edit the English track in BDSup2Sub, you just need to change "langidx: 3" to "langidx: 1" in this case.
Note that when exporting such a Vobsub, BDSup2Sub exports only the language it imported.

But back to your problem:
Chances are that some tools or players ignore the langidx field completely and try to display all subtitles defined in the IDX. Since most English and German subtitles share the same display times or at least overlap, this would explain the erratic behavior.
Therefore I'd generally advise not to use VOBSUBs with more than one language if you don't have to and to split them into one Vobsubs for each language instead. This can be also done with BDSup2Sub: you just need to repeat the following steps for each language:
1) edit the "langidx" field manually to the next available language ID
2) load the VobSub into BDSup2Sub
3) export as Vobsub from BDSup2Sub (use same fps, resolution and the keep palette mode)
In the end you'll have one VobSub for each language.

Anyway, adding a language selection mode and command line option is on my list. Yet I dunno when I will implement it. Besides, this will only allow to select the language that is imported, but won't change the behavior of loading (and therefore exporting) only the loaded language track.

[Chetwood]

Alright, thanks for pointing out the RTFM, I've now changed the langidx to check out the english stream of the sub and I'm getting lots of new warnings: "WARNING: Offset to next fragment is invalid. Fixed to:0x00007000". First I thought, this might be causing trouble but the sub created with Subtitle Creator does not come up with such warnings and still does not run properly. So I'm inclined to believe that whatever is causing these problems stems from the sub file. Cause after VobSubstrip's rewrite of both idx and sub file the sub runs flawlessly.

Originally Posted by 0xdeadbeef

Therefore I'd generally advise not to use VOBSUBs with more than one language if you don't have to

Problem is, I do have to cause contrary to my standalones with Mediatek chipset the HD media players with Realtek chipset do only load the first subtitle file and ignore the rest. Thus

Some.movie.avi
Some.movie.de.idx
Some.movie.de.sub
Some.movie.en.idx
Some.movie.en.sub
Some.movie.en.director's.comment.idx
Some.movie.en.director's.comment.sub

would display only the de sub with no option to cycle through the others. Since other 2 language subs I've encoded with AutoGK do run flawlessly as well, something else must be causing trouble. I've added a sample of this to the rar file. Can you think of anything else when looking at those samples? Thanks again.

[0xdeadbeef]

Originally Posted by Chetwood

Alright, thanks for pointing out the RTFM...

You asked for it.

Originally Posted by Chetwood

I've now changed the langidx to check out the english stream of the sub and I'm getting lots of new warnings: "WARNING: Offset to next fragment is invalid. Fixed to:0x00007000".

Yep, this a somewhat precise description of the problem. The (English) subtitle stream is corrupt, but BDSup2Sub fixed it. Unfortunately, there are quite a few tools out there that create corrupt streams. Sometimes even the stream on the DVD is already defective.

Originally Posted by Chetwood

First I thought, this might be causing trouble but the sub created with Subtitle Creator does not come up with such warnings and still does not run properly.
So I'm inclined to believe that whatever is causing these problems stems from the sub file. Cause after VobSubstrip's rewrite of both idx and sub file the sub runs flawlessly.

Well, all that I can tell is that SC creates a valid subtitle stream. If it still doesn't work ok on your player, it's pretty likely that the player has issues.

Originally Posted by Chetwood

Problem is, I do have to cause contrary to my standalones with Mediatek chipset the HD media players with Realtek chipset do only load the first subtitle file and ignore the rest.

And embedding them into the transport stream doesn't work?

Originally Posted by Chetwood

I've added a sample of this to the rar file. Can you think of anything else when looking at those samples? Thanks again.

Dunno what to do with this. Seems to contain all the last samples plus a new one that doesn't have any issues at all.

Sorry, but I dunno if I get what you expect from me. Shall I analyze software bugs in your player by looking at valid streams it doesn't like? That is a bit optimistic to say the least.

[Chetwood]

Originally Posted by 0xdeadbeef

You asked for it.

I wasn't being ironical.

Originally Posted by 0xdeadbeef

And embedding them into the transport stream doesn't work?

I'm not sure I'm getting your point. What transport stream? I'm using avi as a container so I can playback my files on pretty much every player I have. Thus mkv won't do and I hate hardcoded subs.

Originally Posted by 0xdeadbeef

Sorry, but I dunno if I get what you expect from me. Shall I analyze software bugs in your player by looking at valid streams it doesn't like?

Of course not, I was just hoping that given your expertise and likelyhood of being able to also look into the sub files rather than idx files only, you would notice some difference between the files that could explain why some of them work and some don't. What has changed between the AutoGK sub and the VobsubStrip sub?

Since all subs were done with AutoGK initially I thought it would be the one to blame cause I hadn't expected some DVD streams already being defective as you suggested but which is probably true. However, I'm still baffled that some subs don't get flagged as problematic in BDSup2Sub but still don't work on 2 hardware hd players (with different software implementations but apparently the same chipset).

[Jaja1]

I have two requests.

First, could you add an option to center all the subtitles in the lower part of the image with one keystroke.

Second, could you add an option to close the sup you are working on, so that it becomes unlocked for windows.

Thanks in advance. And BDSup2Sub is still working great.

[0xdeadbeef]

Originally Posted by Jaja1

First, could you add an option to center all the subtitles in the lower part of the image with one keystroke.

Well, one keystroke is asking a bit too much, but I think adding a center (and maybe also left/right) button in the "move all" dialog would be possible.

Originally Posted by Jaja1

Second, could you add an option to close the sup you are working on, so that it becomes unlocked for windows.

Hm, do you mean a simple "Close" item in the file menu? That should be possible of course.

[Jaja1]

Originally Posted by 0xdeadbeef

Well, one keystroke is asking a bit too much, but I think adding a center (and maybe also left/right) button in the "move all" dialog would be possible.

Yeah, one keystroke sounds a bit heavy doesn't it
But adding a center button in the "move all" dialog is just what I meant. I should have phrased my request more precise.

Hm, do you mean a simple "Close" item in the file menu?

Yip, that's it.

Nice to see that you can rephrase my questions in such a way that everybody can now understand them

And thanks for your great support.

[0xdeadbeef]

11.07.2009: 3.9.5 -> 3.9.6

• Changed: Added "Close" menu item in File menu.
• Changed: Added horizontal moving of captions (left, right, center)

[Jaja1]

I love you.