xy-VSFilter Project (High Performance VSFilter Compatible Subtitle Filters)

[Telion]

OK, thank you.

[TheSkiller]

Originally Posted by sneaker_ger

At least Blu-Rays even support simple effects like fading for PGS subs without creating a new picture for every frame of the effect. Also, some RLE compression is used (and zlib in mkv, though optional).

Just for the record, DVD has a fade feature as well (which uses just one subpic for the period of a subtitle) and the bitmaps are RLE compressed as well.
There are also wipe effects (used for karaoke).


One thing to consider when choosing to make subtitles "white" is if you intend to display them on a TV you should not make them 100% opaque and 100% white because it will look blindingly bright if the movie background is dark and you're watching in a dark room.
Better use a light grey color and make the subtitles slightly translucent so that they can adapt to the brightness of the movie background to some degree. My recommendation based on my exhaustive subtitle creations is a color of around 230,230,230 and an opacitiy of 66-75%.

When it comes to visibility I'm 50/50 about white vs yellow. Both can be read equally well from all kinds of backgrounds if only the (black!) outline is thick enough which from what I've seen is rarely the case unfortunately. My recommendation for outline width: about as thick as a stroke of the characters used.

[filler56789]

Originally Posted by Moragg

I normally watch anime, and yellow just doesn't work - since the backgrounds are often bright and have a similar texture it clashes badly (in my eyes) and makes it harder to read.

The argument is a lot stronger for live-action movies, but even there I find them too intrusive, and white wouldn't upset my viewing as much. But here I would also say that yellow could be a better option in many circumstances.

Ideally, the "releasers" should offer BOTH white and yellow(/green/whatever) subtitles. And just for the notes again, it's not a matter of "anime vs. live action", it's just the fact that different eyeballs and/or different visual cortices will perceive brightness, contrast and color saturation differently

But I agree, it would be very-nice if (xy)VSFilter had an option for easily overriding the color palette from the IDX files. While this doesn't happen, you could give a try to Mplayer --- because Mplayer completely ignores the color palette from the bitmap subs and gives you "simple white fonts" all the time

[michkrol]

Thanks for your effort to bring us this software, works great.

While testing various files, I've encountered a rather curious behavior. I have a file that has crazy subs effects (around 10k lines in ~7 seconds), after adjusting madVR CPU queue to 24 this effects plays fine on my PC, with no dropped frames, but ~25 seconds later (with only simple text on screen) the playback freezes completely for a few seconds, also freezing the player controls, the sound plays fine. The playback then recovers with ~150 dropped frames.

What's curious is it only freezes if you play the file without pausing and seeking. If you seek somewhere between the heavy effects and the moment it freezes the playback goes over this part flawlessly as expected. It might be caused by the caches being emptied all at once.

I'm not opening a bug report, since I would like to confirm first, whether it's XySubFilter or madVR causing this.

Below is my system config:

Intel Core i5-3570K
Intel HD4000
16GB RAM
Windows 7 x64
MPC-HC 1.7.0.7646 (f6f1b82) (official nightly build)
LAVFilters 0.58.1 (official release)
XySubFilter 3.0.546 (official beta release)
madVR v0.86.10 (official release)

How to reproduce the problem:

1. Set madVR CPU queue to 24, to get smooth subtitles effects playback. (Of course it will differ between PCs, just don't set it to max.)
2. Open the file in windowed MPC-HC at movie resolution (1280x720).
3. Open the file.
4. The title with effects plays smoothly.
5. Continue playback without pausing and seeking.
6. Around 0:33 playback freezes, recovers for a split second, and freezes again. This takes around 7 seconds in total for me.
7. Playback recovers and the rest plays smoothly.

Link to the file in question:

http://www.mediafire.com/download/cl9plddqpfa0c80/Shingeki_subs_test.zip

I've replaced the video with something pointless (with same parameters as original) and file is zipped to save bandwidth (the subs are over 35MB and compress quite well).
I know the subs in this file are crazy, but inspecting this might help improve performance in other extreme cases.

Thanks for your time and please tell me, if I've overlooked some important info.

[sneaker_ger]

I can reproduce the performance drop at 0:33 - really strange.

[Soukyuu]

I must admit I snickered when I saw the filename. So I guess Commie's typesetting make intel CPUs stutter as well :P
Before that, Koharubi's bakemonogatari ep02 @1080p was my benchmark for subtitle renderer performance, but xySubFilter masters that one thanks to the cache (it had a half second alternation of two typesets covering the whole screen)

[turbojet]

Originally Posted by cyberbeing

You were using xy-VSFilter (VSFilter.dll) then? The reason this was disabled in xy-VSFilter is because the existing VSFilter override behavior for how this was triggered and applied to ASS subtitles was rather annoying.

Have you tested our new style override dialog for ASS subtitles in XySubFilter?

It was subfilter, forcing default style had no effect, is there something I'm missing?

In xy-VSFilter (VSFilter.dll) no, there are no plans to add resize functionality for image-based subtitles.

With XySubFilter, madVR is in charge of resizing all image based subtitles.

Right but madvr needs to be instructed to zoom in/out and subfilter needs an option for zoom level, for example: Double the size of image subs zoom 2.0x, reduce them by half, zoom 0.5x

Wouldn't it make more sense to keep all subtitle options out of madvr to reduce user confusion? There currently is one but wouldn't it be better for subfilter to instruct madvr how to handle that code instead?

[ikarad]

What are the differences between xy-vsfilter and Xysubfilter?

[Moragg]

Originally Posted by michkrol

Link to the file in question:

http://www.mediafire.com/download/cl9plddqpfa0c80/Shingeki_subs_test.zip

I've replaced the video with something pointless (with same parameters as original) and file is zipped to save bandwidth (the subs are over 35MB and compress quite well).
I know the subs in this file are crazy, but inspecting this might help improve performance in other extreme cases.
.

I have the original version of that - 90K lines in 7 secs

But if you want help, press ctrl+j to bring up madvr info, and look down to the queue information. During playback watch the queues, problems usually occur when they fall to very low values (0) - you'll likely find frames dropped/delayed when that happens. If so, try increasing those queue sizes.

If not it may be a memory flushing thing which I have no clue whatsoever about.

Edit: I can reproduce the problem, both in the file you gave me and in Commie's Shingeki ep1v2 - the one with 90k lines - a completely different file. Same thing happens, video freezes and subtitle queue just empties. Other queues are fine.

Originally Posted by ikarad

What are the differences between xy-vsfilter and Xysubfilter?

Xy-vsfilter is the old one and has a stable release, xy-subfilter is the new, improved version but is currently in beta testing, and can only be used with madvr.

[ikarad]

Originally Posted by Moragg

Xy-vsfilter is the old one and has a stable release, xy-subfilter is the new, improved version but is currently in beta testing, and can only be used with madvr.

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?

[Blight]

This is just a heads-up to Zoom Player users,
I plan to add support for this interface in the next release, you will probably see a beta within the next month as I'm currently focusing most of my time on Zoom Commander and other non-coding chores.

[Moragg]

Originally Posted by ikarad

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?

1)not as far as I know. Hasn't for nearly a year, with xysubfilter out I doubt anyone will bother.
2)No.
3)Not entirely sure, but perhaps: better colour accuracy, faster. I know it renders subs at a different res. And in madvr you can make it optimize performance if you have a slow CPU.

[Asteague]

Any reason why xySubFilter would work flawlessly for files with subtitles in the container, but refuses to show up for files with only external subs? In some rare cases it does show up, but I have no idea why it won't always show up.
Any ideas?

My setup:
Win 8 x64
MPC-HC 1.7.0.7646
madVR 0.86.10
LavFilters 0.58.1
xy-vsFilter not installed (didn't work even if installed)

[cyberbeing]

Originally Posted by michkrol

Thanks for your effort to bring us this software, works great.

While testing various files, I've encountered a rather curious behavior. I have a file that has crazy subs effects (around 10k lines in ~7 seconds), after adjusting madVR CPU queue to 24 this effects plays fine on my PC, with no dropped frames, but ~25 seconds later (with only simple text on screen) the playback freezes completely for a few seconds, also freezing the player controls, the sound plays fine. The playback then recovers with ~150 dropped frames.

Issue #159

I noticed this exact same issue with the original script in your sample the other day. Not sure what causes it at the moment, we are looking into it.

I'll attach the sample you provided to the issue, but at this point I'm more interested in obtaining any other scripts which are able to trigger this bug.

[sneaker_ger]

Originally Posted by ikarad

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?
3) What are the ameliorations in comparaison with xyvsfilter?

Originally Posted by Moragg

1)not as far as I know. Hasn't for nearly a year, with xysubfilter out I doubt anyone will bother.
2)No.
3)Not entirely sure, but perhaps: better colour accuracy, faster. I know it renders subs at a different res. And in madvr you can make it optimize performance if you have a slow CPU.

1.) Development of xy-vsfilter has been continuing in the background, even though the last official version is 8 months old. It is not dead and new official versions will come.
2.) Correct. (no)
3.)
The main difference between the two is that xy-vsfilter sits in the DirectShow chain as a link between video decoder and the video renderer. It overlays subtitles onto the video frames it receives from the video decoder and then passes the result on to the video renderer.
This is has following disadvantages for the typical end-user:
- the subtitles will be rendered at the video resolution (and with chroma sub-sampling if applicable), so for videos whose resolutions are much lower than the display resolution they may look blurry or aliased.
- DXVA2 (native) does not work because it requires a direct connection between the video decoder and the video renderer.
- rendering in the black bars can be problematic, because xy-vsfilter does not really know the target rectangle and changing resolutions during runtime may introduce some problems.

XySubFilter on the other hand renders at the display resolution (without chroma sub-sampling) by default, which makes the subtitles always look sharp. It is not connected to the video decoder at all, but only sits between the splitter and the renderer. Since the video decoder can now directly connect to the video renderer again DXVA2 (native) will work fine.

Concerning colors: xy-vsfilter will also have "correct" color handling (similar to XySubFilter) - or rather has the most important stuff already. This is more of an advantage of XySubFilter over MPC-HC's internal renderer (this is related to historic limitations of the old vsfilter, which didn't know anything but limited range BT.601 colors, resulting in obscure workarounds which break colors with MPC-HC's internal renderer/VLC and a bunch of others. A new extension of the ASS "spec" is supposed to harmonize the different renderers again to some extent.)

Any performance differences should ideally only be related to the different resolutions/color spaces at which they render by default - so new optimizations will likely be used in both versions. The madVR performance trade off option should not make XySubFilter faster than xy-vsfilter - it just relegates some color conversion stuff from madVR to XySubFilter, which uses some simpler conversion on the CPU instead of the GPU like madVR (nothing to worry about - that's why it's on by default. cyberbeing's more technical explanation). xy-vsfilter always has to do these things itself since it does not pass on the subtitles through a separate pin to the renderer but overlays them onto the video itself.

About rendering in the black bars:
While the new subtitle interface that connects XySubFilter and madVR has been designed with enabling subtitle rendering in the black bars in mind (if the user wishes so), this has not yet been implemented.

[cyberbeing]

Originally Posted by turbojet

It was subfilter, forcing default style had no effect, is there something I'm missing?

XySubFilter only overrides the styles defined in the header of a script, it does not strip style override tags. Is that it?

If not, please provide a sample which easily reproduces the issue.

Originally Posted by turbojet

Right but madvr needs to be instructed to zoom in/out and subfilter needs an option for zoom level, for example: Double the size of image subs zoom 2.0x, reduce them by half, zoom 0.5x

madVR doesn't need to be instructed. XySubFilter always passes bitmap subtitles at original resolution and madVR scales them as needed automatically.

Originally Posted by turbojet

Wouldn't it make more sense to keep all subtitle options out of madvr to reduce user confusion? There currently is one but wouldn't it be better for subfilter to instruct madvr how to handle that code instead?

I'm a bit confused. If you are requesting that madVR be instructed to apply arbitrary non-default scaling to bitmaps, our subtitle interface does not support that currently. I'd suggest explaining your feature request to madshi in the madVR thread.

Originally Posted by ikarad

What are the differences between xy-vsfilter and Xysubfilter?

XySubFilter
Requires madVR (or other Subtitle Consumer supporting SubRenderIntf.h)
Higher Quality
RGB Subtitles
Renders subtitles after scaling is performed by the video render (allows high resolution subtitles)
Subtitles alpha-blended on the GPU
Image based subtitles are scaled to any video resolution
Works with DXVA Decoding
Allows Deinterlacing via bitstream flags
Compatibility option to support anamorphic video
Allows all colorspaces supported by the video renderer to be used

xy-VSFilter (VSFilter.dll)
Works with any video renderer.
Lower Quality
Usually YCbCr Subtitles
Renders subtitles prior to scaling at original video resolution
Subtitles alpha-blended on the CPU
Image based subtitles are not scaled and must match video resolution
Breaks DXVA Decoding
Blocks Deinterlacing bitstream flags
Requires all subtitles scripts be compensated for VSFilter behavior with anamorphic video
Only a more restrictive set of colorspaces supported by xy-VSFilter can be used by the video renderer

Originally Posted by ikarad

1) xyvsfilter will never updated anew?
2) If I want to use xysubfilter is it necessary to have xyvsfilter installed or not?

1) An official update to xy-VSFilter (VSFilter.dll) is planned after we release a stable build of XySubFilter. Though after that point, the future of legacy VSFilter.dll releases from us is unknown.

2) Update to the latest MPC-HC x86 nightly build and both filters can co-exist, but it is not required to have both installed. XySubFilter will be used with madVR, while xy-VSFilter will be used by all other renderers.

Originally Posted by Asteague

Any reason why xySubFilter would work flawlessly for files with subtitles in the container, but refuses to show up for files with only external subs? In some rare cases it does show up, but I have no idea why it won't always show up.
Any ideas?
...
xy-vsFilter not installed (didn't work even if installed)

Did you remember to disable the MPC-HC ISR by unchecking "Auto-load subtitles" under "Playback" in MPC-HC settings?

Are you positive the external subtitle files have an identical file name to your video? Please double-check by renaming one to the other with copy/paste (not including file extension).

[turbojet]

Originally Posted by cyberbeing

XySubFilter only overrides the styles defined in the header of a script, it does not strip style override tags. Is that it?

If not, please provide a sample which easily reproduces the issue.

http://www.sendspace.com/file/bt4zd8 I'm not 100% sure what format these subs are in, it's been awhile. They have color but extract fine as srt, could be colorized srt. Known srt subs tested style fine in subfilter.

madVR doesn't need to be instructed. XySubFilter always passes bitmap subtitles at original resolution and madVR scales them as needed automatically.

This is concerning the question about considering zooming bitmap subs in the future, many are very small. No subtitle renderer does it yet but I'm pretty sure it's possible to do it like bdsup2sub does. Madvr might be able to help out but the options wouldn't belong in madvr.

I'm a bit confused. If you are requesting that madVR be instructed to apply arbitrary non-default scaling to bitmaps, our subtitle interface does not support that currently. I'd suggest explaining your feature request to madshi in the madVR thread.

As vanilla madvr has nothing to do with subtitles doesn't subtitle options such as the current optimize seem out of place? I'm suggesting have the option in xysubfilter that triggers code in madvr.

[Aleksoid1978]

Question to developers:
1 - can you merge this 2 menu items section :

Then it would be convenient to switch the built-in and external subtitles in the menu
2 - Where can be find source(full or just for interfaces)? It's need for better integration with MPC-BE.

[Toku]

I have a file that causes both MPC and madVR to crash when using XySubFilter but works fine when using xy-VSFilter, no crash occurs when subtitles are disabled

10 Second sample- here

[cyberbeing]

Originally Posted by turbojet

http://www.sendspace.com/file/bt4zd8 I'm not 100% sure what format these subs are in, it's been awhile. They have color but extract fine as srt, could be colorized srt. Known srt subs tested style fine in subfilter.

Yes, those are SRT subtitles. I don't see the problem though, our style override dialog functions just fine on that sample.

Override Style
1) Open your sample in MPC-HC

2) Pause video

3) Right-click MPC-HC window -> Filters -> XySubFilter -> Properties

4) Click "Styles"

5) Click the "Default" tab

6) Edit to bogus value like "Border Width = 10"

7) Click "Ok" to style Dialog

8) Click "Apply" to main setting dialog [note: the need to press Apply here first seems to be a bug]

9) Click "Okay to main setting dialog

10) Style with "Border Width = 10" is applied to subtitles.

OR

Force Default Style
1) Open your sample in MPC-HC

2) Pause video

3) Right-click MPC-HC window -> Filters -> XySubFilter -> Properties

4) Click "Styles"

5) Click the "Global Default" tab

6) Edit to bogus value like "Border Width = 10"

7) Click "Ok" to style Dialog

8) Check "Force Default" in main setting dialog

9) Click "Okay to main setting dialog

10) Style with "Border Width = 10" is applied to subtitles.

11) Exit MPC-HC

12) Re-open your video

13) "Border Width = 10" is still applied to subtitles

Originally Posted by turbojet

This is concerning the question about considering zooming bitmap subs in the future, many are very small. No subtitle renderer does it yet but I'm pretty sure it's possible to do it like bdsup2sub does. Madvr might be able to help out but the options wouldn't belong in madvr.

We currently have no intentions of adding a bitmap resizing function within XySubFilter. The actual resizing code should exist in madVR, even if we ourselves trigger use of such code.

Originally Posted by turbojet

As vanilla madvr has nothing to do with subtitles doesn't subtitle options such as the current optimize seem out of place? I'm suggesting have the option in xysubfilter that triggers code in madvr.

madVR has much to do with the subtitle interface used by XySubFilter. Unless madVR adds such code to madVR, and offers a way for us to trigger such functionality, there is nothing we can do. You need to make a case to madshi for why he should support such a feature in madVR and the subtitle interface.