Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > General > Subtitles

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th October 2022, 17:08   #1  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
PGS sample that uses cropping?

I am looking for a SUP sample file that use the PGS cropping functionality. Simple as that.

I tried to mimick the description in the PGS patent and BDAV patent but I cannot get it to work at all. Is it even part of the Blu-Ray specs nowadays?

There must be at least one sample around, because the FFmpeg developers are aware that the flag mask is 0x80 (while the forced one is masked with 0x40).
cubicibo is offline   Reply With Quote
Old 23rd November 2022, 16:41   #2  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 416
If anyone has a professionally-produced sample, I'll take a copy as well. I've got a PGS library that I maintain. The best I've come up with for testing cropped windows is synthetic unit tests, but no real-world samples.

cubicibo: If you want to mess with Rust, that library should be correct about generating such window definition segments.

On the other hand, I imagine everyone sees that only the simplest uses of PGS serve any real purpose. It shouldn't be so surprising that part of it would be omitted from whatever production specification is in effect.

Last edited by wswartzendruber; 23rd November 2022 at 16:45.
wswartzendruber is offline   Reply With Quote
Old 23rd November 2022, 18:14   #3  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
I do not know how to program in RUST. if you can produce an attempt that mimics the patent drawings 15 (scrolling) or 16 (wipe-out) that also passes your unit tests, I'll check it on real hardware. I failed to reproduce both. I did get my playback device to start the wipe-in on a sample I produced but it always crashed within one or two pixels in. Object was displayed at the wrong location too. But in most of the time the decoder was crashing before anything would show on screen.

Last edited by cubicibo; 23rd November 2022 at 18:18.
cubicibo is offline   Reply With Quote
Old 24th November 2022, 16:25   #4  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 416
If I may, the patent is not authoritative. I can make that assertion because it doesn't deal with forced subtitles in a manner compatible with commercial implementations.
wswartzendruber is offline   Reply With Quote
Old 24th November 2022, 16:56   #5  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
But forced subtitles are just done using an additional flag not described in the original patent? If a PGS decoder is too old to decode the forced flag, it is ignored and the PCS is decoded normally. There are no fewer or additional data fields inserted whatever the flag is decoded or not, unlike cropping.
cubicibo is offline   Reply With Quote
Old 30th November 2022, 16:25   #6  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 416
I suppose that depends on if you're masking against the flags field, or parsing its value outright.

You have a point.
wswartzendruber is offline   Reply With Quote
Old 20th March 2023, 13:56   #7  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
While I can't test any of this at the moment, I think I understood how a stream reproducing a cropping operation is supposed to be structured.
The key is the WDS segment. this segment has the untold property to trigger a refresh of the graphic plane according to the compositions parameters. The patent is voluntarily misleading when it claims that windows are a (fixed) view on the graphic plane. I rather think that windows are just defining a visibility mask evaluated within a DS. The reasons that leads me to believe so are the following:
- A mask is the cheapest hardware implementation. And that would match the overall philosophy of the PGS format.
- The absolute need to provide a WDS anytime an on-screen operation with the compositions is performed. A display set made of [PCS, END] is useless!
- The possibility to change windows in the middle of an epoch. The patent does not recommend it, but does not forbid it.
- The fact that normal case display sets to end an epoch always provide the WDS. But the content of the WDS absolutely does not matter to undisplay objects.

So the structure would be the following:
[PCS(epoch start), WDS, PDS, ODS, END] <-[epoch start]
[PCS(normal case), WDS, (PDS?), END] <- composition with crop params #0
...
[PCS(normal case), WDS, (PDS?), END] <- composition with crop params #n
[PCS(norma case), WDS, END] <- Undisplay [end epoch]
I just don't understand why the patent insist on providing PDS#0 in every display set.

Here is a trial: w/o PDS, w/ PDS
Some subtitle should show at t=5s, then the subtitle is cropped to 1/4 of its area at t=8s. The epoch ends at t=11s.
To test, mux the sup file with a 1080p23.976 Blu-Ray compliant video file and play back on a hardware player. BDMV, AVCHD and .m2ts can be played back from USB sticks on some Sony and Panasonic players. This will not work on software players: FFmpeg does not implement cropping.

EDIT: cropping confirmed to work. Thanks to Prince 7 for testing.

Last edited by cubicibo; 21st March 2023 at 21:22.
cubicibo is offline   Reply With Quote
Old 22nd March 2023, 08:57   #8  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,565
Ah, nice !
Muxed in tsMuxeR as AVCHD, played from USB in Oppo 205:
H-g2.sup: cropping performed, H-ge.sup: no subs presented.
Sample text is presented in upper left half only, and timing is a bit too short to activate subtitles and make a proper sub selection,
Samples with more text and maybe a grid over 1920x1080 would be helpful.
HDMI handshake eats 2..3sec before accepted result becomes visible...

(MPC-HC, MPC-BE: both sups are presented, cropping not performed, as perceived).
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."
Emulgator is offline   Reply With Quote
Old 23rd March 2023, 00:21   #9  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
Thank you for the additional testing. I am just surprised by your statement with "H-ge.sup", because epoch start bytestream is strictly equal in the two streams. It only changes at the displayset shown at t=8s. Maybe the stream packets are arranged differently and you are not able to enable it on time before it is decoded.
Sorry for the test simplicity, I wrote this bytestream by hand with SUPer. So all values were chosen to be easy like coordinates (20, 20) or the graphic used.

Here is a totally different test with cropping. This should perform what the patent calls "scrolling", it is simply in-place scrolling of the ODS content. Effect starts at t=15s, ends at t=44s. Image should scroll vertically, one pixel per frame, without flickering within a window in the center of the image.

I expect "heavy_scroll_{}.sup" tests to fail and "test_crop_simple_{}.sup" to have more chance to work. Prioritise the "nodts" versions but trying others is highly appreciated. I may have messed up the streams when coding them with SUPer so both may fail. Mux parameters are the same (1080p23.976).

Last edited by cubicibo; 23rd March 2023 at 00:24.
cubicibo is offline   Reply With Quote
Old 24th March 2023, 04:50   #10  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,565
Muxed all 6 sup side-by-side together with a black framecounter video using tsMuxeR into BD-folder
Oppo 205 .m2ts play from USB-Stick:

Congratulations, Success !
All six .sup do work as described: smoothly upwards scrolling content within a centered window without stutter.

Switching sub tracks during playback leads to blanking of the old sub track and the new sub track not coming up.
One has to rewind back to before 15s to get the new sub track switched on.
Maybe there is kind of a refresh command needed, per GOP, or per second, to obtain the usual BD subtitle behaviour
(not visible during FFWD/REW, and after FFWD/REW coming up only in PLAY with the next GOP, like 1..2sec later)

(I guess my previous results were spoilt because my, the remote, and HDMI reactions
were too slow to get that done properly within the previous short sample duration)

MPC-HC and -BE do not show these scrolling subpics at all.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."

Last edited by Emulgator; 2nd April 2023 at 11:06.
Emulgator is offline   Reply With Quote
Old 24th March 2023, 09:04   #11  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
Thank you really much! So scrolling subtitles is easy (well…)

Changing subtitles will never refresh the display instantaneously with cropping. This is because the cropping operation is done on existing objects in the PG buffer (the object you see is defined only once in the stream). As you switch, you land in the middle of the epoch and there are no objects to crop, so all subsequent segments are ignored. If you want more frequent hooks, the stream needs to contain acquisition points. But then, the object must be decoded, copied and drawn again and this can take quite a few frames! If your scrolling has a pause, you can do it. If it does not, it is not possible without steutterred movement. Of course, with real subtitles, scrolling will last a handful of seconds, not 20. so there will be frequent hooks.

FFmpeg decoder checks that the displayed object is not larger than the dimensions declared in the PGS. Since 1150 > 1080, (image is 800x1150 I think) it aborts decoding and drops the data. I will try to send a patch to FFmpeg to implement cropping.

BTW, this is the maximum refreshable area when cropping is performed:
MaxArea = 32e6/(2*(fps/k))
For 24p, refreshing every frame (k=1) is limited to 666,500 pixels (e.g 815x815). If you want to update more area, you must scroll every other frames to increment the k factor.
The rule of thumb is 1/4 screen area of a 1920x1080p29.97.

I will try to add scrolling and cropping detection in the bdn.xml convert pipeline of SUPer. Hopefully in a few months.

Last edited by cubicibo; 24th March 2023 at 10:25.
cubicibo is offline   Reply With Quote
Old 24th March 2023, 16:46   #12  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 1,565
Of course, now it becomes clear to me:
Refresh can only come with the next subpicture, and that timing has nothing to do with the next GOP.

As I switch sub streams any rendering can only start after the arrival, buffering and decoding of the next subpicture,
there could not have been anything already buffered from that stream I intended to switch to.
so: Nothing to display for the duration of the recent subpicture I switched into,
and that is to be expected for any animation of already buffered/unbuffered content too of course...
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..."
Emulgator is offline   Reply With Quote
Old 26th March 2023, 22:56   #13  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
Here's the updated pgssubdec.c to add PGS cropping support in FFmpeg.

(And I just realised the samples i sent had the wrong colors, used raw RGB values for YUV :^))

Last edited by cubicibo; 26th March 2023 at 23:04.
cubicibo is offline   Reply With Quote
Old 30th March 2023, 14:31   #14  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
Someone definitely watching this thread was faster to submit a patch to FFmpeg, oh well. If they are kind enough to reply here, where does the 4096 width and height limit hard coded in the patch comes from? I rather believe any dimensions can be used as long as the bitmap fits in the 4 MiB PG object buffer (minus all other slots allocated, of course, but let's not complicate things).

Furthermore, a limit of 4096 prevents full-width scrolling of a presentation object, where the said object is padded horizontally by 1920 pixels so the full object would have a width of 3*1920 = 5760 pixels (height would be limited < 347 pixels to scroll every frame). So I definitely believe it to be incorrect.

I shall also add something about the WDS segment as it is on topic. For very basic epoch structure with only 2 WDS (one in the first display set and one in the last one to undisplay), it is acceptable to provide no window definition in the last one. This is because the WDS swaps the two graphic planes and one of them is empty. If the epoch is more complex with display refresh or updates, All subsequent WDS after the epoch start will hide a graphic plane with content in it. If the windows changes in any of those WDS, left over graphics from past display set may remain and be displayed. This is why the patent does not forbid windows change but strongly imply they should be constant throughout the epoch.
And lastly, palette updates (only valid on single object) don't seem to be constrained by any bandwidth in the PG decoder, not even Rc (256 Mbps). They are instantaneous re-evaluation of the displayed graphic plane with the CLUT block. So only palette updates can do frame by frame effects on full screen bitmaps.

Last edited by cubicibo; 30th March 2023 at 14:53.
cubicibo is offline   Reply With Quote
Old 31st March 2023, 10:07   #15  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,355
Quote:
Originally Posted by cubicibo View Post
where does the 4096 width and height limit hard coded in the patch comes from? I rather believe any dimensions can be used as long as the bitmap fits in the 4 MiB PG object buffer (minus all other slots allocated, of course, but let's not complicate things).
Its not my patch, but from the BD PGS specification:
Code:
object_width: This field specifies the width of this Object in pixels. 
    All Objects shall be equal to or greater than 8 pixels in width. 
    Objects in HDMV Presentation Graphics streams shall have a width equal to or less than 4096 pixels. 
    Objects in HDMV Interactive Graphics streams shall have a width equal to or less than video_width. 
object_height: This field specifies the height of this Object in pixels. 
    All Objects shall be equal to or greater than 8 pixels in height. 
    Objects in HDMV Presentation Graphics streams shall have a height equal to or less than 4096 pixels. 
    Objects in HDMV Interactive Graphics streams shall have a height equal to or less than video_height.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 31st March 2023, 13:25   #16  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 110
Of course, it is much easier with BD specifications at hand. Thank you.
So it is not possible to perform scrolling across the entire video width in FHD. A pity.

Last edited by cubicibo; 31st March 2023 at 13:35.
cubicibo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.