Doom9's Forum - View Single Post

mkver · 25th May 2018, 09:21

I see what the issue is. But first some information: The H.264 standard knows two kinds of "packets" (if you want to call them) called SPS (Sequence Parameter Set) and PPS (Picture Parameter Set). They contain information that is essential for the decoding process (like how many reference frames there are, what the picture dimensions are). They can be several of them simultaneously (each PPS has a PPS id, each SPS a SPS id) and every picture uses exactly one of them during decoding (which one is written in the header of the picture (or actually "slice")). If the bitstream contains a PPS with a PPS id that is already known, the new PPS instantaneously overwrites the old one (with the same PPS id) and is used thereon.
Your file contains one constant SPS and PPS with id in the range [0..3]. And they change in the middle of a GOP (GOP = Group Of Pictures; a keyframe by definition starts a GOP and end the last one (if there was one) so that a GOP consists of the frames between two keyframes (the last keyframe is part of the GOP it starts, not of the GOP of the preceding keyframe)). If you instruct tsmuxer to insert SEI and VUI data, then it seems to strip the SPS and PPS that were in the bitstream away and replaces them with new ones. Said new ones are prepended to keyframes (not to the frames they were originally in) and they contain all the SPS and PPS that existed at the time of said keyframe. When a PPS is overwritten mid-GOP, the file produced by tsmuxer doesn't contain the new PPS when it is needed, because the last keyframe only contains the old, non-overwritten PPS.
Can this be fixed? Hypothetically, yes. Practically, no: Whenever you get decoding errors, you can try another PPS. Where to get the other PPS? Encoder that change SPS/PPS midstream usually use only a few values for each PPS (in your case, I counted four PPS with id 0, five with id 1, four with id 2 and two with id 3) and change between these values. The best guess for the PPS is probably the PPS that is prepended to the next keyframe. But honestly, it's probably not worth the effort.
PS: There is actually a second bug: recovery_point SEI messages are discarded by tsmuxer which means that a lot of keyframes of the input end up not being keyframes of the output (but tsmuxer still prepends SPS and PPS in front of them).

25th May 2018, 09:21	#9 \| Link
mkver Registered User Join Date: May 2016 Posts: 197	I see what the issue is. But first some information: The H.264 standard knows two kinds of "packets" (if you want to call them) called SPS (Sequence Parameter Set) and PPS (Picture Parameter Set). They contain information that is essential for the decoding process (like how many reference frames there are, what the picture dimensions are). They can be several of them simultaneously (each PPS has a PPS id, each SPS a SPS id) and every picture uses exactly one of them during decoding (which one is written in the header of the picture (or actually "slice")). If the bitstream contains a PPS with a PPS id that is already known, the new PPS instantaneously overwrites the old one (with the same PPS id) and is used thereon. Your file contains one constant SPS and PPS with id in the range [0..3]. And they change in the middle of a GOP (GOP = Group Of Pictures; a keyframe by definition starts a GOP and end the last one (if there was one) so that a GOP consists of the frames between two keyframes (the last keyframe is part of the GOP it starts, not of the GOP of the preceding keyframe)). If you instruct tsmuxer to insert SEI and VUI data, then it seems to strip the SPS and PPS that were in the bitstream away and replaces them with new ones. Said new ones are prepended to keyframes (not to the frames they were originally in) and they contain all the SPS and PPS that existed at the time of said keyframe. When a PPS is overwritten mid-GOP, the file produced by tsmuxer doesn't contain the new PPS when it is needed, because the last keyframe only contains the old, non-overwritten PPS. Can this be fixed? Hypothetically, yes. Practically, no: Whenever you get decoding errors, you can try another PPS. Where to get the other PPS? Encoder that change SPS/PPS midstream usually use only a few values for each PPS (in your case, I counted four PPS with id 0, five with id 1, four with id 2 and two with id 3) and change between these values. The best guess for the PPS is probably the PPS that is prepended to the next keyframe. But honestly, it's probably not worth the effort. PS: There is actually a second bug: recovery_point SEI messages are discarded by tsmuxer which means that a lot of keyframes of the input end up not being keyframes of the output (but tsmuxer still prepends SPS and PPS in front of them).