About "Skipping Identical Frames" on Eac3to?

[Columbo]

Originally Posted by tebasuna51

but with AC3 frames (32 ms) was more necessary

Exactly! And that's why DG stuff uses the "Bresenham's approach", it works with all formats.

[hubblec4]

Many thanks for this link and info.

Yes, for TrueHD is it more complex. The reason is the very short duration for TrueHD frames (1/1200).
One frame alone is it really noticeable (a real question, no snark)?
I don't know how many TrueHD frames are needed to be noticeable.

In principle, the Blu-ray creators could have simply omitted the last few frames that follow the final video frame.
But I suspect they consider several TrueHD frames together as a single "meaningful frame."

That's why there are so many TrueHD frames.
Okay, so we can say: there can be identical frames for TrueHD streams.


I'll order the Blu-ray with my next movie order, because this looks very interesting.

[Columbo]

CARS_2 has 94 segments IIRC, or was it MONSTERS.

1/1200 * 94 = ~78ms

People would not be wanting gaps correction with THD if it were not an issue.

Even without identical frames there are issues, because each segment does not end cleanly with end of video and end of audio coinciding. As tebasuna pointed out, with a 32ms frame size, this quickly accumulates. The DG approach handles this with a single algorithm for all formats. The only difference is what you can cut without breaking the stream.

For a player it doesn't matter, because the PTS timestamps can be used to qualify the playback across the seam. But when demuxing ES the timestamps are lost.

[hubblec4]

Originally Posted by Columbo

CARS_2 has 94 segments IIRC, or was it MONSTERS.

1/1200 * 94 = ~78ms

OK. Interesting, 78ms are very long in relation to other audio frames.

Originally Posted by Columbo

People would not be wanting gaps correction with THD if it were not an issue.

Yes. There are issues and we need solutions. One for demuxing, and one for direct stream copy,write to MKV.

Originally Posted by Columbo

Even without identical frames there are issues, because each segment does not end cleanly with end of video and end of audio coinciding. As tebasuna pointed out, with a 32ms frame size, this quickly accumulates. The DG approach handles this with a single algorithm for all formats. The only difference is what you can cut without breaking the stream.

Yes, that's true; whenever a film consists of multiple segments, there are problems with the transitions.

But this problem can be directly solved when creating an MKV file; or rather, it doesn't occur in the first place because the frames, audio, and video always retain their exact timestamps.

Originally Posted by Columbo

For a player it doesn't matter, because the PTS timestamps can be used to qualify the playback across the seam. But when demuxing ES the timestamps are lost.

Yes, exactly, that's the difference. Missing timestamps are always a problem, of course.

-----------------
A general question about the algorithm for removing audio frames.

Since you're familiar with this, could you perhaps confirm the following or correct me if necessary?

At the end of each M2TS file, the algorithm surely checks how much excess audio has accumulated.

If it reaches (or almost reaches) the duration of a video frame, an audio frame is discarded. I'd like to know which audio frame is discarded here: the one from the current M2TS file or the next one?

After the audio frame is removed, it's still very unlikely that the audio and video will be exactly the same length. A small offset of 1ms to 10ms is certainly not uncommon.

Let's say that an audio frame can be removed after the second M2TS file, and we cut the last audio frame from the current M2TS file.
However, an offset of 4ms remains.

This means that the first audio frame of the third m2ts file is no longer synchronized with the first video frame of the third m2ts file. All audio frames from the third m2ts file now have a 4ms offset to the video frames.

Is this correct?

[Columbo]

The accumulated video and audio times are maintained. When the audio excess passes the duration of 1/2 an audio frame, the audio stream is marked for needing a deletion. That causes the next audio frame to be deleted.

The cut will have made things such that the current state is that accumulated vid duration > audio duration, and the algorithm won't get triggered again until enough excess audio has again accumulated. Just like the Bresenham line drawing algorithm hugging the true line, the gaps algorithm hugs a desync of ~1/2 audio frame duration. It does not necessarily delete a frame at every gap but rather only when needed.

In practice pathologies occur that make the audio excess larger than 1 frame so the algorithm allows for multiple frames to be deleted if needed.

Of course, it's not "perfect" but it works well in practice. And it is applicable to all formats.

domy's idea was to try to make things perfect. But it can't be perfect for various reasons, and if you try to salvage the idea, edge cases proliferate and make things quite complex. For example, audio content can be content "identical" but not bit identical (think of low-order bit differences). So now to detect duplicated content you need a fuzzy comparison of some kind, negating to some extent the idea of perfection. You can see by looking at the issues at his git (https://github.com/domyd/mlp/issues) that he abandoned trying to keep things working in the face of edge cases and pathologies, and stopped development. DG's algorithm is simple and robust and works well enough to be quite useful. We've been maintaining our tools since 2003.

Improvements have been suggested and we're happy to consider them but so far nothing has been proposed that justifies the added complexity. One criticism that has been leveled is that if the video exceeds the audio at gaps then correction is never triggered, but for bluray we have never seen that in practice and it may be counter to the spec in any case.

Now you have all our secrets! Watch me get fired.

I understand that your method for MKV in effect retains the timestamps for things, which in conjunction with some mkvtoolnix magic and expected behavior of the players allows you to bypass the gaps problem. That is a creative approach, but doesn't work for demuxing ES, where timestamps are lost. You acknowledged that when you said:

"There are issues and we need solutions. One for demuxing, and one for direct stream copy, write to MKV."

I note that MKV is not the only target format, and that ES is often needed for other things than immediate playback.

[hubblec4]

Originally Posted by Columbo

The accumulated video and audio times are maintained. When the audio excess passes the duration of 1/2 an audio frame, the audio stream is marked for needing a deletion. That causes the next audio frame to be deleted.
.......
Of course, it's not "perfect" but it works well in practice. And it is applicable to all formats.

Thank you very much for the explanations.
I have one small comment about this.
The Blu-ray specs state that a new m2ts file must always begin with a video and audio keyframe.
Therefore, I'm wondering if it wouldn't be better NOT to delete the next audio frame, because it definitely/usually/should be a keyframe.

Originally Posted by Columbo

domy's idea was to try to make things perfect. But it can't be perfect for various reasons, and if you try to salvage the idea, edge cases proliferate and make things quite complex. For example, audio content can be content "identical" but not bit identical (think of low-order bit differences). So now to detect duplicated content you need a fuzzy comparison of some kind, negating to some extent the idea of perfection. You can see by looking at the issues at his git (https://github.com/domyd/mlp/issues) that he abandoned trying to keep things working in the face of edge cases and pathologies, and stopped development. DG's algorithm is simple and robust and works well enough to be quite useful. We've been maintaining our tools since 2003.

Yes, perfection is always a tricky thing. One can invest a lot of time and energy and still not achieve success.
And I've also been dealing with this problem since 2002. It took me a lot of time to understand why the error occurs and where in the processing chain something is wrong.
I never would have thought it was MTX that wasn't working optimally. It's not an MTX fault, but Mosu made a decision back then about how streams are appended. There's not much that can be done about that now.
The DG-Team is so great!!! I'm happy that DGDemux exists!

Originally Posted by Columbo

Improvements have been suggested and we're happy to consider them but so far nothing has been proposed that justifies the added complexity. One criticism that has been leveled is that if the video exceeds the audio at gaps then correction is never triggered, but for bluray we have never seen that in practice and it may be counter to the spec in any case.

Yes, I can well believe that; many will have encountered this problem and considered it.
One possibility would be to create a timestamp file during demuxing. This could then be reused whenever the audio stream needs to be processed.

Originally Posted by Columbo

Now you have all our secrets! Watch me get fired.

I don't think Rocky will fire you, he's a really nice guy.

Originally Posted by Columbo

I understand that your method for MKV in effect retains the timestamps for things, which in conjunction with some mkvtoolnix magic and expected behavior of the players allows you to bypass the gaps problem. That is a creative approach, but doesn't work for demuxing ES, where timestamps are lost. ...

First of all: MKVToolNix (MTX) doesn't perform any magic here; on the contrary, MTX needs help so it doesn't automatically decide how the timestamps should look. Without a timestamp file, MTX won't correctly join the tracks.

Similarly, the players themselves don't really have any magic. VLC, for example, can't handle video gaps at all; the movie always freezes briefly, and video artifacts appear. The best players here are MPC-HC or mpv.
The whole magic is just the timestamp file.

Originally Posted by Columbo

I note that MKV is not the only target format, and that ES is often needed for other things than immediate playback.

You're absolutely right. I sometimes forget that there are other containers besides Matroska. For me, there's only Matroska, as it's the best container in the world.

[Columbo]

Thank you for your reply.

Originally Posted by hubblec4

The Blu-ray specs state that a new m2ts file must always begin with a video and audio keyframe.
Therefore, I'm wondering if it wouldn't be better NOT to delete the next audio frame, because it definitely/usually/should be a keyframe.

The only relevant audio format for which every frame is not a keyframe is THD. That is handled differently as I implied when I said that the only difference between formats is what you are allowed to cut. Also, if the two frames are duplicates it doesn't matter which one you cut.

The DG-Team is so great!!! I'm happy that DGDemux exists!

Thanks, I appreciate that.

I don't think Rocky will fire you, he's a really nice guy.

Yes he is.

First of all: MKVToolNix (MTX) doesn't perform any magic here; on the contrary, MTX needs help so it doesn't automatically decide how the timestamps should look. Without a timestamp file, MTX won't correctly join the tracks.

The magic is that it accepts a timestamps file.

Similarly, the players themselves don't really have any magic. VLC, for example, can't handle video gaps at all; the movie always freezes briefly, and video artifacts appear. The best players here are MPC-HC or mpv. The whole magic is just the timestamp file.

I was thinking of your multi-edition support. Does that not require player support?

You're absolutely right. I sometimes forget that there are other containers besides Matroska. For me, there's only Matroska, as it's the best container in the world.

When all you have is a hammer, everything is a nail.

[hubblec4]

Originally Posted by Columbo

Thank you for your reply.

The only relevant audio format for which every frame is not a keyframe is THD. That is handled differently as I implied when I said that the only difference between formats is what you are allowed to cut. Also, if the two frames are duplicates it doesn't matter which one you cut.

Okay. I understand that TrueHD is really different. I'm really looking forward to the Blu-ray when I order it.

Originally Posted by Columbo

I was thinking of your multi-edition support. Does that not require player support?

Yes, a player needs to be able to do certain things to provide proper support for multi-editions.
Ordered chapters are already built into many players, not always well or optimally, but at least the basics are there.
MPC-HC does all of this very well, and mpv is close behind. If you want to test some more Matroska features, you can try my mpvMatroska.

[Columbo]

Can I use your timestamps method to make a normal single edition MKV from files that have not been gaps-corrected during demux and that will be in sync? What would the process be?

[hubblec4]

Hi Columbo

Absolute yes. A single Edition is the same as multiple Editions. The different is only: the used m2ts files.
But my cE not works with a single Edition in the Multi-Edition mode.

To create the timestamp files manually is not really easy(almost impossible).
In short:
1. the main timeline are the video frames
2. count the audio frames in each m2ts
3. the first audio frame of each m2ts gets the timestamp of the first video frame from the m2ts.

[Columbo]

Originally Posted by hubblec4

To create the timestamp files manually is not really easy (almost impossible).

That's a deal breaker. Don't you have a tool that will do this for single edition? I guess it would be quite popular.

[hubblec4]

I don't think so. But I don't know enough about it.

I asked Rocky back then if it would make sense for DGDemux to also create timestamp files while extracting the tracks. But we decided against it.

[Columbo]

Why did you decide against it?

[hubblec4]

Originally Posted by Columbo

Why did you decide against it?

To complicated, to less interested user.

I see my cE is able to create Timestamp files for single Editions, but it wont work for THD. I use a "fast" mode for TS-file creation, and THD breaks this small algo.
As info: THD has two FPS values: 1/1200 for 48kHz; and 1/1102,5 for 44,1kHz

I have no time currently to work on cE to fix that. But maybe future.