H.264 with keyframe value larger than the framerate are not muxed correctly in MXF

[FranceBB]

If any of you is using mxf as a container, you should know that both FFMpeg and BBC BMX Transwrap are having issues when muxing H.264 Files with Long GOP or when muxing any kind of arbitrary H.264 file which makes use of motion compensation, as the final muxed mxf file seems not to be alright 'cause decoders report some errors at decoding time.

here is an example:

x264.exe "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\AVS Script.avs" --preset medium --profile high422 --level 5.2 --ref 4 min-keyint 1 --keyint 40 --bitrate 500000 --vbv-maxrate 500000 --vbv-bufsize 500000 --deblock -4:-4 --overscan show --colormatrix bt709 --range tv --log-level info --thread-input --transfer bt709 --colorprim bt709 --videoformat component --nal-hrd cbr --aud --output-csp i422 --output-depth 10 --output "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\raw_video.h264"

ffmpeg.exe -i "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\AVS Script.avs" -vn -sn -c:a pcm_s24le -ar 48000 -af loudnorm=I=-24:LRA=1:tp=-2 -f wav "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\audio.wav"

ffmpeg.exe -i "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\raw_video.h264" -i "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\audio.wav" -c:v copy -c:a copy -f mxf -y "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\output.mxf"

pause

The code above uses Avisynth to index a sample file and encodes it at 500 Mbit/s with 1 keyframe every 40. Then, such a file, called raw_file.h264 is muxed in mxf using FFMpeg, however the resulting file - although it completes the muxing - has something wrong 'cause once it's played some players don't decode all frames: some are decoded, some other are not and the players report "requested frame not found".

It's possible to verify this by trying to play the encoded and muxed video in VirtualDub FilterMod and it will report the very same error.
Muxing the very same video file (i.e raw_video.h264) into another container like .ts via TSMuxer works and shows no playback/decoding issues whatsoever. Same goes for .mp4 using MP4Box, so it's just mxf that seems not to be ok.

Remuxing the files with BBC BMX Transwrap like:

ffmpeg.exe -i "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\raw_video.h264" -i "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\audio.wav" -c:v copy -c:a copy -f mxf -y "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\pre-final_output.mxf"

bmxtranswrap.exe -p -y 10:00:00:00 -t op1a -o "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\final_output.mxf" "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\pre-final_output.mxf"

improves things and produces a better file with less decoding errors and - if the GOP is shorter and I have exactly 1 intra every second (so keyint = framerate) - the BBC BMX Transwrap remuxed file works without any decoding issue.


I made 4 samples (colorbars, nothing exciting).

Test 1:

Test1_H264_30p_keyint30_BMX.mxf
Test1_H264_30p_keyint30_FFMpeg.mxf

Test 2:

Test2_H264_30p_keyint40_BMX.mxf
Test2_H264_30p_keyint40_FFMpeg.mxf


About Test1, the framerate is actually 29,970 so 30000/1001, but anyway, it has been encoded with --min-keyint 1 --keyint 30, so after the raw_video.h264 has been muxed with BMX it works just fine, whereas the FFMpeg muxed version has decoding issues.

Of the 100 frames encoded, this is the BMX muxed file being correctly decoded as:

Frame 0, Intra
Frame 1, B
Frame 2, B
Frame 3, B
Frame 4, P
Frame 5, B
Frame 6, B
Frame 7, B
Frame 8, P
Frame 9, B
Frame 10, B
Frame 11, B
Frame 12, P
Frame 13, B
Frame 14, B
Frame 15, B
Frame 16, P
Frame 17, B
Frame 18, B
Frame 19, B
Frame 20, P
Frame 21, B
Frame 22, B
Frame 23, B
Frame 24, P
Frame 25, B
Frame 26, B
Frame 27, B
Frame 28, P
Frame 29, P
Frame 30, I
Frame 31, B
Frame 32, B
Frame 33, B
Frame 34, P
Frame 35, B
Frame 36, B
Frame 37, B
Frame 38, P
Frame 39, B
Frame 40, B
Frame 41, B
Frame 42, P
Frame 43, B
Frame 44, B
Frame 45, B
Frame 46, P
Frame 47, B
Frame 48, B
Frame 49, B
Frame 50, P
Frame 51, B
Frame 52, B
Frame 53, B
Frame 54, P
Frame 55, B
Frame 56, B
Frame 57, B
Frame 58, P
Frame 59, P
Frame 60, I
Frame 61, B
Frame 62, B
Frame 63, B
Frame 64, P
Frame 65, B
Frame 66, B
Frame 67, B
Frame 68, P
Frame 69, B
Frame 70, B
Frame 71, B
Frame 72, P
Frame 73, B
Frame 74, B
Frame 75, B
Frame 76, P
Frame 77, B
Frame 78, B
Frame 79, B
Frame 80, P
Frame 81, B
Frame 82, B
Frame 83, B
Frame 84, P
Frame 85, B
Frame 86, B
Frame 87, B
Frame 88, P
Frame 89, P
Frame 90, I
Frame 91, B
Frame 92, B
Frame 93, B
Frame 94, P
Frame 95, B
Frame 96, B
Frame 97, B
Frame 98, P
Frame 99, B
Frame 100, P

while this is the FFMpeg muxed file (so same raw_video.h264 but muxed with FFMpeg) and it has a clear recurring pattern error while being decoded:

Frame 0, Intra
Frame 1, requested frame not found
Frame 2, requested frame not found
Frame 3, B
Frame 4, B
Frame 5, requested frame not found
Frame 6, requested frame not found
Frame 7, B
Frame 8, B
Frame 9, requested frame not found
Frame 10, requested frame not found
Frame 11, B
Frame 12, B
Frame 13, requested frame not found
Frame 14, requested frame not found
Frame 15, B
Frame 16, B
Frame 17, requested frame not found
Frame 18, requested frame not found
Frame 19, B
Frame 20, B
Frame 21, requested frame not found
Frame 22, requested frame not found
Frame 23, B
Frame 24, B
Frame 25, requested frame not found
Frame 26, requested frame not found
Frame 27, B
Frame 28, B
Frame 29, P
Frame 30, I
Frame 31, requested frame not found
Frame 32, requested frame not found
Frame 33, B
Frame 34, B
Frame 35, requested frame not found
Frame 36, requested frame not found
Frame 37, B
Frame 38, B
Frame 39, requested frame not found
Frame 40, requested frame not found
Frame 41, B
Frame 42, B
Frame 43, requested frame not found
Frame 44, requested frame not found
Frame 45, B
Frame 46, B
Frame 47, requested frame not found
Frame 48, requested frame not found
Frame 49, B
Frame 50, B
Frame 51, requested frame not found
Frame 52, requested frame not found
Frame 53, B
Frame 54, B
Frame 55, requested frame not found
Frame 56, requested frame not found
Frame 57, B
Frame 58, B
Frame 59, P
Frame 60, I
Frame 61, requested frame not found
Frame 62, requested frame not found
Frame 63, B
Frame 64, B
Frame 65, requested frame not found
Frame 66, requested frame not found
Frame 67, B
Frame 68, B
Frame 69, requested frame not found
Frame 70, requested frame not found
Frame 71, B
Frame 72, B
Frame 73, requested frame not found
Frame 74, requested frame not found
Frame 75, B
Frame 76, B
Frame 77, requested frame not found
Frame 78, requested frame not found
Frame 79, B
Frame 80, B
Frame 81, requested frame not found
Frame 82, requested frame not found
Frame 83, B
Frame 84, B
Frame 85, requested frame not found
Frame 86, requested frame not found
Frame 87, B
Frame 88, B
Frame 89, P
Frame 90, I
Frame 91, requested frame not found
Frame 92, requested frame not found
Frame 93, B
Frame 94, B
Frame 95, requested frame not found
Frame 96, requested frame not found
Frame 97, B
Frame 98, B
Frame 99, requested frame not found
Frame 100, requested frame not found

So essentially the pattern of the GOP is: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPP and then it goes back to I and starts over and apparently the decoding issue is consistent in time.

Now, as to test 2, the file is again a 29,970p, so 30000/10001 but this time it has been encoded with --min-keyint 1 --keyint 40, so the GOP is longer than 1 second (keyint != framerate) and in this case both BMX and FFMpeg muxed files have decoding issues.

Let's start with BMX:

Frame 0, Intra
Frame 1, B
Frame 2, B
Frame 3, B
Frame 4, P
Frame 5, B
Frame 6, B
Frame 7, B
Frame 8, P
Frame 9, B
Frame 10, B
Frame 11, B
Frame 12, P
Frame 13, B
Frame 14, B
Frame 15, B
Frame 16, P
Frame 17, B
Frame 18, B
Frame 19, B
Frame 20, P
Frame 21, B
Frame 22, B
Frame 23, B
Frame 24, P
Frame 25, B
Frame 26, B
Frame 27, B
Frame 28, P
Frame 29, B
Frame 30, B
Frame 31, B
Frame 32, P
Frame 33, requested frame not found
Frame 34, B
Frame 35, B
Frame 36, B
Frame 37, B
Frame 38, B
Frame 39, P
Frame 40, I
Frame 41, B
Frame 42, B
Frame 43, B
Frame 44, P
Frame 45, B
Frame 46, B
Frame 47, B
Frame 48, P
Frame 49, B
Frame 50, B
Frame 51, B
Frame 52, P
Frame 53, B
Frame 54, B
Frame 55, B
Frame 56, P
Frame 57, B
Frame 58, B
Frame 59, B
Frame 60, P
Frame 61, B
Frame 62, B
Frame 63, B
Frame 64, P
Frame 65, B
Frame 66, B
Frame 67, B
Frame 68, P
Frame 69, B
Frame 70, B
Frame 71, B
Frame 72, P
Frame 73, requested frame not found
Frame 74, B
Frame 75, B
Frame 76, B
Frame 77, B
Frame 78, B
Frame 79, P
Frame 80, I
Frame 81, B
Frame 82, B
Frame 83, B
Frame 84, P
Frame 85, B
Frame 86, B
Frame 87, B
Frame 88, P
Frame 89, B
Frame 90, B
Frame 91, B
Frame 92, P
Frame 93, B
Frame 94, B
Frame 95, B
Frame 96, P
Frame 97, B
Frame 98, B
Frame 99, B
Frame 100, P

Now the FFMpeg muxed one:

Frame 0, Intra
Frame 1, requested frame not found
Frame 2, requested frame not found
Frame 3, B
Frame 4, P
Frame 5, requested frame not found
Frame 6, requested frame not found
Frame 7, B
Frame 8, P
Frame 9, requested frame not found
Frame 10, requested frame not found
Frame 11, B
Frame 12, P
Frame 13, requested frame not found
Frame 14, requested frame not found
Frame 15, B
Frame 16, P
Frame 17, requested frame not found
Frame 18, requested frame not found
Frame 19, B
Frame 20, P
Frame 21, requested frame not found
Frame 22, requested frame not found
Frame 23, B
Frame 24, P
Frame 25, requested frame not found
Frame 26, requested frame not found
Frame 27, B
Frame 28, P
Frame 29, requested frame not found
Frame 30, requested frame not found
Frame 31, B
Frame 32, P
Frame 33, requested frame not found
Frame 34, requested frame not found
Frame 35, B
Frame 36, B
Frame 37, requested frame not found
Frame 38, B
Frame 39, P
Frame 40, I
Frame 41, requested frame not found
Frame 42, requested frame not found
Frame 43, B
Frame 44, P
Frame 45, requested frame not found
Frame 46, requested frame not found
Frame 47, B
Frame 48, P
Frame 49, requested frame not found
Frame 50, requested frame not found
Frame 51, B
Frame 52, P
Frame 53, requested frame not found
Frame 54, requested frame not found
Frame 55, B
Frame 56, P
Frame 57, requested frame not found
Frame 58, requested frame not found
Frame 59, B
Frame 60, P
Frame 61, requested frame not found
Frame 62, requested frame not found
Frame 63, B
Frame 64, P
Frame 65, requested frame not found
Frame 66, requested frame not found
Frame 67, B
Frame 68, P
Frame 69, requested frame not found
Frame 70, requested frame not found
Frame 71, B
Frame 72, P
Frame 73, requested frame not found
Frame 74, requested frame not found
Frame 75, B
Frame 76, B
Frame 77, requested frame not found
Frame 78, B
Frame 79, P
Frame 80, I
Frame 81, requested frame not found
Frame 82, requested frame not found
Frame 83, B
Frame 84, P
Frame 85, requested frame not found
Frame 86, requested frame not found
Frame 87, B
Frame 88, P
Frame 89, requested frame not found
Frame 90, requested frame not found
Frame 91, B
Frame 92, P
Frame 93, requested frame not found
Frame 94, requested frame not found
Frame 95, B
Frame 96, P
Frame 97, requested frame not found
Frame 98, requested frame not found
Frame 99, B
Frame 100, P

Command line for Test 1:

x264.exe "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\AVS Script.avs" --preset medium --profile high422 --level 5.2 --ref 4 min-keyint 1 --keyint 30 --bitrate 500000 --vbv-maxrate 500000 --vbv-bufsize 500000 --deblock -4:-4 --overscan show --colormatrix bt709 --range tv --log-level info --thread-input --transfer bt709 --colorprim bt709 --videoformat component --nal-hrd cbr --aud --output-csp i422 --output-depth 10 --output "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\raw_video.h264"

Command line for Test 2:

x264.exe "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\AVS Script.avs" --preset medium --profile high422 --level 5.2 --ref 4 min-keyint 1 --keyint 40 --bitrate 500000 --vbv-maxrate 500000 --vbv-bufsize 500000 --deblock -4:-4 --overscan show --colormatrix bt709 --range tv --log-level info --thread-input --transfer bt709 --colorprim bt709 --videoformat component --nal-hrd cbr --aud --output-csp i422 --output-depth 10 --output "\\mibctvan000.avid.mi.bc.sky.it\Ingest\MEDIA\temp\raw_video.h264"

Test download: https://we.tl/t-cNrs0AByEk
(link valid for 7 days)


It looks like ffmpeg doesn't indicate frame reordering in the index, while BMX does:





Here you're gonna find the container check report, highlighting the errors for each file:

- Test1_H264_30p_keyint30_BMX-IRT--RDD9-errors.html
- Test1_H264_30p_keyint30_FFMpeg-IRT--RDD9-errors.html​
- Test2_H264_30p_keyint40_FFMpeg-IRT--RDD9-errors.html
- Test2_H264_30p_keyint40_BMX-IRT--RDD9-errors.html​


I've opened a ticket in both the BMX bug tracker and the FFMpeg bug tracker.

I'll keep you posted.

[FranceBB]

Fixed by Philip de Nier in the BBC muxer (BBC BMX Transwrap): https://sourceforge.net/p/bmxlib/bmx...4318d6fa05c8b/

I tested it and it does indeed solve the issue.
New build with the fix here (I just made it): https://mega.nz/folder/qA8nDa7Z#Px3EHeBn_nHfYbsPg0YOlg

FFMpeg still has the issue, though, an nobody seem to care... https://trac.ffmpeg.org/ticket/9647#comment:5