Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-2 Encoding

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th March 2022, 06:44   #1  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Encoding XDCAM HD422 with ffmpeg for compatibility and max quality

Hello!

I'm having to implement XDCAM HD422 encoding (in 2022 lol) and am using ffmpeg to do it, so I'm using their MPEG-2 encoder.

Based on some digging, here's the settings I'm using for 1080i29.97 content. Any suggestions on how I can improve quality?

Quote:
ffmpeg -i input -c:v mpeg2video -g 15 -bf 2 -mpv_flags +strict_gop -b:v 50M -minrate 50M -maxrate 50M -bufsize 17825792 -rc_init_occupancy 17825792 -intra_vlc 1 -non_linear_quant 1 -qmin 2 -qmax 12 -dc 8 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -seq_disp_ext 1 -video_format component -color_range tv -chroma_sample_location topleft -signal_standard 4 -intra_matrix 8,10,22,27,29,37,37,40,9,12,14,28,29,37,39,40,9,14,27,31,34,37,40,48,12,22,27,29,34,37,40,58,26,27,29,34,37,38,48,58,26,27,29,36,38,38,48,69,18,27,34,36,38,38,48,69,26,26,34,34,38,40,58,79 -inter_matrix 16,20,22,26,28,32,32,36,18,20,22,28,28,32,34,36,18,22,26,30,30,32,36,38,20,22,26,28,30,32,36,42,24,26,28,30,32,34,38,40,24,26,28,32,34,34,38,42,24,26,30,32,34,34,38,42,24,24,30,30,34,36,40,44 -top 1 -alternate_scan 1 -flags +ildct+ilme video.m2v
For 24p I'd drop the interlacing related flags at the end of that command and of course drop the GOP length to 12.

These encodes compare pretty well with the Mainconcept encoder in Adobe Media Encoder, but I do think the latter is still a bit better. Can I squeeze any more out of ffmpeg?
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 9th March 2022, 07:24   #2  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 343
Where exactly have your specified 422 ? Also "-chroma_sample_location topleft", I want to believe right ? Maybe try flat matrices ? Why are you using such a bad encoder ?
rwill is offline   Reply With Quote
Old 9th March 2022, 19:10   #3  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Quote:
Where exactly have your specified 422?
I omitted that in my sample but we do set -pix_fmt yuv422p. Good catch!


Quote:
Why are you using such a bad encoder
It's free, and not too bad. Our tooling also has an extensive ffmpeg integration so this is pretty easy to add. The other option I could easily use would be AWS MediaConvert. Their MPEG-2 encoder is quite good, so I might use that instead if we have problems with the ffmpeg produced files.


Quote:
Also "-chroma_sample_location topleft", I want to believe right ?
Hmm, maybe so. I guess it depends on the input. I may just omit this.

Quote:
Maybe try flat matrices ?
The custom matrix here did seem to improve quality a fair bit.
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 9th March 2022, 20:42   #4  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 343
I do not know if -pix_fmt yuv422p is enough to trigger 422 Profile @ High Level signaling in ffmpeg. Maybe "-profile 422 -level high" will help because High Profile supports 422 as well. And I do not think ffmpeg does Level Limit checking.

I would drop -qmax or set it to 28 if you really want to use -non_linear_quant, don't know if vbv constraints are upheld in complex content otherwise.

I do not think mpeg2 has a syntax element for chroma_sample_location; And 422 formats do not have a concept of chroma being "topleft", only "left".

The matrices being not flat matters if quality is high on clean, non-dithered content. It being the ffmpeg encoder quality wont get too high probably though.

The ffmpeg mpeg2 encoder kicks the shit out of mpeg2 encoders from the late 90s but is not modern in any way. There have been some additions over time that help somewhat.

So I would do two passes and add the following:

Quote:
-cmp sad -subcmp satd -mbcmp rd -ildctcmp rd -dia_size 800 -precmp sad -pre_dia_size 800 -me_range 512 -mbd rd -trellis 1
This will get you 'some' RDO and thus quality at the cost of speed.

Shameless plug:
Before you give the resulting quality a pass, maybe try what y262 can do. Although it is very rough around the edges with limited error checking its pretty close to x264 'veryslow' tool wise at the core. It aims to be pretty and not fast.

https://github.com/rwillenbacher/y262

2-pass:

Quote:
<ffmpeg> -i somefile.mov -pix_fmt yuv422p -vcodec rawvideo -f rawvideo -acodec none - | y262.exe -in - -size 1920 1080 -threads 1 4 -profile 422 -level 422high -chromaf 422 -bitrate 40000 -vbv 17000 -vbvrate 50000 -quality 20 -frcode 4 -interlaced -arinfo 3 -nump 4 -numb 2 -psyrd 256 -flatmat -videoformat 709 -out pass1.m2v -rcmode 1 -mpout stats.p1

<ffmpeg> -i somefile.mov -pix_fmt yuv422p -vcodec rawvideo -f rawvideo -acodec none - | y262.exe -in - -size 1920 1080 -threads 1 4 -profile 422 -level 422high -chromaf 422 -bitrate 40000 -vbv 17000 -vbvrate 50000 -quality 20 -frcode 4 -interlaced -arinfo 3 -nump 4 -numb 2 -psyrd 256 -flatmat -videoformat 709 -out pass2.m2v -rcmode 2 -mpin stats.p1
rwill is offline   Reply With Quote
Old 10th March 2022, 08:56   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
XDCAM is still well widespread across broadcasters for FULL HD materials.
This is mostly due to the fact that most of them used MPEG-2 for SD with standards like IMX30 and IMX50 and decided to stick with the same codec for FULL HD by using XDCAM-50.
There has been XDCAM-85 for a very brief moment but it has never really been officially supported and has never been as widespread as the 50 Mbit/s variant was.
Most playout ports still rely on it like Harmonic Omneon playout ports.
This being said, this is the command line created automatically by FFAStrans (the free software we work on) for XDCAM encodes in 25i using the built-in FFMpeg MPEG-2 encoder when Avisynth delivers 50p input:

Quote:
"\\\\mibcisilonsc\\avisynth\\Server\\encoder\\encoder\\processors\\FFMpeg\\x64\\ffmpeg.exe" -stats_period 1.2 -hide_banner -follow 1 -rw_timeout 1000000 -i "\\\\mibcisilonsc\\avisynth\\TempFiles\\20210325-1012-0574-5bf0-5c10b079b2d8\\20220309-1839-0045-56d9-3f4e769c072a\\1-34-34~220310020803642~21788~20170208-150859-290-044E9EC8E471~enc_av_xdcamhd.avs" -f lavfi -i "aevalsrc=0" -f lavfi -i "color=color=black:size=1920x1080" -shortest -map_metadata -1 -map 0:0 -filter_complex "[0:1]pan=1|c0=c0[a1],[0:1]pan=1|c0=c1[a2],[0:1]pan=1|c0=c2[a3],[0:1]pan=1|c0=c3[a4],[0:1]pan=1|c0=c4[a5],[0:1]pan=1|c0=c5[a6],[0:1]pan=1|c0=c6[a7],[0:1]pan=1|c0=c7[a8],[a1]amerge=1,apad[astr1],[a2]amerge=1,apad[astr2],[a3]amerge=1,apad[astr3],[a4]amerge=1,apad[astr4],[a5]amerge=1,apad[astr5],[a6]amerge=1,apad[astr6],[a7]amerge=1,apad[astr7],[a8]amerge=1,apad[astr8]" -map "[astr1]" -c:a:0 pcm_s24le -ar:a:0 48000 -map "[astr2]" -c:a:1 pcm_s24le -ar:a:1 48000 -map "[astr3]" -c:a:2 pcm_s24le -ar:a:2 48000 -map "[astr4]" -c:a:3 pcm_s24le -ar:a:3 48000 -map "[astr5]" -c:a:4 pcm_s24le -ar:a:4 48000 -map "[astr6]" -c:a:5 pcm_s24le -ar:a:5 48000 -map "[astr7]" -c:a:6 pcm_s24le -ar:a:6 48000 -map "[astr8]" -c:a:7 pcm_s24le -ar:a:7 48000 -vf "sidedata=delete,metadata=delete,framerate=50,setfield=tff,separatefields,select='not(mod(n\\,4))+not(mod(n+1\\,4))',weave=t,setfield=tff,setdar=16/9,setsar=1" -timecode 00:00:00:00 -c:v mpeg2video -r 25 -pix_fmt yuv422p -aspect 16:9 -b:v 50000000 -minrate 50000000 -maxrate 50000000 -intra_vlc 1 -dc 8 -g 12 -bf 2 -mpv_flags +strict_gop -qmin 5 -qmax 28 -qsquish 99 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -seq_disp_ext 1 -video_format component -color_range tv -chroma_sample_location topleft -signal_standard 4 -field_order tt -top 1 -alternate_scan 1 -flags +ildct+ilme+cgop -sc_threshold 1000000000 -intra_matrix 8,10,22,27,29,37,37,40,9,12,14,28,29,37,39,40,9,14,27,31,34,37,40,48,12,22,27,29,34,37,40,58,26,27,29,34,37,38,48,58,26,27,29,36,38,38,48,69,18,27,34,36,38,38,48,69,26,26,34,34,38,40,58,79 -inter_matrix 16,20,22,26,28,32,32,36,18,20,22,28,28,32,34,36,18,22,26,30,30,32,36,38,20,22,26,28,30,32,36,42,24,26,28,30,32,34,38,40,24,26,28,32,34,34,38,42,24,26,30,32,34,34,38,42,24,24,30,30,34,36,40,44 -non_linear_quant 1 -f mxf -max_muxing_queue_size 700 -map_metadata -1 -metadata "creation_time=now" -
ffmpeg stats and -progress period set to 1.2.
1) Chroma Sample Location

About the chroma sample location, top_left is right in the sense that top_left and left is the same for 4:2:2, we had this discussion in the Avisynth section, however for whatever reason XDCAM files (every one of them I received) had "top_left" specified, so I think we should stick with top_left.

2) Qmin Qmax

Unfortunately it really depends on the content. I noticed that you set qmin and qmax to be really low, so you aim for the best quality, however, depending on how complex your scene is, you might end up with a buffer underflow. In such an event, FFMpeg will raise a warning, but the encode will keep going, but of course the quality will be really really bad. In my personal experience, if you want something that always works, you can use -qmax 28, however quality will of course suffer. In FFAStrans we created 4 different quality presets which the user can choose and which will select a different qmin qmax setting. The reason why I say this, is that even something like -qmin 5 -qmax 25 ended up with some buffer issue in some scenarios.

3) Open GOP vs Closed GOP

Ask the one you're sending the file to. The overwhelming majority require closed GOP as per their specs for legacy reasons and have software like Cerify, Aurora, Baton etc set to check the closed GOP.
You can achieve closed GOP with: -flags +cgop
Some playout ports might play Open GOP files as well just fine, but it really depends on who you're sending those files to.

4) Wrong GOP issues

Unfortunately, in FFMpeg, even if you specify something like:
Code:
-g 12 -bf 2 -mpv_flags +strict_gop
to try to get the right GOP in PAL 25i, it will end up with the GOP of the right size, but with the wrong order. In other words, FFMpeg cannot create the right GOP as the very first GOP of every file is gonna be wrong, all the others will be right. Again, this is a non issue as files are played just fine.

5) container

No matter what, do not, ever, use the FFMpeg mxf muxer unless you really really really have to. I've had several regressions over the past 7 years and I've been one of the very first people to report them every time and after a while I've said: enough is enough. The built-in mxf muxer is rather sloppy and has several issues, so once your file is encoded, I would strongly suggest you to remux it with BBC BMX Transwrap, which is the official BBC MXF Muxer. Ever since I moved to it, I've never had any issues. The good thing is that you don't have to wait for the encode to finish, you can pipe the FFMpeg output to bmx and let it mux it on the fly while the file is in encoding, thus saving time.

There are other muxers like ommcp which is the one made by Harmonic which is the creator of Omneon, however is a paid muxer which is licensed to those using Omneon playback ports etc (but it has its flaws too, just saying...).
FranceBB is offline   Reply With Quote
Old 10th March 2022, 19:50   #6  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Thank you all for the comments and suggestions
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.