I can only give you an example of how it works for me. I use XP and an old nvidia card. Newer versions of Windows and/or a different video card may produce different results (I don't fully understand how it works).
For me, when I play a video with MPC-HC, the choice of colorimetry is based on resolution. High definition should use rec.709 and standard definition should use rec.601 (the colours are slightly different according to the method used to convert the video to RGB on playback). I worked out the rule by which colorimetry is determined for me, and I'm pretty sure for video to display using rec.709, the width must be equal to or greater than 1200 AND the height must be equal to or greater than 578.
The upshot of that is.... a 1280x544 encode, which should display using rec.709, displays using rec.601, but that's a playback issue. It's still encoded correctly.
One way for me to fix the problem is in MPC-HCs options, under Internal Filters. Click the "video decoder" button. Disable all the 8 bit output formats except RGB32 and RGB24. That'll force LAV to convert to RGB and it seems to obey any colorimetry info written to the h264 video stream. It also seems to make better colorimetry choices based on resolution, so a 1280x544 encode should display using the correct colours.
Another option might be to use the
MadVR renderer. It should display correctly without LAV converting to RGB but it's been a while since I've used it. The Windows renderers (VMR9 or EVR etc) all seem to work the same way for me.
I'm curious..... anyone using MPC-HC with a newer version of Windows and do you know the resolution at which it switches colorimetry or if it obeys the colorimetry written to the video stream? Does either depend on the renderer used?