Code:
-x264opts colormatrix=bt709:transfer=bt709:colorprim=bt709
and you can also make sure in Vapoursynth to make sure about input, not sure if it is redundant to include transfer and color primaries as well:
Code:
#clip is NTSC DV avi
#here is clip 4:1:1 (loaded by ffms2) and matrix, transfer and colorprimaries are unknown
clip = core.std.SetFrameProp(clip, prop="_Matrix", intval=6)
clip = core.std.SetFrameProp(clip, prop="_Transfer", intval=6)
clip = core.std.SetFrameProp(clip, prop="_Primaries", intval=6)
#here you have matrix, transfer and color primaries as NTSC DV
clip = core.resize.Bicubic(clip, format=vs.YUV422P8, matrix_s = '709', transfer_s = '709', primaries_s = '709' )
#here you have all as bt709 and also YUV422P8