Workflow from VapourSynth to DaVinci Resolve

[Tormaid]

Originally Posted by sunshine

i’m looking for a good lossless option for archival, and will use x264 in mp4 or mov for local use and pushing to youtube.

ffv1?

[sunshine]

Originally Posted by poisondeathray

So I am suggesting -vf setsar=sar=10/11 , but you have to check the actual footage with known objects or references because all sorts of things might distort the AR (studio screw up before broadcast, bad cap etc...) But a native DV camcorder will use ITU rules 99.9% of the time

My tests confirm the same -

Approach:
- Captured video directly over a perfect circle, filling the frame as much as possible
- Transferred via FireWire WinDV
- Transcoded using the ffmpeg seen in my script, using both the original setsar I'd proposed, and the recommendation you made, but I added a -vf filter to include the frame number so I can compare frame to frame between them easily.
- Opened both videos in Resolve, with a 1920x1080 timeline square pixel
- Added both videos to the timeline
- Took screen-shots of the fullscreen scaled 1920x1080 (still 4:3, with pillar boxes) from both versions of the video (the sar 8:9/dar 4:3 and sar 10:11)
- Imported the screenshots into Gimp, and used various circle tools to examine them

Result:

The SAR 10:11 version was a more perfect circle than the other.

I repeated the same test for video captured using a legacy SD Hi8 analog camcorder, with the same results.

Thanks again all - it's time to let my machine crunch on the videos. Soon I'll be color grading in Resolve.

I'll follow-up with a reply that includes my full script etc.

[sunshine]

Originally Posted by Tormaid

ffv1?

Not available from Resolve :/

[sunshine]

OK here's the full version of my workflow. Perhaps it'll help someone one day.

Source: MiniDV (Digital8 and Hi8)

Connect camcorder via FireWire and import video using WinDV. Save the clips somewhere with a lot of space..

#!/usr/bin/bash

# I include my VapourSynth script right inside my script that automates all of this.. it keeps things in one place.
################
read -r -d '' VAPOURSYNTH <<'VSScript'
#!/usr/bin/env python

# import the needed modules
import vapoursynth as vs
import havsfunc as haf
import edi_rpow2 as edi

core = vs.get_core()

clip = core.ffms2.Source(source=INVID)
clip = core.resize.Bicubic(clip, format=vs.YUV422P8, matrix_in_s="170m", matrix_s="709")
clip = haf.QTGMC(clip, Preset="Slower", TFF=False)

clip.set_output()

VSScript
##############

# This outputs the python VapourSynth script to "Deinterlace.py"
echo "$VAPOURSYNTH" > ./Deinterlace.py


# List all your clips here, full paths, one per line
ALLVIDS="/clips/clip1.avi
/clips/clip2.avi
/clips/clip3.avi"

# I took this approach vs. a "find /clips -name \*.avi | while read INVID; do" approach because ffmpeg seems to mess with STDIN/ OUT and makes looping that way painful.. This avoids that pain.

for INVID in $ALLVIDS; do
        echo "--------------------------------------"
        OUTVID=$(echo "$INVID" | sed -e 's/\.avi/\.mov/')
        #OUTVID=$(echo "$OUTVID" | sed -e 's/\.mov/\.10-11\.mov/')
        #OUTVID=$(echo "$OUTVID" | sed -e 's/\.mov/\.8-9\.4-3\.mov/')
        if [ -e "$OUTVID" ]; then
                if [ -e "$OUTVID.complete" ]; then
                        echo "Found existing complete version of this video; skipping"
                        echo "--------------------------------------"
                        continue
                else
                        rm "$OUTVID"
                fi
        fi


        mediainfo "$INVID" | grep -qi "16:9"
        if [ $? -eq 0 ]; then
                # some of my videos are wide-angle SD DV.. I'll deal with them later.
                echo "This video is in 16:9 - skipping it and do it by hand later"
                echo "--------------------------------------"
                continue
        fi


        echo "Processing Input $INVID and outputting to $OUTVID"
        vspipe --y4m ./Deinterlace.py -a "INVID=$INVID" - | ffmpeg -nostdin -i pipe: -i "$INVID" -pix_fmt yuv422p -c:v libx264  -crf 0 -tune fastdecode -x264opts keyint=1 -c:a copy -map 0:0 -map 1:1 -vf setsar=sar=10/11 "$OUTVID" 2>&1
  
        if [ $? -eq 0 ]; then
                echo "Successfully converted $INVID"
                (
                        echo "Vapoursynth Script:"
                        echo "$VAPOURSYNTH"
                        echo
                        echo "ffmpeg:"
                        echo "vspipe --y4m ./Deinterlace.py -a \"INVID=$INVID\" - | ffmpeg -nostdin -i pipe: -i \"$INVID\" -pix_fmt yuv422p -c:v libx264  -crf 0 -tune fastdecode -x264opts keyint=1 -c:a copy -map 0:0 -map 1:1 -vf setsar=sar=10/11 \"$OUTVID\""
                ) > "$OUTVID.complete"

        fi
        echo "--------------------------------------"
done | tee process.$(date +%Y%m%d%H%M%S).log

[_Al_]

there is no "BT709" flag during encoding, so be careful, colors (reds, orange) are going to be most likely wrong if you try to view those archives , players might be defaulting to BT601 because of resolution

[sunshine]

Originally Posted by _Al_

there is no "BT709" flag during encoding, so be careful, colors (reds, orange) are going to be most likely wrong if you try to view those archives , players might be defaulting to BT601 because of resolution

Oh oh!

What have I missed Al, and do you have a suggestion for how I might fix it?

Should I output 601 from the VapourSynth deinterlacing? If so, I'm not sure what I'd need to change.

Should I somehow flag that it's 709 during ffmpeg encode? If so, again.. I'm not sure what I'd need to change.

Thanks for spotting that Al!


FYI - I checked my input video with the following:

#!/usr/bin/env python

# import the needed modules
import vapoursynth as vs
import havsfunc as haf
import edi_rpow2 as edi

core = vs.get_core()

# read the clip
clip = core.ffms2.Source(source=INVID)
clip = core.text.FrameProps(clip)
clip.set_output()

Chroma Location: 2
Matrix: 2
Primaries: 2
Transfer: 2

Hopefully that's helpful info... but I recall you saying seeing a "2" there means unknown. All I know from MediaInfo is that the original AVIs are yuv411.

[_Al_]

-x264opts colormatrix=bt709:transfer=bt709:colorprim=bt709

and you can also make sure in Vapoursynth to make sure about input, not sure if it is redundant to include transfer and color primaries as well:

#clip is NTSC DV avi
#here is clip 4:1:1 (loaded by ffms2) and matrix, transfer and colorprimaries are unknown
clip = core.std.SetFrameProp(clip, prop="_Matrix", intval=6)
clip = core.std.SetFrameProp(clip, prop="_Transfer", intval=6)
clip = core.std.SetFrameProp(clip, prop="_Primaries", intval=6)
#here you have matrix, transfer and color primaries as NTSC DV
clip = core.resize.Bicubic(clip, format=vs.YUV422P8, matrix_s = '709', transfer_s = '709', primaries_s = '709'  )
#here you have all as bt709 and also YUV422P8

[poisondeathray]

I think he's using SD, and not upscaling at that stage ; so probably shouldn't use 709

(although resolve doesn't care if SD import has wrong 709 flag, other programs might)

[_Al_]

that's another way to go:

clip = core.resize.Bicubic(clip, format=vs.YUV422P8)

and

-x264opts colormatrix=smpte170m:transfer=smpte170m:colorprim=smpte170m

is that ffmpeg x264opts correct?

[poisondeathray]

@_Al_
yes, smpte170m

@sunshine
In vapoursynth, the matrix_in_s="170m", matrix_s="709" arguments were for when you were upscaling the dimensions. It's similar to using colormatrix for 601->709 . But since you decided to change strategy, it's no longer required . However, when you export for a HD targets, you still need to take care of that (if the programs you are using do not take care of it automatically)

And I question the use of 422 , it will just make the filesize larger. Resolve will handle 420 AVC fine, unless there is some special way resolve handles 422 preferably to 420, from a 411 DV source ? Maybe on the BM forum that I don't know about ? (420 still exceeds the 411 source chroma - it's technically upscaled chroma)

422 has benefits in some programs for handing interlace, but you're QTGMCing it here