VDub2 Canopus HQX 16-bit negotiation issue

[kolak]

Originally Posted by Hotte

Are you are saying (ffmpeg DNxHD profile=dnxhr_hqx) = DNxHR ? So that's even better!
Still I can see a difference (the slightly reduced microcontrast I was talking about) between ffmpeg DNxHR profile=dnxhr_hqx and native DNxHR Exports from Edius. But that could be a difference between the professional and the ffmpeg implementation - couldn`t it ?


Ah! Does that mean that if VDub2 Compress-Dialog would implement HQX as "ffmpeg HQX" (rather than "HQX" without ffmpeg as today) it would compress 10 bit ? Now I understood, that HQX is not native VDub2 but came with Edius... Sometimes it takes time to understand - sorry. But then ffmpeg HQX would be another great implementation candidate...

DNxHR differences can be down to implementations. FFmpeg code is different than AVID reference one (it's the same as you have 10 different h264 encoders- some codec but you get slightly\very different results). You also have to make sure your avs etc path is correct.
For me ffmpeg is actually slightly better than AVID encoder. And because I use plain simple test (v210 source directly encoded ) I have no place for any error (except actually ffmpeg issue if exist). And because ffmpeg gives slightly higher PSNR and also when I visually compare in Resolve ffmpeg DNxHR show less difference I simply don't believe in your tests (your path is full of possible issues- it involves so many "engines").

Yes, but atm. ffmpeg has no HQX encoder- just decoder, so it won't happen.

[Hotte]

Thanks a lot Kolak for your extensive and well understandable explanations!

With your and poisondeathray's help I was able to understand the traps of the workflow and how I can handle them. With respect to my intended workflow Camera > Edius > AVS+ > Edius would you be so kind to check if these conclusion sounds basically consistent to you:

- My camera delivers H.264 YUV422 10bit. I should always record in rec709 with full luminance range
- I learned my NLE has a YUV engine. I can use the Primary Color Correction filter, no need to be concerned about that as possible conversion loss is not relevant in the amateur field
- Export HQX superfine 10bit as MOV is ok. I am staying in YUV
- Import into avs+ with LSMASH in YUV422P10. Check that rec709 arrives. If not, convert it.
- whenever possible use 16-bit AVS+ YUV422 capable filters
- to monitor interim results in VDub2 convertToRGB24(matrix="pc.709"). (monitor only, not for export)
- Compress the result with appropriate YUV422 10bit codec to import back into Edius (both of us seem to like ffmpeg DNxHD )

And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

Thank you so much.

[poisondeathray]

Originally Posted by Hotte

@Poisondeathray, did I get it get it right or do you see anything to be improved with this little code snippet ?

I am getting almost correct colors and contrast with this. There are very subtle color differences, only viewable at very large zoomlevels:

Edius TIFF: HQX-10bit superfine
https://imgur.com/3bWpRUR

VDub2 TIFF: HQX-10bit superfine converted to RGB24 pc.709 as above
https://imgur.com/YZiFJAv

Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

-There are slightly different algorithms to convert YUV422P10 to RGB . For example, chroma upsampling algorithm. (one might use bicubic, one might use slightly different coefficients of bicubic, or bilinear etc...). Chroma center location interpretation. Avisynth internal resizers use "center" instead of "mpeg2". You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")

2. I am not sure I fully understood the luminance range matter.

You did. The flag in the original recording signalled Edius to convert to RGB (for preview) using full range equation. You can simulate the same using "PC" matrix in avs

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

Pros/cons to acquisition format . Full range is probably better if the tools used can manipulate in float . If not, there is more loss because you have to adjust the ranges to a larger extent to make them "standard range"

But final version should always be "standard" range. So fast turn around situations (direct to TV, broadcast ENG) never use "full range" for acquistion

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).

Vapoursynth can do this correctly. In order to be 100% lossless the chroma up/down sampling has to use nearest neighbor algorithm, and float used on all calculations

But these are relatively minor issues. Don't loose sight of the "big picture" as Kolak said

I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

If it works in RGB48, not float, then potentially there is a problem. You'd have to "legalize" YUV beforehand, or risk significant clipping

And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

PSNR is a measure of "quality" compared to a "source".
One way can use ffmpeg to measure
https://en.wikipedia.org/wiki/Peak_s...to-noise_ratio
https://ffmpeg.org/ffmpeg-filters.html#psnr

It's not a good measure for subjective testing, but it's excellent to determine if something is lossless or not (infinity dB)

[kolak]

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

[poisondeathray]

Originally Posted by kolak

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped

PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

[kolak]

ffmpeg -i ref_source -i source -filter_complex psnr -f null -

this how you do it with ffmpeg. At the end you get stats. Anything within +-3dB is not a very meaningful difference. Eg ProRes may show average PSNR of 56dB and DNxHR 54dB. For clean good quality sources (like my BRAW 12K sample downscaled to HD) PSNR can go above 60dB which means loos is very small. For difficult scenes PSNR for intermediate codecs can go <50dB.

You can read about HQX here in its whitepaper:

https://www.syntex.sk/media/document...whitepaper.pdf

you also have Apple white paper about ProRes which has very similar results: https://www.apple.com/final-cut-pro/...hite_Paper.pdf )

you have PSNR graphs against other codecs. GV uses same source as Apple (STEM footage), which is specially prepared footage for digital cinema validation. It's never compressed (shot on 35M film) source which is very important. I use to have access to it- very demanding source.
If you take some source from the internet (which you have no idea how it was shot) your results can be very misleading. You may get some nice source which was shot on eg. ALEXA in ProRes natively. Then if you convert that to ProRes then it's 2nd generation fo the same codec and your PSNR will be very high. It won't be true for other codecs as they use different math. You have to really know what you are doing in those tests as you can easily produce something very false Cineform behaves quite differently on uncompressed sources with original noise/grain compared to source which are already went through DCT based compression (so ProRes, DNxHR etc). Internet is full of fundamentally flawed tests.

[kolak]

Originally Posted by poisondeathray

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped




Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 601) in Rec.601 and then get it back?

[poisondeathray]

Originally Posted by kolak

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 620) in Rec.601 and then get it back?

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

[kolak]

Originally Posted by poisondeathray

YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).

Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

[kolak]

Originally Posted by poisondeathray

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

[poisondeathray]

Originally Posted by kolak

Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range

It's a bit inconsistent handling, because ffmpeg dnxhr does not limit the range. Neither do official implementations of v210 or dnxhr

[kolak]

Originally Posted by poisondeathray

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range

This comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion

[poisondeathray]

Originally Posted by kolak

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

You're talking about slightly different things -

Think about it - you're not producing a file in float. It's for intermediate. You're not clamping anything

Recall , the topic was question was changing 709/601 in YUV. A colormatrix transform, as if the file in YUV had used 709 instead (or 601 instead)

The background info was intermediate RGB preview (recall vdub uses 601) , was different than using 709

Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")

And this is correct - You can apply the colormatrix transorm in YUV444PS . It's reversible. You can get the original file's code values back

[kolak]

Originally Posted by poisondeathray

You're talking about slightly different things -

Yep, different things.

[poisondeathray]

Originally Posted by kolak

This is comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion

Yes, but "v210" exported from Adobe, Resolve , etc, retain 0-1023 code values. ffmpeg v210 does not.

Do you see the problem ?

Yeah, it's not important in the big picture; but you better believe that "little detail" becomes important when testing decoders, psnr . Details matter. I reported it before on ffmpeg bug tracker. Nobody seems to care

At the very least there is inconsitent behaviour within ffmpeg, and between professional apps

[kolak]

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.


PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.

[poisondeathray]

Originally Posted by kolak

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.


PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.

Check if your original code values had Y <4, >1019 to begin with. Make sure export had "retain sub-black...superwhite" enabled if using resolve

Verify in other programs that the code values are there with the export. (e.g. vapoursynth, vsedit colorpicker, or avisynth can now with histogram(bits=10))

I use a wrapper function to turn right , left

Thisto(10)

function THisto(clip c, int "bits")
{
bits   = Default(bits, 8)
c.TurnRight().Histogram(bits=bits).TurnLeft()
}

ffmpeg v210 encoder is definitely not "infinity" on a ramp, and you can see it on the waveform in multiple programs. You don't need PSNR to know it's not lossless. You can see it. Even ffmpeg's own -vf waveform detects the discrepancy.

To be clear, ffmpeg can still DEcode 0-1023 from other formats e.g. if you test 10bit422 AVC input, it still outputs 0-1023 raw values. But if you encode a v210 version , that will only contain 4-1019 values.

Let me know if me to upload a demo package



(When using ffmpeg psnr, it's good idea to reset timebase - different containers use different timebases and this can cause ffmpeg to compare different frames (not important for 1 frame
obviously, but when testing "video"))

This is a 0-1023 ramp test.

Resolve v210 vs. original v210
ffmpeg -r 24 -i 1_resolve_v210.mov -r 24 -i 0_1023x720_0-1023.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
[Parsed_psnr_4 @ 00000027ffcb73c0] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

FFmpeg v210 vs. Resolve v210
ffmpeg -r 24 -i 2_ffmpeg_v210.mov -r 24 -i 1_resolve_v210.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

[Parsed_psnr_4 @ 0000001488292ec0] PSNR y:72.519000 u:inf v:inf average:75.529300 min:75.529300 max:75.529300

[kolak]

I think Resolve ramp generator also skips them, so this is why I see no issue.

[poisondeathray]

Here is a 10bit ramp package. 1024x720 perfect ramp. Each column corresponds to pixel value in Y'
https://www.mediafire.com/file/cvulp...tests.zip/file

0_1023x720_0-1023.mov is "original"

1_resolve_v210.mov is original exported from resolve

2_ffmpeg_v210.mov is generated from
ffmpeg -i 1_resolve_v210.mov -c:v v210 -an 2_ffmpeg_v210.mov



e.g ffmpeg -vf waveform
ffmpeg -i 1_resolve_v210.mov -vf waveform=g=green 1_resolve_v210.png



ffmpeg -i 2_ffmpeg_v210.mov -vf waveform=g=green 2_ffmpeg_v210.png

[kolak]

Got it now- those ramp generators in Resolve/Premiere are crap

So this is basically v210 encoder and ProRes encoder (and decoders are passing whatever is there)?

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out

DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.