Quote:
Originally Posted by FranceBB
32bit float:
Full Range 0.0-0.99609375 / Limited TV Range: 0.0625-0.91796875
|
This is what theory says, as explained in
http://avisynth.nl/index.php/Autoscale_parameter,
http://avisynth.nl/index.php/Convert and other places:
Quote:
* The special-purpose matrices PC.601 and PC.709 keep the range unchanged, instead of converting between 0d-255d RGB and 16d-235d YUV, as is the normal practice.
|
Which implies that these ranges apply to RGB as well. And this makes sense to me as using multiples of 1/256 should help avoid rounding errors.
However, the question is: do filters actually expect pixel values to be in that range?
ConvertBits, for instance, seems not to:
Quote:
Conversion from and to float is always full-scale.
|
Practical examples show different behaviours:
Code:
BlankClip(height=240).ConvertToPlanarRGB()
Expr(String(255.0/256.0), format="RGBPS") # 255d
StackVertical( Expr("x 256 *", format="RGBP8").ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("'D notation'-compliant conversion"),
\ ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("Default conversion") )
Code:
BlankClip(pixel_type="YV24")
Expr("0.0", format="YUV444PS")
ColorYUV(levels="PC->TV") # Bug in ColorYUV? Picture turns green
ColorYUV(analyze=true) # OK: luma is 0.6250 == 16/256
Expr("1.0", format="YUV444PS")
ColorYUV(levels="PC->TV")
ColorYUV(analyze=true) # Confusing: luma is 0.91796875 == 235/256, but input expected to be 1.0 rather than 255/256
Expr(String(235.0/256.0), format="YUV444PS")
ColorYUV(levels="TV->PC")
ColorYUV(analyze=true) # Same as before: 235/256 translates into 1.0 instead of 255/256
BlankClip(color_yuv=$108080, pixel_type="YV24") # Plain black in limited range
ConvertBits(32)
ColorYUV(analyze=true) # Inconsistent: 16d represented as 16/255 == 0.062745... (periodic number)
From ColorYUV's behaviour, one could assume that [16d..235d] is used for limited range and [0..1] for full range. But, by looking at ConvertBits, one would deduce that [16/255..235/255] and [0..1] are to be used instead. Documentation, conversely, speaks of [16d..235d] and [0d..255d].
In addition, RGBAdjust and Compare are unable to analyze floating-point pixel types, which makes it more difficult to see what's going on.