Thread: Avisynth+
View Single Post
Old 21st August 2019, 15:16   #4824  |  Link
magiblot
Eurobeat Fan
 
Join Date: Sep 2014
Posts: 108
Quote:
Originally Posted by FranceBB View Post
32bit float:
Full Range 0.0-0.99609375 / Limited TV Range: 0.0625-0.91796875
This is what theory says, as explained in http://avisynth.nl/index.php/Autoscale_parameter, http://avisynth.nl/index.php/Convert and other places:

Quote:
* The special-purpose matrices PC.601 and PC.709 keep the range unchanged, instead of converting between 0d-255d RGB and 16d-235d YUV, as is the normal practice.
Which implies that these ranges apply to RGB as well. And this makes sense to me as using multiples of 1/256 should help avoid rounding errors.

However, the question is: do filters actually expect pixel values to be in that range?

ConvertBits, for instance, seems not to:

Quote:
Conversion from and to float is always full-scale.
Practical examples show different behaviours:

Code:
BlankClip(height=240).ConvertToPlanarRGB()
Expr(String(255.0/256.0), format="RGBPS") # 255d
StackVertical( Expr("x 256 *", format="RGBP8").ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("'D notation'-compliant conversion"),
             \ ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("Default conversion") )


Code:
BlankClip(pixel_type="YV24")
Expr("0.0", format="YUV444PS")
ColorYUV(levels="PC->TV") # Bug in ColorYUV? Picture turns green
ColorYUV(analyze=true) # OK: luma is 0.6250 == 16/256

Expr("1.0", format="YUV444PS")
ColorYUV(levels="PC->TV")
ColorYUV(analyze=true) # Confusing: luma is 0.91796875 == 235/256, but input expected to be 1.0 rather than 255/256

Expr(String(235.0/256.0), format="YUV444PS")
ColorYUV(levels="TV->PC")
ColorYUV(analyze=true) # Same as before: 235/256 translates into 1.0 instead of 255/256

BlankClip(color_yuv=$108080, pixel_type="YV24") # Plain black in limited range
ConvertBits(32)
ColorYUV(analyze=true) # Inconsistent: 16d represented as 16/255 == 0.062745... (periodic number)
From ColorYUV's behaviour, one could assume that [16d..235d] is used for limited range and [0..1] for full range. But, by looking at ConvertBits, one would deduce that [16/255..235/255] and [0..1] are to be used instead. Documentation, conversely, speaks of [16d..235d] and [0d..255d].

In addition, RGBAdjust and Compare are unable to analyze floating-point pixel types, which makes it more difficult to see what's going on.

Last edited by magiblot; 21st August 2019 at 15:57.
magiblot is offline