Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
13th June 2018, 14:22 | #4122 | Link | |
Retried Guesser
Join Date: Jun 2012
Posts: 1,373
|
Quote:
That's not a bug, it's a consequence of 8-bit math doing something (round trip) it wasn't designed to do. Attempting to fix it results in a slightly worse one-way conversion. Increasing the source bit depth decreases the error; with 32-bit float it's un-measureable. Alternatively, when you know you are doing a round trip, you could add an offset manually: Code:
function s(clip c) { return c.ColorYUV(levels="TV->PC") \ .ColorYUV(levels="PC->TV", off_y=1, off_u=1, off_v=1) } |
|
15th June 2018, 09:01 | #4123 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
New build r2719
https://drive.google.com/open?id=1gA...Zn31Z8JLfZJ3yo And a test masktools2 (2.2.15) that matches this avisynth version https://drive.google.com/open?id=1H0...0hVlXCwd2c-afI I'm posting both test builds here, because they are tightly connected. real.finder, who is responsible with many old scripts, asked me for these features for a long time. And I resisted because I was against such autoconversion. Anyway, you can start experimenting but you have to understand when you can use this feature safely (but maybe it's less prone to errors than mis-using scaleb and scalef) - LUT functions in masktools2 and Expr in Avisynth+ have now two new parameters: scale_inputs and clamp_float (see below). In same cases they allow a quite a convenient way of using old 8 bit lut expressions to be used for generic bit-depth. - The other feature in masktools2 is the new 'use_expr' parameter. This parameter allows passing the whole expression to Avisynth+ when the lut filter would use the slow interpreted realtime way of expression calculation (for 16 bits no xy lut table exists because it is too big to fit in the memory). Try setting use_expr=1 for mt_lutxy (that will pass the expression to Avisynth for 10 or more bits) will drastically speed up a 16bit mt_lutxy. Avisynth: Code:
- New: Expr: implement 'clip' three operand operator like in masktools2 Description: clips (clamps) value: x minvalue maxvalue clip -> max(min(x, maxvalue), minvalue) - New: Expr: Parameter "clamp_float" True: clamps 32 bit float to valid ranges, which is 0..1 for Luma or for RGB color space and -0.5..0.5 for YUV chroma UV channels Default false, ignored (treated as true) when scale_inputs scales float - New: Expr: parameter "scale_inputs" (default "none") Autoscale any input bit depths to 8-16 bit for internal expression use, the conversion method is either full range or limited YUV range. Feature is similar to the one in masktools2 v2.2.15 The primary reason of this feature is the "easy" usage of formerly written expressions optimized for 8 bits. Use - "int" : scales limited range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16') - "intf": scales full range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16') - "float" or "floatf" : only scales 32 bit float format to 8 bit range (or bit depth specified by 'i8'..'i16') - "all": scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses limited_range logic (mul/div by two's power) - "allf": scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses full scale logic (stretch) - "none": no magic Usually limited range is for normal YUV videos, full scale is for RGB or known-to-be-fullscale YUV By default the internal conversion target is 8 bits, so old expressions written for 8 bit videos will probably work. This internal working bit-depth can be overwritten by the i8, i10, i12, i14, i16 specifiers. When using autoscale mode, scaleb and scalef keywords are meaningless, because there is nothing to scale. How it works: - This option scales all 8-32 bit inputs to a common bit depth value, which bit depth is 8 by default and can be set to 10, 12, 14 and 16 bits by the 'i10'..'i16' keywords For example: scale_inputs="all" converts any inputs to 8 bit range. No truncation occurs however (no precision loss), because even a 16 bit data is converted to 8 bit in floating point precision, using division by 256.0 (2^16/2^8). So the conversion is _not_ a simple shift-right-8 in the integer domain, which would lose precision. - Calculates expression (lut, lut_xy, lut_xyz, lut_xyza) - Scales the result back to the original video bit depth. Clamping (clipping to valid range) and converting to integer occurs here. The predefined constants such as 'range_max', etc. will behave according to the internal working bit depth Warning#1 This feature was created for easy porting earlier 8-bit-video-only lut expressions. You have to understand how it works internally. Let's see a 16bit input in "all" and "allf" mode (target is the default 8 bits) Limited range 16->8 bits conversion has a factor of 1/256.0 (Instead of shift right 8 in integer domain, float-division is used or else it would lose presision) Full range 16->8 bits conversion has a factor of 255.0/65535 Using bit shifts (really it's division and multiplication by 2^8=256.0): result = calculate_lut_value(input / 256.0) * 256.0 Full scale 16-8-16 bit mode ('intf', 'allf') result = calculate_lut_value(input / 65535.0 * 255.0 ) / 255.0 * 65535.0 Use scale_inputs = "all" ("int", "float") for YUV videos with 'limited' range e.g. in 8 bits: Y=16..235, UV=16..240). Use scale_inputs = "allf" (intf, floatf) for RGB or YUV videos with 'full' range e.g. in 8 bits: channels 0..255. When input is 32bit float, the 0..1.0 (luma) and -0.5..0.5 (chroma) channel is scaled to 0..255 (8 bits), 0..1023 (i10 mode), 0..4095 (i12 mode), 0..16383(i14 mode), 0..65535(i16 mode) then back. Warning#2 One cannot specify different conversion methods for converting before and after the expression. Neither can you specify different methods for different input clips (e.g. x is full, y is limited is not supported). Code:
- 32 bit float U and V chroma channels are now zero based (+/-0.5 for full scale). Was: 0..1, same as luma (Following the change in Avisynth+ over r2664: use this plugin with r2996 or newer) Affected predefined expression constants when plane is U or V: cmin and cmax (limited range (16-128)/255 and (240-128)/255 instead of 16/255.0 and 240/255.0 range_max: 0.5 instead of 1.0 new: introduce range_min: -0.5 for float U/V chroma, 0 otherwise range_half (0.0 instead of 0.5) (range_size remained 1.0) - New expression syntax for Lut expressions: autoscale any input (x,y,z,a) bit depths to 8-16 bits for internal expression use. The primary reason of this feature is the "easy" usage of formerly written 8 bit optimized expressions. New parameters for lut functions: String "scale_inputs": "all","allf","int","intf","float","floatf","none", default "none" and Boolean "clamp_float": default false, but treated as always true (and thus ignored) when scale_inputs involves a float autoscale. and Boolean "use_expr": default 0, calls fast JIT-compiled "Expr" in Avisynth+ for mt_lut, lutxy, lutxyz, lutxyza 0: no Expr, use slow internal realtime calc if needed (as before) 1: call Expr for bits>8 or lutxyza 2: call Expr, when masktools would do its slow realtime calc (see 'realtime' column in the table above) Extends and replaces experimental clamp_xxxx keywords. Last edited by pinterf; 15th June 2018 at 11:47. |
15th June 2018, 11:33 | #4126 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
expr scale seems work fine
but I got mt_lutXX does not have a named argument "scale_inputs" in xp I used in VM, same for use_expr I did check the dll date and it seems very old!
__________________
See My Avisynth Stuff |
15th June 2018, 11:44 | #4128 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
google drive maybe give the 1st test you did not the overwritten new one
__________________
See My Avisynth Stuff |
16th June 2018, 12:51 | #4131 | Link | |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
Quote:
__________________
See My Avisynth Stuff |
|
17th June 2018, 21:11 | #4132 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
Thanks, real.finder for the report. scale_inputs "float" with input bit depth integer was misinterpreted. r2721 fixes that problem.
AviSynth+ r2721 test build (hopefully one of the last tests): https://drive.google.com/open?id=1Qk...wN9QyQMK-vHhG6 Matching masktools2 2.2.15 test 5 (not changed since last version) https://drive.google.com/open?id=1H0...0hVlXCwd2c-afI Last edited by pinterf; 17th June 2018 at 21:22. |
22nd June 2018, 04:19 | #4133 | Link | ||
Registered User
Join Date: Sep 2005
Posts: 178
|
Quote:
Quote:
Edit: I think I can cook up a better example with the Histogram() function that doesn't involve repeated conversions. I'll post back here when I have time (unless that patch has since made it into the release builds). Last edited by `Orum; 22nd June 2018 at 04:22. |
||
26th June 2018, 14:06 | #4134 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
And some test builds again.
Fixes 'scalef' and 'scaleb' for U/V chroma parts of 32bit float formats (bith in MaskTools2 lut and Avisynth+ Expr) Thanks to real.finder for the report. Not tightly avs+ related but since I put them here together: Finished a new feature for masktools2, requested by real.finder a long time ago: New masktools2 parameter: 'cplace' for mt_merge. Parameter String 'cplace' : "mpeg1" or "mpeg2" (default) chroma placement for 4:2:0 formats when luma=true. Avisynth+ r2722 (installer w/o VC redistributables) https://drive.google.com/open?id=1N3...qg_w0Ur2AxrE_n Masktools2 2.2.15_test8 https://drive.google.com/open?id=1g-...LIiOpzctMjKhoB Last edited by pinterf; 28th June 2018 at 10:38. Reason: update link masktools test7 to test8 |
27th June 2018, 23:04 | #4136 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
it's added in avs26 not avs+
__________________
See My Avisynth Stuff |
2nd July 2018, 14:31 | #4138 | Link |
Registered User
Join Date: Jan 2014
Posts: 2,314
|
New release with plenty of new features and changes.
Download Avisynth+ r2728 Most of the changes are already documented on avisynth.nl pages. Note: This release will soon be followed by some updated plugins, mostly because of the 32 bit float format internal changes: masktools2 2.2.17 updated FFT3DFilter 2.5 updated RgTools 0.97 updated Code:
20180702 r2728 -------------- - New: Expr: implement 'clip' three operand operator like in masktools2 Description: clips (clamps) value: x minvalue maxvalue clip -> max(min(x, maxvalue), minvalue) - New: Expr: Parameter "clamp_float" True: clamps 32 bit float to valid ranges, which is 0..1 for Luma or for RGB color space and -0.5..0.5 for YUV chroma UV channels Default false, ignored (treated as true) when scale_inputs scales float - New: Expr: parameter "scale_inputs" (default "none") Autoscale any input bit depths to 8-16 bit for internal expression use, the conversion method is either full range or limited YUV range. Feature is similar to the one in masktools2 v2.2.15 The primary reason of this feature is the "easy" usage of formerly written expressions optimized for 8 bits. Use - "int" : scales limited range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16') - "intf": scales full range videos, only integer formats (8-16bits) to 8 (or bit depth specified by 'i8'..'i16') - "float" or "floatf" : only scales 32 bit float format to 8 bit range (or bit depth specified by 'i8'..'i16') - "all": scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses limited_range logic (mul/div by two's power) - "allf": scales videos to 8 (or bit depth specified by 'i8'..'i16') - conversion uses full scale logic (stretch) - "none": no magic Usually limited range is for normal YUV videos, full scale is for RGB or known-to-be-fullscale YUV By default the internal conversion target is 8 bits, so old expressions written for 8 bit videos will probably work. This internal working bit-depth can be overwritten by the i8, i10, i12, i14, i16 specifiers. When using autoscale mode, scaleb and scalef keywords are meaningless, because there is nothing to scale. How it works: - This option scales all 8-32 bit inputs to a common bit depth value, which bit depth is 8 by default and can be set to 10, 12, 14 and 16 bits by the 'i10'..'i16' keywords For example: scale_inputs="all" converts any inputs to 8 bit range. No truncation occurs however (no precision loss), because even a 16 bit data is converted to 8 bit in floating point precision, using division by 256.0 (2^16/2^8). So the conversion is _not_ a simple shift-right-8 in the integer domain, which would lose precision. - Calculates expression (lut, lut_xy, lut_xyz, lut_xyza) - Scales the result back to the original video bit depth. Clamping (clipping to valid range) and converting to integer occurs here. The predefined constants such as 'range_max', etc. will behave according to the internal working bit depth Warning#1 This feature was created for easy porting earlier 8-bit-video-only lut expressions. You have to understand how it works internally. Let's see a 16bit input in "all" and "allf" mode (target is the default 8 bits) Limited range 16->8 bits conversion has a factor of 1/256.0 (Instead of shift right 8 in integer domain, float-division is used or else it would lose presision) Full range 16->8 bits conversion has a factor of 255.0/65535 Using bit shifts (really it's division and multiplication by 2^8=256.0): result = calculate_lut_value(input / 256.0) * 256.0 Full scale 16-8-16 bit mode ('intf', 'allf') result = calculate_lut_value(input / 65535.0 * 255.0 ) / 255.0 * 65535.0 Use scale_inputs = "all" ("int", "float") for YUV videos with 'limited' range e.g. in 8 bits: Y=16..235, UV=16..240). Use scale_inputs = "allf" (intf, floatf) for RGB or YUV videos with 'full' range e.g. in 8 bits: channels 0..255. When input is 32bit float, the 0..1.0 (luma) and -0.5..0.5 (chroma) channel is scaled to 0..255 (8 bits), 0..1023 (i10 mode), 0..4095 (i12 mode), 0..16383(i14 mode), 0..65535(i16 mode) then back. Warning#2 One cannot specify different conversion methods for converting before and after the expression. Neither can you specify different methods for different input clips (e.g. x is full, y is limited is not supported). - Fix: Expr: expression string order for planar RGB is properly r-g-b like in original VapourSynth version, instead of counter-intuitive g-b-r. - Fix: Expr: check subsampling when a different output pixel format is given - Fix: ColorYUV: round to avoid green cast on consecutive TV<>PC - Enhanced: Limiter to work with 32 bit float clips - Enhanced: Limiter new parameter bool 'autoscale' default false. If set, minimum/maximum luma/chroma values are treated as they were in 8 bit range (but non-integer values are allowed), limiter will autoscale it. Default: does not scale at all, parameters are used as-is. Parameters now are of float type to handle 32 bit float values. - New: function bool VarExist(String variable_name) Checks if a variable exists Returns true if variable exists even if it holds 'Undefined', false otherwise - Fix: RGBAdjust memory leak when used in ScriptClip - Enhanced: RGBAdjust new parameter: conditional (like in ColorYUV) The global variables "rgbadjust_xxx" with xxx = r, g, b, a, rb, gb, bb, ab, rg, gg, bg, ag are read each frame, and applied. It is possible to modify these variables using FrameEvaluate or ConditionalReader. - Enhanced: RGBAdjust: support 32 bit float ('analyze' not supported, 'dither' silently ignored) - Enhanced: AviSource to support more formats with 10+ bit depth. http://avisynth.nl/index.php/AviSource When pixel_type is not specified or set to "FULL", AviSource will try to request the formats one-by-one in the order shown in the table below. When a classic 'pixel_type' shares more internal formats (such as YUV422P10 first tries to request the v210 then P210 format) you can specify one of the specific format directly. Note that high bit-depth RGBP is prioritized against packed RGB48/64. The 'FourCCs for ICDecompressQuery' column means that when a codec supports the format, it will serve the frame in that one, Avisource then will convert it to the proper colorspace. Full support list (* = already supported): 'pixel_type' Avs+ Format FourCC(s) for ICDecompressQuery YV24 YV24 *YV24 YV16 YV16 *YV16 YV12 YV12 *YV12 YV411 YV411 *Y41B YUY2 YUY2 *YUY2 RGBP10 RGBP10 G3[0][10] r210 R10k r210 RGBP10 r210 R10k RGBP10 R10k RGBP RGBP10 G3[0][10] r210 R10k RGBP12 G3[0][12] RGBP14 G3[0][14] RGBP16 G3[0][16] RGBAP10 G4[0][10] RGBAP12 G4[0][12] RGBAP14 G4[0][14] RGBAP16 G4[0][16] RGB32 RGB32 *BI_RGB internal constant (0) with bitcount=32 RGB24 RGB24 *BI_RGB internal constant (0) with bitcount=24 RGB48 RGB48 BGR[48] b48r RGB64 RGB64 *BRA[64] b64a Y8 Y8 Y800 Y8[32][32] GREY Y Y8 Y800 Y8[32][32] GREY Y10 Y1[0][10] Y12 Y1[0][12] Y14 Y1[0][14] Y16 Y1[0][16] YUV422P10 YUV422P10 v210 P210 v210 YUV422P10 v210 P210 YUV422P10 P210 YUV422P16 YUV422P16 P216 P216 YUV422P16 P216 YUV420P10 YUV420P10 P010 P010 YUV422P10 P010 YUV420P16 YUV420P16 P016 P016 YUV422P16 P016 YUV444P10 YUV444P10 v410 v410 YUV444P10 v410 - Changed (finally): 32bit float YUV colorspaces: zero centered chroma channels. U and V channels are now -0.5..+0.5 (if converted to full scale before) instead of 0..1 Note: filters that relied on having the U and V channel center as 0.5 will fail. Why: the old UV 0..1 range was a very-very early decision in the high-bitdepth transition project. Also it is now compatible with z_XXXXX resizers (zimg image library, external plugin at the moment). - New function: bool IsFloatUvZeroBased() For plugin or script writers who want to be compatible with pre r2672 Avisynth+ float YUV format: Check function availablity with FunctionExists("IsFloatUvZeroBased"). When the function does not exists, the center value of 32 bit float U and V channel is 0.5 When IsFloatUvZeroBased function exists, it will return true (always for official releases) if U and V is 0 based (+/-0.5) - Fix: RGB64 Turnleft/Turnright (which are also used in RGB64 Resizers) - Fix: Rare crash in FrameRegistry - Enhanced: Allow ConvertToRGB24-32-48-64 functions for any source bit depths - Enhanced: ConvertBits: allow fulls-fulld combinations when either clip is 32bits E.g. after a 8->32 bit fulls=false fulld=true: Y: 16..235 -> 0..1 U/V: 16..240 -> -0.5..+0.5 Note: now ConvertBits does not assume full range for YUV 32 bit float. Default values of fulls and fulld are now true only for RGB colorspaces. - Fix: couldn't see variables in avsi before plugin autoloads (colors_rgb.avsi issue) - Fix: LoadVirtualdubPlugin: Fix crash on exit when more than one instances of a filter was used in a script - New: LoadVirtualdubPlugin update: - Update from interface V6 to V20, and Filtermod version 6 (partial) - VirtualDub2 support with extended colorspaces Allow RGB24, RGB48, RGB64 besides RGB32 AutoConvert 8 bit Planar RGB to/from RGB24, RGBPA to/from RGB32 (lossless) AutoConvert RGB48 and 16 bit Planar RGB(A) to/from RGB64 (lossless) Support YUV(A) 8 bits: YV12, YV16, YV24, YV411, YUVA420P8, YUVA422P8, YUVA444P8 Support YUV(A) 10-16 bits (properly set "ref_x" maximum levels, no autoconvert) - Supports prefetchProc2 callback (API >= V14 and prefetchProc2 is defined) for multiple input frames from one input clip PrefetchFrameDirect and PrefetchFrame are supported. PrefetchFrameSymbolic not supported - Supports prefetchProc callback (API >= V12 and prefetchProc is defined) - Supports when filter changes frame count of the output clip - Extra filter parameter added at the end of filter's (unnamed) parameter list Imported Virtualdub filters are getting and extra named parameter to the end: String [rangehint] This parameter can tell the filter about a YUV-type clip colorspace info Allowed values: "rec601": limited range + 601 "rec709": limited range + 709 "PC.601": full range + 601 "PC.709": full range + 709 "" : not defined (same as not given) Parameter will be ignored when clip is non-YUV How it works: the hint will _not_ change the internal VirtualDub colorspace constant (e.g. kPixFormat_YUV420_Planar -> kPixFormat_YUV420_Planar_709 will not happen). Instead the base color space is kept and colorSpaceMode and colorRangeMode will set in PixmapLayout.formatEx. Filter can either use this information or not, depending on supported API version and its implementation. E.g. Crossfade(20,30) -> Crossfade(20,30,"rec601") though this specific filter won't use it. - New function: BuildPixelType Creates a video format (pixel_type) string by giving a colorspace family, bit depth, optional chroma subsampling and/or a template clip, from which the undefined format elements are inherited. "[family]s[bits]i[chroma]i[compat]b[oldnames]b[sample_clip]c" string family: YUV, YUVA, RGB, RGBA, Y int bits: 8, 10, 12, 14, 16, 32 string chroma: for YUV(A) 420,422,444,411. Ignored for RGB(A) and Y bool compat (default false): returns packed rgb formats for 8/16 bits (RGB default: planar RGB) bool oldnames (default false): returns YV12/YV16/YV24 instead of YUV420P8/YUV422P8/YUV444P8 clip sample_clip: when supported, its format is overridden by specified parameters (e.g. only change bits=10) Example#1: define YUV 444 P 10 family = "YUV" bits = 10 chroma = 444 compat = false oldformat = false s = BuildPixelType(family, bits, chroma, compat, oldformat) BlankClip(width=320,height=200,length=len,pixel_type=s,color=$008080).Info() Example#2: Change only the bit depth of the format to 16 newbits = 16 c = last s = BuildPixelType(bits=newbits, sample_clip=c) BlankClip(width=320,height=200,length=len,pixel_type=s,color=$008080).Info() - Source: move to c++17, 'if constexpr' requires. Use Visual Studio 2017 (or GCC 7?). CMakeLists.txt changed. - Source: C api: AVSC_EXPORT to dllexport in capi.h for avisynth_c_plugin_init - Source: C api: avs_is_same_colorspace VideoInfo parameters to const - Project struct: changelog to git. - Include current avisynth header files and def/exp file in installer, when SDK is chosen Last edited by pinterf; 10th July 2018 at 15:08. Reason: filter dependencies updated mt2 changed again |
2nd July 2018, 15:52 | #4139 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
__________________
Groucho's Avisynth Stuff |
|
|