Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st December 2021, 16:36   #1621  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
Yes, lutxy, the lower block is for reference. Anyway I haven't updated GradePack for some months, so better test with ex_binarize(mode="otsu") for now. Will check test31.

Testing with test31:
Code:
expr("x 80 > 255 0 ?","",lut=1)
Ok with lut=0
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 1st December 2021 at 16:45.
Dogway is offline   Reply With Quote
Old 2nd December 2021, 11:32   #1622  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Next round with qyot27's interface additions
Avisynth+ 3.7.1 test build 32 (20211202)
Code:
20211202 WIP
------------
- Fix: MinMax runtime filter family: check plane existance (e.g. error when requesting RPlaneMinMaxDifference on YV12)
- Fix: prevent x64 debug AviSynth builds from crashing in VirtualDub2 (opened through CAVIStreamSynth)
- ExtractY/U/V/R/G/B/A, PlaneToY: delete _ChromaLocation property. Set _ColorRange property to "full" if source is Alpha plane
- AviSynth interface additions: extend queryable internal environment properties.
  Since Interface version 8 IScriptEnvironment::GetEnvProperty (Avisynth.h) and avs_get_env_property (avisynth_c.h)
  interface functions can query some specific internal properties of AviSynth core. Thread count, etc..
  These are mainly for internal use but some can be useful for plugins and external applications.
  Each requested property has an identification number, they are found in avisynth.h and avisynth_c.h

  This addition brought new properties to query: host system's endianness, interface version and bugfix subversion.
  Relevant enum names start with AEP_ (cpp) or AVS_AEP_ (c) (AEP stands for Avisynth Environment Property)
  
  Details:
  
  AEP_HOST_SYSTEM_ENDIANNESS (c++) AVS_AEP_HOST_SYSTEM_ENDIANNESS (c)
    Populated by 'little', 'big', or 'middle' based on what GCC and/or Clang report at compile time.

  AEP_INTERFACE_VERSION (c++) AVS_AEP_INTERFACE_VERSION (c)
    for requesting actual interface (main) version. An long awaited function. 
    So far the actual interface version could be queried only indirectly, with trial and error, by starting from e.g. 10 then
    going back one by one until CheckVersion() did not report an exception/error code. 

    Even for V8 interface this was a bit tricky, the only way to detect was the infamous
      has_at_least_v8 = true;
      try { env->CheckVersion(8); } catch (const AvisynthError&) { has_at_least_v8 = false; }
    method.
    
    Now (starting from interface version 8.1) a direct version query is supported as well.
    Of course this (one or two direct call only) is the future.
    Programs or plugins which would like to identify older systems still must rely partially on the CheckVersion method.

    CPP interface (through avisynth.h).

      IScriptEnvironment *env = ...
      int avisynth_if_ver = 6;
      int avisynth_bugfix_ver = 0;
      try { 
        avisynth_if_ver = env->GetEnvProperty(AEP_INTERFACE_VERSION); 
        avisynth_bugfix_ver = env->GetEnvProperty(AEP_INTERFACE_BUGFIX);      
      } 
      catch (const AvisynthError&) { 
        try { env->CheckVersion(8); avisynth_if_ver = 8; } catch (const AvisynthError&) { }
      }
      has_at_least_v8 = avisynth_if_ver >= 8; // frame properties, NewVideoFrameP, other V8 environment functions
      has_at_least_v8_1 = avisynth_if_ver > 8 || (avisynth_if_ver == 8 && avisynth_bugfix_ver >= 1);
      // 8.1: C interface frameprop access fixed, IsPropertyWritable/MakePropertyWritable support, extended GetEnvProperty queries
      has_at_least_v9 = avisynth_if_ver >= 9; // future

    C interface (through avisynth_c.h)

      AVS_ScriptEnvironment *env = ...
      int avisynth_if_ver = 6; // guessed minimum
      int avisynth_bugfix_ver = 0;
      int retval = avs_check_version(env, 8);
      if (retval == 0) {
        avisynth_if_ver = 8;
        // V8 at least, we have avs_get_env_property but AVS_AEP_INTERFACE_VERSION query may not be supported
        int retval = avs_get_env_property(env, AVS_AEP_INTERFACE_VERSION);
        if(env->error == 0) {
          avisynth_if_ver = retval;
          retval = avs_get_env_property(env, AVS_AEP_INTERFACE_BUGFIX);
          if(env->error == 0)
            avisynth_bugfix_ver = retval;
        }
      }
      has_at_least_v8 = avisynth_if_ver >= 8; // frame properties, NewVideoFrameP, other V8 environment functions
      has_at_least_v8_1 = avisynth_if_ver > 8 || (avisynth_if_ver == 8 && avisynth_bugfix_ver >= 1);
      // 8.1: C interface frameprop access fixed, IsPropertyWritable/MakePropertyWritable support, extended GetEnvProperty queries
      has_at_least_v9 = avisynth_if_ver >= 9; // future
 

  AEP_INTERFACE_BUGFIX (c++) AVS_AEP_INTERFACE_BUGFIX (c)
    Denotes situations where there isn't a breaking change to the API,
    but we need to identify when a particular change, fix or addition
    to various API-adjacent bits might have occurred.  Could also be
    used when any new functions get added.

    Since the number is modelled as 'changes since API bump' and
    intended to be used in conjunction with checking the main
    AVISYNTH_INTERFACE_VERSION, whenever the main INTERFACE_VERSION
    gets raised, the value of INTERFACE_BUGFIX should be reset to zero.

    The BUGFIX version is added here with already incremented once,
    both because the addition of AVISYNTH_INTERFACE_BUGFIX_VERSION
    itself would require it, but also because it's intended to signify
    the fix to the C interface allowing frame properties to be read
    back (which was the situation that spurred this define to exist
    in the first place).

- CMake build environment:
  While we do need the compiler to support C++17 features, we can 
  get by on older GCC using CMake 3.6 and -std=c++-1z with some other fixes.
  CMAKE_CXX_STANDARD can be raised intelligently to 17 based on whether we detect CMake 3.8 or higher.
- Add AVISYNTHPLUS_INTERFACE_BUGFIX_VERSION

- Avisynth programming interface V8.1 or V9(?)):
  Add 'MakePropertyWritable' to the IScriptEnvironment (CPP interface), avs_make_property_writable (C interface)
  Add 'VideoFrame::IsPropertyWritable' (CPP interface), avs_is_property_writable (C interface)
  (AviSynth interface version will be stepped to V9 in the release version?)

    bool env->MakePropertyWritable(PVideoFrame *);
    bool VideoFrame::IsPropertyWritable();
  
  'MakePropertyWritable' is similar to 'MakeWritable' but it does not copy all bytes of the frame content in order to have a writable property set.
  
  Reason: 'propSet' is a filter which does not alter frame content, but sets the given frame property in its each GetFrame.
  So far it used MakeWritable to obtain a safely modifiable copy of frame properties, however - as a side-effect - full copy of frame content was performed.
  (env->getFramePropsRW alone does not ensure a uniquely modifiable property set, it just obtains a pointer which can be used in the property setter functions)
  (Note: frame properties of frames obtained by NewVideoFrame, MakeWritable and SubFrame are still safe to modify)
- Expr: when actual bit depth is too large for building LUT table, fallback to realtime mode.
  lut_x 1D (realtime when 32 bit) 
  lut_xy 2D (realtime when 16 or 32 bits)
- Expr: allow 'f32' as internal autoscale target (was: i8, i10, i12, i14, i16 were accepted, only integers)
  affects: 'scale_inputs' when "int", "intf", "all", "allf"
  more on that (todo: refresh docs) http://avisynth.nl/index.php/Expr
- Expr: fix conversion factor (+correct chroma scaling) when integer-to-integer full-scale automatic range scaling was required
- New: Expr: new parameter integer 'lut'
  integer 'lut' (default 0)
    0: realtime expression
    1: expression is converted to 1D lut (lut_x)
    2: expression is converted to 2D lut (lut_xy)
   Valid bit depths: lut=1 : 8-16 bits. lut=2 : 8-14 bits. Note: a 14 bit 2D lut needs (2^14)*(2^14)*2 bytes buffer in memory per plane (~1GByte)
   In lut mode some keywords are forbidden in the expression: sx, sy, sxr, syr, frameno, time, relative pixel addressing
pinterf is offline   Reply With Quote
Old 2nd December 2021, 16:40   #1623  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352


Everything seems to be working fine now. A bit sad because I lost one month of notes and drafts (the AvsPmod session file got cleansed : (

In big scripts like ex_binarize(mode="otsu") I don't see speed improvements but probably this is expected.

While updating the functions I got into a situation where the yexpr needs "intf" scale_inputs, and uexpr/vexpr requires "none" (or for 32-bits "none" and "floatUV" respectively). I could work around this by for example adding i16 (for 16-bit inputs) to the chroma expr, but this was slow so I baked the expression when 32-bit is being fed in.

Code:
# normalize to full range and back for limited range sources
# works fine over int HBD types and 32-bit float.
rangePCc = tv ? "f32 x 255 240 / *" : "f32 x"
rangeTVc = tv ? "240 255 / * "      : "     "
Another issue I found is ExtractU() sometimes doesn't work for float inputs.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 2nd December 2021, 18:34   #1624  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 996
Quote:
Originally Posted by Dogway View Post
A bit sad because I lost one month of notes and drafts (the AvsPmod session file got cleansed : (
Due to an error in the program? or how did that happen?
If the files are so important (one month of work) then the best thing to do is to make backup copies of them.
__________________
Live and let live
gispos is offline   Reply With Quote
Old 2nd December 2021, 19:03   #1625  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
I think it was my fault since I didn't enable avspmod's backups, but I'm not sure it works as I think it does.
I had a session from 24th of October, that's something.
I simply was writing something somewhere while avspmod launched, some letter got typed into the script editor and the last session got reset.
Is there an option for the current session to be stored as a "ghost" file (like Office does) until avspmod is closed?
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 3rd December 2021, 18:00   #1626  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 996
Quote:
Originally Posted by Dogway View Post
I think it was my fault since I didn't enable avspmod's backups, but I'm not sure it works as I think it does.
I had a session from 24th of October, that's something.
I simply was writing something somewhere while avspmod launched, some letter got typed into the script editor and the last session got reset.
Is there an option for the current session to be stored as a "ghost" file (like Office does) until avspmod is closed?
The _last_session_.ses is overwritten with the current session.
If the backup is active, It will be overwritten every time the script has changed and the clip is re-initialized.
Hence the name 'last_session'

So if a session is important to you, you have to save this session yourself 'manually' with your own name.
A session you have saved yourself is never changed.
A shadow copy is therefore unnecessary, but I will think about whether it makes sense to save the currently opened '_last_session_.ses'.

Sorry Ferenc, I've finished.
__________________
Live and let live
gispos is offline   Reply With Quote
Old 4th December 2021, 18:43   #1627  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
Manual save manual load, too manual for a program.

@pinterf: I noticed pixel addressing isn't being scaled with 'scale_inputs', at least for f32. Is it possible to include scaling the clip before pixels are being fetched?
Concerning 'lut' I didn't test but maybe it would come handy if it works when expression is scaled to supported bitdepth.
Also noticed there isn't a bool frame property, not sure if there's a reason for this.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 4th December 2021 at 20:20.
Dogway is offline   Reply With Quote
Old 5th December 2021, 08:05   #1628  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
scaling for pixels obtained from rel.addressing: good catch, done in my work copy.
bool frame properties do not exist (they did not exist in VapourSynth), one must use integers for that purpose.
pinterf is offline   Reply With Quote
Old 5th December 2021, 08:09   #1629  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
@gispos: when you'd like to recognize Avisynth versions which have a fixed C interface for getting frame properties, you can do that since latest 3.7.1 test32.
See changelog for an example how to identify plain v8, then 8.1 or future 9 interface versions.
pinterf is offline   Reply With Quote
Old 5th December 2021, 13:39   #1630  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
one must use integers for that purpose.
One advantage that tactic might have over bool is,
if originally some property defined as bool and you decide it aint enough and you want change to int ...
no longer a prob.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 5th December 2021, 14:02   #1631  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
It can be applied to _Interlaced, _ColorRange (there's also "extended" but oh well), _SceneChange, etc. Numbers imply several options and also forces you check against a number if either true or false.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 5th December 2021, 15:08   #1632  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
That is true, [life's a bitch - and then you die]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 5th December 2021, 18:44   #1633  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Avisynth+ 3.7.1 test build 33 (20211205)
I think it's time to make a feature freeze.
Code:
20211205 WIP
------------
- New array modifier function: ArraySet

  For memo here is the list of avaliable array manipulator functions
  - ArrayAdd - append
  - ArrayDel - delete at position
  - ArrayIns - insert before position
  - ArraySet - replace at position

  ArrayIns
  ^^^^^^^^

  ArrayIns(array_to_mod, value_to_insert, index1 [, index2, index3...])

      Insert a value into an array or into its subarray.
      Returns a new array with value_to_insert inserted into array_to_mod (1D array) or array_to_mod[index1 (, index2, index3...)] (multi-dimensional array)
      The indexes point to the insertion point. Index 0 will insert at the beginning of the array.
      Index (ArraySize) will insert after the last element (same as ArrayAdd - append)
      Original array (as with the other functions) remains untouched.

  ArrayAdd
  ^^^^^^^^

  ArrayAdd(array_to_mod, value_to_append [, index1, index2, index3...])

      Appends value to the end of an array or its subarray
      Returns a new array with value_to_append appended to array_to_mod (1D array) or array_to_mod[index1 (, index2, index3...)] (multi-dimensional array).
      Original array (as with the other functions) remains untouched.

  ArrayDel
  ^^^^^^^^

  ArrayDel(array_to_mod, index1 (, index2, index3...])

      Returns a new array in which the requested position was deleted.
      Original array (as with the other functions) remains untouched.

  ArraySet
  ^^^^^^^^

  ArraySet(array_to_mod, replacement_value, index1 [, index2, index3...])

      Returns a new array with array_to_mod[index1 (, index2, index3...)] = replacement_value
      Original array (as with the other functions) remains untouched.

- Array modifier functions to allow multidimensional subarray indexes

Example:

    ColorbarsHD()
    # array indexes are zero based
    a = []
    a=ArrayAdd(a,[1,2]) # [[1,2]]
    a=ArrayIns(a,3,0) # [3,[1,2]]
    a=ArrayAdd(a,"s1") # [3,[1,2],"s1"]
    a=ArrayAdd(a,"s2") # [3,[1,2],"s1","s2"]
    a=ArrayDel(a,2) # [3,[1,2],"s2"]
    a=ArraySet(a,"g",1,0) # [3,["g",2],"s2"]
    a=ArrayAdd(a,"h",1) # [3,["g",2,"h"],"s2"]
    a=ArrayAdd(a,[10,11,12],1) # append to (1) -> [3,["g",2,"h",[10,11,12]],"s2"]
    a=ArrayDel(a,1,3,0) # del from (1,3,0) -> [3,["g",2,"h",[11,12]],"s2"]
    a=ArrayAdd(a,"added") # [3,["g",2,"h",[11,12]],"s2","added"]
    a=ArrayAdd(a,["yet","another","sub"]) # [3,["g",2,"h",[11,12]],"s2","added",["yet","another","sub"]]
    x=a[0] #3
    x=a[1,0] #g
    x=a[1,2] #h
    x=a[1,3,1] #12
    x=a[3] #"added"
    x=a[4,1] #"another"
    SubTitle("x = " + String(x) + " Size=" + String(a.ArraySize()))

- Expr: allow auto scaling effect on pixels obtained from relative addressing
- ConvertBits: ordered dither: possible to dither down with more than 8 bits difference like in
    Clip16.ConvertBits(8, dither=0, dither_bits=4)
  Such conversion is made in two phases. First the clip is converted to (dither_bits+8) bits; in the above example it is 12.
  If the temporary bit depth would be odd (no 9 or 11 bit support in Avisynth+) then it is made even.
  bit depth that differs in only 8 bits for the target. Then this intermediate clip is converted to the required end target.
- Quicker ClearProperties and CopyProperties filters (by using MakePropertyWritable instead of MakeWritable).
pinterf is offline   Reply With Quote
Old 5th December 2021, 20:45   #1634  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Thanks for the new version!

Quote:
Originally Posted by pinterf View Post
I think it's time to make a feature freeze.
Oh, so this is gonna be the extensive testing part and Test 33 is the release candidate before the stable release?!
And perhaps the stable will be released for Christmas?

FranceBB is offline   Reply With Quote
Old 5th December 2021, 22:24   #1635  |  Link
gispos
Registered User
 
Join Date: Oct 2018
Location: Germany
Posts: 996
Quote:
Originally Posted by Dogway View Post
Manual save manual load, too manual for a program.
It is automatically backed up and restored, and if you want to back up something, no program can only do it with good talk.
But I can also save all sessions with the date... and nobody looks through it afterwards and knows what is in the files.

There will be a backup copy of the last session and a previous one.

Quote:
Originally Posted by pinterf View Post
@gispos: when you'd like to recognize Avisynth versions which have a fixed C interface for getting frame properties, you can do that since latest 3.7.1 test32.
See changelog for an example how to identify plain v8, then 8.1 or future 9 interface versions.
Thanks Ferenc, had already read that.
And since this is not present in older versions, I had already determined it this way:
From header version 8 only the matrix is read (if necessary with Eval), and from version 3.71 the properties are also read.

Edit: Where can I find your integer properties value conversion to string?
__________________
Live and let live

Last edited by gispos; 5th December 2021 at 22:28.
gispos is offline   Reply With Quote
Old 6th December 2021, 01:39   #1636  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
pinterf, do you know if Clip32.Expr(a,b,"x y - abs 128 > 255 0 ?", lut=2, scale_inputs="all") is a compatible lut expression? I think this is not documented.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 6th December 2021, 09:09   #1637  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by Dogway View Post
pinterf, do you know if Clip32.Expr(a,b,"x y - abs 128 > 255 0 ?", lut=2, scale_inputs="all") is a compatible lut expression? I think this is not documented.
Valid, because when the given bit depth is not available for LUT mode it fallbacks to realtime (lut=0). You said you didn't like error messages

Code:
- Expr: when actual bit depth is too large for building LUT table, fallback to realtime mode.
  lut_x 1D (realtime when 32 bit) 
  lut_xy 2D (realtime when 16 or 32 bits)
pinterf is offline   Reply With Quote
Old 6th December 2021, 13:20   #1638  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
Sure, it's not about error messages but if it is possible to use lut calculations on scaled_inputs. The above for example could use an 8-bit lut table.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 6th December 2021, 14:04   #1639  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Doggie, Probably slower than realtime. [I think]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 8th December 2021, 01:02   #1640  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
test33 crashes avspmod with a simple bilinearresize(), same in avsmeter, has this to do with the interface change in test32? Strange because I was using test32 without issues.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.