Avisynth+

[pinterf]

Originally Posted by TheFluff

Continuing the discussion from here, let's talk about those nasty memory offsets again.


In Avs+ all these size_t's are (32-bit signed) int instead. In Avisynth it doesn't really matter in practice because a frame is always allocated as one big chunk of memory, and nobody really has any use for more than 2^31 bytes of vfb. If you ever wanted to stop doing this though and use one individual pointer per plane like VS does, this won't fly because you can't reliably stuff a 64-bit pointer in a 32-bit integer, and it's generally bad coding practice to not use size_t (or ptrdiff_t) for these things.

Way back in the day ultim claimed that he didn't want to follow IanB's lead and switch to size_t because it'd break a number of completely irrelevant plugins (that would have been trivial to recompile). I didn't realize it at the time, but I'm pretty sure that in practice, he was wrong even on the technical aspect. If you look at these functions above, the only one that conceivably might end up getting called by plugins in reality is VideoFrame::GetOffset(), and I'm 99% sure that its return value just gets passed in a register and zero-extended, so as long as you keep passing the same old less-than-2^31 offsets, everything will keep working just fine (but most plugins just use GetRead/WritePtr). New VideoFrames and VFB's are only constructed by env->NewVideoFrame, so changing the signature of the VideoFrame constructor shouldn't be an issue.

Thanks, then it's worth a try.

[Myrsloik]

Since we're on the subject of changes. Why was the return type of GetCPUFlags() changed from long to int? That one makes absolutely no sense to me.

[pinterf]

Originally Posted by Myrsloik

Since we're on the subject of changes. Why was the return type of GetCPUFlags() changed from long to int? That one makes absolutely no sense to me.

Another piece of history:
"Standardize some type usage to prevent confusion of devs with GCC background."
https://github.com/AviSynth/AviSynth...dd60d203ea9504

[pinterf]

Originally Posted by enctac

r2508 RGB->Y8 [incorrect]
r2508 RGB->YV24->Y8 [correct]
2.6MT RGB->Y8 [correct]

Thanks for the report, fixed. Rec601, Rec709 (limited range) RGB->Y conversion was affected. Regression since r2266 (a lot of high bit depth work at that time)

[pinterf]

Originally Posted by pinterf

Thanks, then it's worth a try.

Turned out that programs using C interface could be broken.

Current x264 is one of such programs, it fails, because it reads zero pitches (strides).

x264 [error]: Input picture width (640) is greater than stride (0)
x264 [error]: x264_encoder_encode failed

Having a look at the x264 source code, it contains a hybride avisynth_c.h (I guess that it is a mix of some previous headers from the "classic" avs line, inserted new high bit depth stuff from current avs+) in which the function avs_get_read_ptr_p is inlined and directly accesses the fields of AVS_VideoFrame struct
It fails because when we change the type of offset-like variables from int (32 bits) to size_t (64 bits on x64), the positions of fields in AVS_VideoFrame are shifted and a "baked" reference to the "pitch" now becomes the higher 32 bits of "offset", which is usually zero.

And that's only for x264. Anyway, when I'll make a test build, I will provide this "offsets are size_t instead of int" version as a separate test version.

If it could be changed it may work for current (int) and future (size_t) version of avisynth. Maybe it was inlined purposely, because older avisynth versions did not support avs_get_read_ptr_p through C interface (did not have time to dig into its history)?

To tell the truth, I had a look at the current avisynth_c.h from avs+ project, and although this specific call is O.K. in current version, there are two other problematic functions, namely
avs_get_row_size and avs_get_height (the ones without a "plane" parameter) so they apply on plane 0 (PLANAR_Y).

These functions are still directly accessing the AVS_VideoFrame contrary to the big warning "DO NOT USE THIS STRUCTURE DIRECTLY", so c interface header in avs+ is still inconsistent, at least for AVS_VideoFrame.

[TheFluff]

Ah, I hadn't considered that. Tricky.

[Shirtfull]

Originally Posted by pinterf

Thanks for the report, fixed. Rec601, Rec709 (limited range) RGB->Y conversion was affected. Regression since r2266 (a lot of high bit depth work at that time)

Rgb24 to yv12 seems incorrect as well, Frame served Progressive/interlaced testcube RGB24 from Virtualdub/VirtualDub_FilterMod to AvsPmod.

Rgb>yv12



Rgb>yv16

[pinterf]

Originally Posted by Shirtfull

Rgb24 to yv12 seems incorrect as well, Frame served Progressive/interlaced testcube RGB24 from Virtualdub/VirtualDub_FilterMod to AvsPmod.

Rgb>yv12

Rgb>yv16

I can see something like a rotation blur on the yv12 sample. How did you achieve that exactly?

[Shirtfull]

SetFilterMTMode("AVISource", x)  #2or3
AVISource("Path to frameserve file.avi").ConvertToYv12() #12 or 16
bob()
Prefetch (4)

Only difference between pics was the conversion.

Take out bob, they both look the same.

[pinterf]

Originally Posted by Shirtfull

SetFilterMTMode("AVISource", x)  #2or3
AVISource("Path to frameserve file.avi").ConvertToYv12() #12 or 16
bob()
Prefetch (4)

Only difference between pics was the conversion.

Take out bob, they both look the same.

Specify interlaced=true in ConvertToYV12.
RGB->YV12 conversion is a two-phase conversion. First comes RGB->YV24, then YV24->YV12. This latter needs the hint that the clip is an interlaced one.
(remark: parameter "interlaced" is used only in conversions where yv12 is involved either as source or target)

[Shirtfull]

Thanks, that fixed it.

[wonkey_monkey]

Originally Posted by Shirtfull

SetFilterMTMode("AVISource", x)  #2or3
AVISource("Path to frameserve file.avi").ConvertToYv12() #12 or 16
bob()
Prefetch (4)

Only difference between pics was the conversion.

Take out bob, they both look the same.

Anyone know why it looks like some kind of rotation with the bob() in there? Seems weird.

[shekh]

Originally Posted by davidhorman

Anyone know why it looks like some kind of rotation with the bob() in there? Seems weird.

I think VD does not have way to request interlaced conversion to YV12.
Looks better:
deinterlace (unfold)
convert format (yv12)
deinterlace (fold)

[Shirtfull]

The other way round, using VDFiltermod to generate cube and then frame-serving to avisynth. I recall it can only serve RGB.

[johnmeyer]

It appears that Prefetch cannot be used in a script which uses Return.

Here is my test. The following script runs at the same speed with, or without, the Prefetch statement. It is clear that multi-threading is NOT being used.

LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")
source=AVISource("E:\fs.avi").killaudio().ConvertToYV12()
output=MDegrain2i2(source,8,4,400,0)  
return output
Prefetch(5)

If I move Prefetch to the line before the "return output" statement, the script throws an error, "Invalid arguments to function 'Prefetch' ".

If I re-code so the script ends with the implied "last," as shown below, I get a 4x speedup (i.e., multi-threading is working), and don't get an error message. In other words, this works. However, it is a PITA to have to re-write scripts to avoid the return statement because I often want to do comparisons between the initial and final states of a denoising script, so it is useful to have a final "output" variable.

Is there a way to get Prefetch to work with Return, or I have I just found the only way?

LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")
source=AVISource("E:\fs.avi").killaudio().ConvertToYV12()
MDegrain2i2(source,8,4,400,0)  
Prefetch(5)

[poisondeathray]

Originally Posted by johnmeyer

Is there a way to get Prefetch to work with Return, or I have I just found the only way?

Don't use "return" , just call the variable directly

eg.

source = whateversource()
A = source().filter1()
B = source().filter2()

#source
A
#B
prefetch(5)

If you wanted "B" , comment out "A" and uncomment out B. If you wanted "source" - same idea, uncomment out source, comment out A and B

[johnmeyer]

Originally Posted by poisondeathray

Don't use "return" , just call the variable directly

eg.

source = whateversource()
A = source().filter1()
B = source().filter2()

#source
A
#B
prefetch(5)

If you wanted "B" , comment out "A" and uncomment out B. If you wanted "source" - same idea, uncomment out source, comment out A and B

I just learned something: I thought I had to use "return." I need to re-read the AVISynth doc. Thanks!

[TheFluff]

return output.prefetch(5)

may or may not work

[LigH]

AviSynth (plus as well as legacy) internally uses a clip variable "last" where an explicit assignment is omitted (and an implicit "return last" where any return was omitted). Explicitly, this would act like:

source = whateversource()
A = source().filter1()
B = source().filter2()

#source
last = A
#last = B
last.prefetch(5)
return last

AviSynth will probably even detect that B is never used, therefore never execute "B = source().filter2()".

[StainlessS]

I nearly suggested same as Fluffy, but as I've never used MT/Avs+, thought it might be daft.

Perhaps docs could be updated to reflect that Prefetch takes a [EDIT: compulsory] clip arg (if indeed it does), docs as given on Avisynth.org/Avs+ below.

Enabling MT

The other difference is how you actually enable multithreading. Calling SetFilterMTMode() is not enough, it sets the MT mode, but the MT mode only has an effect if MT is enabled at all. Note this means you can safely include/import/autoload your SetFilterMTMode() calls in even single-threaded scripts, and they will not be messed up. Uhm, onto the point: You enable MT by placing a single call to Prefetch(X) at the *end* of your script, where X is the number of threads to use.
Example

# This line causes all filters that don't have an MT mode explicitly use mode 2 by default.
# Mode 2 is a relatively safe choice until you don't know most of your calls to be either mode 1 or 3.
# Compared with mode 1, mode 2 trades memory for MT-safety, but only a select few filters will work with mode 1.
SetFilterMTMode("DEFAULT_MT_MODE", 2)
or
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)

# FFVideoSource(), like most of all source filters, needs MT mode 3. 
# Note: starting  with AviSynth+ r2069, it will now automatically recognize source filters.
# If it sees a source filter which has no MT-mode specified at all, it will automatically use 
# mode 3 instead of the default MT mode.
SetFilterMTMode("FFVideoSource", 3)
or 
SetFilterMTMode("FFVideoSource", MT_SERIALIZED)

# Now comes your script as usual
FFVideoSource(...)
Trim(...)
QTGMC(...)
...

# Enable MT!
Prefetch(4)