Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th April 2020, 11:25   #41  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
As in above EDIT:, I forced a refresh for it to appear in Update Manager.
Thanks for the info.
EDIT: Maybe I have it set to install unstable packages or something [I remember I was looking for something that was unavailable as standard].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 19th April 2020, 12:32   #42  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
Quote:
Originally Posted by MeteorRain View Post
SssS: Yes that's the 20200405 snapshot of GCC 10 that's still in development. I don't know why a distribution would put a software still in development on users' computer.

Clang 10 however, has just released. But that doesn't support all the features I believe.
I might be able to make a GCC9.3 compatible version if anyone needs it. I would have to rewrite a bunch of concepts and constraints stuff with SFINAE tho.
feisty2 is offline   Reply With Quote
Old 19th April 2020, 17:21   #43  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@Are_
can you run a speed test for the latest gauss blur again?
I have eliminated some frequent memory reallocations which could potentially be the performance bottleneck in earlier versions
feisty2 is offline   Reply With Quote
Old 19th April 2020, 19:39   #44  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
Code:
(vsFilterScript)
Output 100000 frames in 28.35 seconds (3527.91 fps)
(c_filter)
Output 100000 frames in 9.06 seconds (11036.62 fps)
(Convolution)
Output 100000 frames in 9.05 seconds (11045.38 fps)
This is not the same computer but it's quite similar in specs.
Are_ is offline   Reply With Quote
Old 19th April 2020, 20:24   #45  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
so it wasn't reallocation either...
I guess it can only be the cost of small objects then (Plane::Proxy for each pixel)
feisty2 is offline   Reply With Quote
Old 20th April 2020, 00:56   #46  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by MeteorRain View Post
SssS: Yes that's the 20200405 snapshot of GCC 10 that's still in development. I don't know why a distribution would put a software still in development on users' computer.
Judging by some of the responses here:
https://forums.linuxmint.com/viewtopic.php?t=317303

I'm pretty sure it was due to the ubuntu-toolchain-r PPA, not the distro.
qyot27 is offline   Reply With Quote
Old 20th April 2020, 03:30   #47  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by qyot27 View Post
I'm pretty sure it was due to the ubuntu-toolchain-r PPA, not the distro.
Yep mine was same problem, thanx qyot.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 20th April 2020, 06:44   #48  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
I have pinpointed the performance bottleneck, it is the indirection (Plane::Proxy and Plane::Offset)
and it shouldn't be a problem unless the filter is ultra fast, like 3x3 gauss blur. it is a constant cost for a given format and should be negligible for slow filters like NLMeans.
feisty2 is offline   Reply With Quote
Old 20th April 2020, 20:42   #49  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
revamped: Preprocess() is now replaced by RegisterInvokingSequence(), you may invoke a sequence of filters (including the very filter that is being defined, you may invoke this "self" filter via SelfInvoker) any way you like. basically, you get to write something like a vpy script in RegisterInvokingSequence and if this function is absent in your filter, it is assumed that the output is SelfInvoker()

say you defined a 3x3 blurring filter in DrawFrame and you wanna repeat it 10 times:
Code:
auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
	for (auto _ : Range{ 10 })
		InputClip = SelfInvoker("clip", InputClip);
	Console.Receive(InputClip);
}
you wanna upsize it by a factor of 2 after blurring.
Code:
auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
	for (auto _ : Range{ 10 })
		InputClip = SelfInvoker("clip", InputClip);
	InputClip = Core["fmtc"]["resample"]("clip", InputClip, "width", InputClip.Width * 2, "height", InputClip.Height * 2);
	Console.Receive(InputClip);
}
and you wanna transpose the clip before all this happens
Code:
auto RegisterInvokingSequence(auto Core, auto&& SelfInvoker, auto Console) {
	InputClip = Core["std"]["Transpose"]("clip", InputClip);
	for (auto _ : Range{ 10 })
		InputClip = SelfInvoker("clip", InputClip);
	InputClip = Core["fmtc"]["resample"]("clip", InputClip, "width", InputClip.Width * 2, "height", InputClip.Height * 2);
	Console.Receive(InputClip);
}

Last edited by feisty2; 21st April 2020 at 16:14.
feisty2 is offline   Reply With Quote
Old 21st April 2020, 16:14   #50  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
revamped: you can now call SelfInvoker() with an argument list, the calling syntax is unified with external filters. However unlike external filters, SelfInvoker does not have any error handling capacity. It is the filter developer's responsibility to ensure that only valid arguments are passed to SelfInvoker since it represents the "self" filter and it's private to the developer. The behavior is undefined (you will most likely encounter a segfault) if you call SelfInvoker with invalid arguments. all above examples have been updated to reflect the new calling syntax.
feisty2 is offline   Reply With Quote
Old 29th April 2020, 15:19   #51  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
Transpose is now replaced by the more interesting nnedi3_rpow2 example. center shift correction and other features that require resampling are however not implemented for the sake of simplicity.
feisty2 is offline   Reply With Quote
Old 15th May 2020, 10:19   #52  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
removed hasattr, it seems to fail on some corner cases.
feisty2 is offline   Reply With Quote
Old 2nd June 2020, 22:15   #53  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
new functionality: calling python functions, the syntax is the same as invoking filters (MATLAB function call syntax)
new functionality: Frame and Function can now be map items, all VSPropTypes are now supported.
new functionality: PeekFrameFormat() for map items, if you need to convert a map item to a Frame and the PixelType of the Frame cannot yet be determined, use this function to peek the format before type conversion.

a new ModifyFrame example is added to demonstrate the use of Function, a second order example for ModifyFrame is also provided.
feisty2 is offline   Reply With Quote
Old 26th July 2020, 13:36   #54  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
new functionality: support for filters that return multiple clips, this also affects SelfInvoker. SelfInvoker now returns an std::vector<Clip> if the filter returns multiple clips, otherwise, it still returns a Clip. Console.Receive() now accepts a Clip or a sequence of clips stored in any iterable container.
issue: support for calling external filters that return multiple clips is not yet implemented, this breaks the static type system of C++ since it is only known at runtime if an external filter returns a Clip or an std::vector<Clip>, I haven't figured out an elegant approach to deal with this issue.

new example: Palette, this is a simple source filter showing you how to write filters that return multiple clips. it accepts an array of N numbers, and returns N blank clips, each filled with a shade of gray that corresponds to a number in the array.
Code:
x, y, z = core.test.Palette([0, 0.5, 1])
# x is filled with black, y is filled with gray, and z is filled with white

Last edited by feisty2; 26th July 2020 at 13:39.
feisty2 is offline   Reply With Quote
Old 26th July 2020, 21:29   #55  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
can someone do a quick test and see if this thing compiles with the latest version of clang?
msvc support will soon be possible with the implementation of abbreviated templates in msvc 16.8
feisty2 is offline   Reply With Quote
Old 27th July 2020, 23:10   #56  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
new functionality: support for calling external filters with multiple outputs.
support for filters that return multiple clips is now complete, to receive every clip returned by external filters with multiple outputs, you have to call ConfigureForMultipleOutputs(), otherwise it is assumed that the filter only has one output and only the first returned clip will be fetched.

Code:
auto ffms2 = Core["ffms2"]["Source"].ConfigureForMultipleOutputs();
auto composite = ffms2("source", "something_with_alpha_channel.mov");
auto& content_clip = composite[0];
auto& alpha_channel = composite[1];
is equivalent to, in Python

Code:
content_clip, alpha_channel = core.ffms2.Source("something_with_alpha_channel.mov")

Last edited by feisty2; 28th July 2020 at 00:34.
feisty2 is offline   Reply With Quote
Old 29th July 2020, 05:15   #57  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
revamped: various syntactic sugar for stuff dealing with multiple outputs, this affects both SelfInvoker() and the calling syntax of external filters.

for SelfInvoker():
a) if RegisterVideoInfo() returns a VSVideoInfo scalar, it is assumed that the filter has only one output, and SelfInvoker() returns a Clip.
b) if RegisterVideoInfo() returns a VSVideoInfo sequence stored in a container of dynamic length (something like std::vector), it is assumed that the filter has multiple outputs, and the number of outputs could not be determined at compile time, in this case SelfInvoler() returns an std::vector<Clip>.
c) if RegisterVideoInfo() returns a VSVideoInfo sequence stored in a container of constexpr length N (something like std::array), it is assumed that the filter has N outputs and SelfInvoker() returns an std::array<Clip, N>, in this case you can unpack the returned clip array using a structural binding:
Code:
auto [clip_1, clip_2, ..., clip_n] = SelfInvoker(...);
d) if RegisterVideoInfo() is not defined, it is obvious that SelfInvoker() will never be called and thus it doesn't matter what it may return.

for external filters:
a) if the filter is obtained by Core["namespace"]["filter"], it is assumed that the filter has only one output, and filter() returns a Clip.
b) if the filter is obtained by Core["namespace"]["filter"].ConfigureForMultipleOutputs(), it is assumed that the filter has multiple outputs, and the number of outputs could not be determined at compile time, in this case filter() returns an std::vector<Clip>.
c) if the filter is obtained by Core["namespace"]["filter"].ConfigureForMultipleOutputs(N), where N is a compile time constant, it is assumed that the filter has N outputs and filter() returns an std::array<Clip, N>, in this case you can unpack the returned clip array using a structural binding:
Code:
auto [clip_1, clip_2, ..., clip_n] = filter(...);
it is safe to specify an N that is not equal to the actual number of returned clips M. if N > M, the surplus items left in the returned array would simply be empty Clips. if N < M, only the first N clips will be fetched.



the ffms2 example above could now be rewritten in a more pythonic manner:
Code:
auto ffms2 = Core["ffms2"]["Source"].ConfigureForMultipleOutputs(2);
auto [content_clip, alpha_channel] = ffms2("source", "something_with_alpha_channel.mov");
feisty2 is offline   Reply With Quote
Old 31st July 2020, 17:59   #58  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@Are_
could you compile the latest version and run a speed test for core.test.GaussBlurFast? I added a new direct access mode for maximum performance.

Last edited by feisty2; 31st July 2020 at 18:02.
feisty2 is offline   Reply With Quote
Old 31st July 2020, 19:09   #59  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
This looks really good:
Code:
Convolution:
Output 100000 frames in 9.29 seconds (10769.29 fps)

GaussBlur:
Output 100000 frames in 27.54 seconds (3631.56 fps)

GaussBlurFast:
Output 100000 frames in 9.16 seconds (10917.06 fps)
Are_ is offline   Reply With Quote
Old 1st August 2020, 07:39   #60  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
update:
revamped: you may now also directly pass arbitrary number of clips to Console.Receive() as its arguments, no need to manually wrap them in a container.
removed: Buffer, it's not really that relevent and I might add it back later with various new utility stuff.
new functionality: direct frame access. when you're done sketching the prototype of your filter and ready to optimize it for speed, you should enable direct access mode by specifying the DirectAccess template parameter whenever your filter reads a frame. this exposes raw pointer access to every row of every plane of a frame. out-of-bound access detection, automatic padding and the ability to create views will all be disabled and you have to manually take care of the corner cases. with direct access mode on, vsFilterScript provides only zero cost abstraction and the execution speed of your filter will be no different from filters written directly with the low level C API.

A GaussBlurFast example is provided to demonstrate how to write filters with direct frame access, you can compare it to the earlier GaussBlur example which utilizes indirect access features like automatic padding and views for cleaner and more elegant code at the cost of slower execution speed.
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.