Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th August 2021, 07:15   #1  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
fmtconv/Avisynth+: resize, bitdepth and colorspace conversions

I just ported my fmtconv Vapoursynth plug-in to Avisynth+.

>>> fmtconv-r30.zip <<<

Fmtconv is a format-conversion plug-in for the Vapoursynth and Avisynth+ video processing engines. It does:
  • Resizing and convolution.
  • Bitdepth conversion with dithering.
  • Colorspace conversion (matrix, transfer characteristics and chromatic adaptation).

Supports:
  • 8-–12-, 14- and 16-bit integer, 32-bit float
  • Colorspaces: RGB, Y, YUV, YCoCg, YDzDx and ICtCp in 4:4:4, 4:2:2, 4:2:0 and 4:1:1 chroma subsampling factors.
  • Progressive and interlaced content.

Fmtconv is focussed primarily on quality and exactness rather than execution speed. This does not mean it is slow or unoptimized, but fmtconv is clearly not on par with the fastest equivalent 8-bit filters.

The full documentation is included in the zip file.

Source code is also available as Git repository.

Please note:
  • This plug-in will work only in Avisynth+ (v3.7.0 and above), not in regular Avisynth, because it makes extensive use of the new high-bitdepth colorspaces. Stack-16 format is not supported.
  • Fmtconv is not internally threaded at the moment (excepted resample that can use AVSTP), but could be if there is a real need. All the functions support the MT_NICE_FILTER multi-treaded mode. Enabling multi-threading is currently not automatic, you have to activate it from your scripts with SetFilterMTMode and use the Prefetch function at the right location. This may change later.
  • 32-bit version is strongly discouraged. I may remove it completely in the future.
  • The whole thing is complex and testing every parameter combination is impossible. The core functionalities work and have been tested for several years with Vapoursynth, but the Avs+ interface is new and may have related issue. Please report here anything suspicious, I’ll try to fix it as soon as possible.
  • Documentation lacks examples for Avs+, but this will come later.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding

Last edited by cretindesalpes; 31st August 2022 at 08:02. Reason: r30
cretindesalpes is offline   Reply With Quote
Old 17th August 2021, 08:22   #2  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
Thanks a lot! Will play with it today. I still have to finish TransformsPack but fmtconv will come handy to double check results.
Finally can use Void and Cluster!
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 17th August 2021, 08:23   #3  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Thank you very much cretindesalpes!
Nice to see you back around
Reel.Deel is offline   Reply With Quote
Old 17th August 2021, 17:22   #4  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
thanks for avs+ porting

Quote:
Originally Posted by cretindesalpes View Post
Fmtconv is not internally threaded at the moment, but could be if there is a real need. All the functions support the MT_NICE_FILTER multi-treaded mode. Enabling multi-threading is currently not automatic, you have to activate it from your scripts with SetFilterMTMode and use the Prefetch function at the right location. This may change later.
isn't fmtconv used to use avstp?
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 17th August 2021, 17:53   #5  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
Quote:
Originally Posted by real.finder View Post
thanks for avs+ porting



isn't fmtconv used to use avstp?
You will replace dither with fmtconv in your scripts???
kedautinh12 is offline   Reply With Quote
Old 17th August 2021, 18:17   #6  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
real.finder:

LOL right, I wrote this too quickly!
Indeed resample can use AVSTP if available.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 17th August 2021, 18:20   #7  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by kedautinh12 View Post
You will replace dither with fmtconv in your scripts???
yes, I plan to do that for at least nnedi3_resize16 now, I will think about the others later
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 17th August 2021, 19:00   #8  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by cretindesalpes View Post
real.finder:

LOL right, I wrote this too quickly!
Indeed resample can use AVSTP if available.
I did try

Code:
LoadPlugin("C:\a\avstp.dll")
LoadPlugin("C:\b\fmtcavs.dll")
ColorBars(width=640, height=480, pixel_type="yv12")
ConvertBits(16).fmtc_resample(Width(),Height()*2).fmtc_resample(Width(),Height()).ConvertBits(8)
I can't see any multi threaded speed with avstp
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 17th August 2021, 19:28   #9  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Nice one.
It's good to see you back.
FranceBB is online now   Reply With Quote
Old 17th August 2021, 20:26   #10  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
real.finder:

1. Make sure that avstp.dll is found by the fmtconv. Check the thread list with Process Explorer or any other task manager.

2. resample splits its workload in tiles, which are dependent on the resizing parameters and input clip. The tiles are processed in different threads. If there is only one tile per frame, there won’t be any gain.

Anyway fmtconv has only minimal support for avstp at the moment, because it was primarily developed for Vapoursynth which handles threading very well natively and it doesn’t need avstp at all. If threading is still a performance issue on Avisynth+, I’ll add a better threading support.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding

Last edited by cretindesalpes; 17th August 2021 at 20:33.
cretindesalpes is offline   Reply With Quote
Old 17th August 2021, 21:14   #11  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
2. resample splits its workload in tiles, which are dependent on the resizing parameters and input clip. The tiles are processed in different threads. If there is only one tile per frame, there won’t be any gain.
yes with fmtc_resample(Width()*2,Height()*2).fmtc_resample(Width(),Height()) I can see the speedup!
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 18th August 2021, 00:06   #12  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
Don't sure about problem of avstp or mvtools2. If you have freetime, can you check it??
https://forum.doom9.org/showthread.p...42#post1942642
kedautinh12 is offline   Reply With Quote
Old 22nd August 2021, 03:15   #13  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Thanks for porting the filter.

It seems i420 is not supported.
Quote:
FFVideoSource() #yuv 8-bit input
fmtc_bitdepth(16) # error
Edit: Converting yuv < 32-bit to yuv 32-bit: _ColorRange is still 1. Shouldn't be 0?

Last edited by StvG; 22nd August 2021 at 03:21.
StvG is offline   Reply With Quote
Old 22nd August 2021, 09:03   #14  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
Quote:
Originally Posted by StvG View Post
It seems i420 is not supported.
Got it. I was naively trying to build a I420 high-bitdepth format which is not supported by Avs+. It is now converted to a regular YUV420Pxx. Here are some fixed binaries that you can try in the meantime of a new release.

Quote:
Converting yuv < 32-bit to yuv 32-bit: _ColorRange is still 1. Shouldn't be 0?
The _ColorRange property has no meaning for floating point formats. So I just keep it as a trace of the original format. Should I erase it? I don’t know. What is the common practice in this case?
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding

Last edited by cretindesalpes; 22nd August 2021 at 09:07.
cretindesalpes is offline   Reply With Quote
Old 22nd August 2021, 15:47   #15  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Quote:
Originally Posted by cretindesalpes View Post
The _ColorRange property has no meaning for floating point formats. So I just keep it as a trace of the original format. Should I erase it? I donÂ’t know. What is the common practice in this case?
There is still no common practice... I think. I was curious because:
1. If you do yuv < 32-bit to 32-bit (default options) you have limited->full range.
2. If you do yuv < 32-bit + fulls=true (mimic internal conversion functions) to 32-bit you have no range range expand.

You have same _ColorRange for both cases.

Edit: Thanks for the new binaries.

Last edited by StvG; 22nd August 2021 at 15:55.
StvG is offline   Reply With Quote
Old 25th August 2021, 19:33   #16  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Can you add top left cplace for fmtc_resample?
StvG is offline   Reply With Quote
Old 25th August 2021, 20:55   #17  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
Yes, this can be accommodated.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 25th August 2021, 23:00   #18  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
I can suggest next additions:
1. Add also sinc interpolation as option to lmpulse kernel setting - as made in UserDefined2Resize in jpsdr plugins. I like an idea of extending filter support as user-provided setting instead of hardcoded value.
2. Add option to weight to zero at the end some end lobes of sinc resample kernel (for example linear-trapezioid as made in SincLin2Resize). So users can use sinc resize with lower taps value without computational issue at the end of kernel when taps lower about 70.
DTL is offline   Reply With Quote
Old 26th August 2021, 09:52   #19  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
Quote:
Originally Posted by DTL View Post
I can suggest next additions:
1. Add also sinc interpolation as option to lmpulse kernel setting
You mean replacing the current linear interpolation with a sinc one? This would make the function unnecessarily complex, especially for a niche functionality. Because sinc is infinite and has to be windowed. But which window? How long? It is expected that the user provides a pre-oversampled kernel (64x is more than enough), the linear interpolation is just here to square the aliasing level (-25 dB goes to -50 dB) at almost no cost, computationally (when generating the final kernel) and spatially.

Quote:
2. Add option to weight to zero at the end some end lobes of sinc resample kernel.
blackman, blackmanminlobe and lanczos are already sinc kernels of adjustable length with a window. But I can add a configurable Tukey window; I’m not sure how useful it would be, but this is not much work anyway.
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 26th August 2021, 10:39   #20  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"replacing the current linear interpolation with a sinc one?"

Adding sinc interpolation option and a user-accessible switch to select between old linear and new sinc.

"Because sinc is infinite and has to be windowed. But which window?"

Simple 'natural-box-window' of filter-support that is not require additional computation or programming.
I.e. filter-support is about (number_of_user_provided kernel samples)*2+1.
Also I see you have filter support as user-definable in the field value and the Impulse may be array of any number of values. It greatly expands functionality of UserDefined2 function based on limited ability of passing only 2 kernel members via equivalent of BicubicResize 2-user arguments instead of unlimited list. I do not know how to pass any required number of user-provided kernel samples as an array so currently we have only fixed by 2 kernel samples UserDefined2 (in the plans are UserDefined4 or more) but if Avisynth API allow to pass any required kernel members in user-provided in the field array it is much better. For example for more precise test data generation we need up to 10 kernel members. They are provided in the book by Krivosheev in about 197x or 198x year (when engineers was high level and think about new digital video systems as very precise and high quality not what awful mix that we got today in 202x).

"It is expected that the user provides a pre-oversampled kernel "

Using sinc-interpolation allow to provide very short number of kernel samples down to 2 and it already works very well. For typical user to provide even 2 kernel samples is not very easy task. One of the main idea from that book about impulse responce of ideal band-limited data - if it is designed as a linear sum of weighted sincs it will be band-limited (close to ideally as sinc itself) with any values of weighting samples. And linear interpolation require user to provide highly oversampled kernel array as I see.

"blackman, blackmanminlobe and lanczos are already sinc kernels of adjustable length with a window. But I can add a configurable Tukey window; I’m not sure how useful it would be, but this is not much work anyway."

The most 'classic DSP/math' windows have special purposes in the other domain - like having special form of Fourier transform. And here we just make bug-fixed sinc resize for the very limited number of taps to get enough computation speed. And as the best bugfix it is better to have self-configured bugfix window without user thinking about it. From my quick tests simple trapezoidal starting from taps/2 works about good and not require additional user input and allow to lower non-visible distortions level taps number from about 70 to about 10 that is good speed increase because with typical good conditioned data we rarely need >10 or >20 taps in sinc kernel to good restore highest possible frequences. And set taps >70 for just fix end of kernel calcultaion issue is wasting of time and resources. Ofcourse it is better to search for better possible window to make sinc resize possible with as low taps as possible without that end of kernel issue _but_ still keeping as much as possible 'full-strike/full-blood' sinc lobes untouched for better restoration of highest valid frequencies with lowest possible kernel taps and so lowest possible filter support and highest possible computing speed.

It may be called 'fixed-sinc' without direct naming of the weighting window because it is programmer-side. The requirements for the window:
1. Work without computational errors at the end of kernel at least with 8bit output integer precision (for >8bit may be more taps or adjustments of window required)
2. Restore with higher possible amplitude the highest valid frequencies with equal number of taps with typical windows like Lanczos. Because we can set Lanczos with taps >50 and have about 10 starting lobes of sinc almost untouched but speed will be less in compare with sinc(taps=10).

Here is current state:
For linear in space and in levels test sequence (16), 27,132,235,132,27,(16)
Test pattern file link https://www.sendspace.com/file/ubhv78 . It is not 2D-ideal elementary point but for exact H and V directions from center it about good conditioned against ringing for sinc-based workflow.
Script
Code:
LoadPlugin("fmtcavs.dll")
LoadPlugin("ResampleMT.dll")

ImageReader("1_point_132_27_lim_lin.bmp")

ConvertToPlanarRGB()

#fmtc_resample(width*10, height*10, kernel="sinc", taps=10)
SincLin2ResizeMT(width*10, height*10, taps=10)

ConvertToRGB24()

Levels(10,1,50,0,255)
Results:




Also the overall field of fmtc output is unclear with looks like +-1 LSB ringing error - may be something wrong with conversion from float to int like +0.5f rounding shift missed or added or compiler error with float to int conversion or other.

Last edited by DTL; 26th August 2021 at 11:38.
DTL is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.