Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 23rd January 2018, 00:02   #3901  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
resample? rescale? I think in general true/false parameters like that should be verbs, not nouns, since it indicates what it will do (or not do).
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline  
Old 23rd January 2018, 07:22   #3902  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
Well I did some tests ...results are not exactly what I was expecting
(I was expecting truerange=true to be left-shifted and truerange=false to be left-padded)

baseline --


8bit in, convert to 10bit, back to 8bit:






...any ideas?

Here's the script; requires Utils-r41.avsi for EvalShow function
Code:
ColorbarsHD(width=480, height=320, pixel_type="YUV444P8")
Subtitle("Source PixelType = '" + PixelType + "'", y=24)

#[[ choose one (or none) - convert bit depth
#EvalShow("ConvertBits(10, truerange=true)")
#EvalShow("ConvertBits(10, truerange=false)")
#]]

Subtitle("Converted PixelType = '" + PixelType + "'", y=48)

ScriptClip("""
Subtitle(
\ "Y min = " + String(YPlaneMin, "%0.2f") + ", max = " + String(YPlaneMax, "%0.2f") + "\n"
\+"U min = " + String(UPlaneMin, "%0.2f") + ", max = " + String(UPlaneMax, "%0.2f") + "\n"
\+"V min = " + String(VPlaneMin, "%0.2f") + ", max = " + String(VPlaneMax, "%0.2f"),
\ lsp=0, y=80)
""")

#[[ choose one (or none) - downconvert for viewing purposes
#EvalShow("ConvertBits(8, truerange=true)", align=1)
#EvalShow("ConvertBits(8, truerange=false)", align=1)
#]]
raffriff42 is offline  
Old 23rd January 2018, 08:06   #3903  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
@rr42 - Maybe that's only for "10bit in 16bit" ? and might not apply to "8bit in 10bit" ?

I'm just trying to think of what usage scenarios would you NOT want to scale the values for 8 to 10bit? I can't think of any

The docs (which obviously could be wrong at this point) say "convert 10-16 bit formats without re-scaling underlying pixel data." I think that means source 10 to 16 . The example given is ConvertBits(16, truerange=false)

But for the 10bit case, "10bit in 16bit code words" is still used by some people using dither tools workflows
poisondeathray is offline  
Old 23rd January 2018, 10:39   #3904  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
Quote:
The docs (which obviously could be wrong at this point) say "convert 10-16 bit formats without re-scaling underlying pixel data." I think that means source 10 to 16 .
I think it means "convert to/from formats with between 10 and 16 bits without rescaling."

When converting up, should there be an option to rescale and copy in the upper bits to the new lower bits? So that full white remains full white? E.g, from 8 to 12 bits:

Code:
11111111 => 111111111111
11010011 => 110100111101
01101111 => 011011110110
This can be shifted back down and still return to the original value.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline  
Old 23rd January 2018, 16:01   #3905  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
poisondeathray,
>The docs (which obviously could be wrong at this point)

"The docs" were typed in by me, using whatever information I could gather at the time.

Note the big red TODO which is my way of saying I don't have a frickin clue here.

If you scroll down to scale, fulls and fulld you will see more confusion. Fact-finding assistance is welcome.

Your mention of 10bit in 16bit data sounds plausible.
raffriff42 is offline  
Old 23rd January 2018, 17:13   #3906  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
This parameter, whatever stupid name it has, was introduced at the very beginning of the high bit depth project. Its only purpose was to 'typecast' a 16 bit video format over an existing 10 bit content.
Probably in those times native 10 bit support by plugins or external applications was practically non-existing, some external programs could handle 16 bits or one could use stacked 16 bit format.
I guess these were the reasons.
pinterf is offline  
Old 25th January 2018, 15:27   #3907  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
pinterf, it's okay to take a wrong turn or two in such an undertaking. Overall, AVS+ is a huge step forward, and thanks again for the work you've done.

Maybe truerange, scale, fulls and fulld can be left undocumented - labeled "deprecated/experimental" (or something) for the time being?
raffriff42 is offline  
Old 26th January 2018, 07:10   #3908  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Yes, truerange and scale should surely die silently.
pinterf is offline  
Old 26th January 2018, 18:21   #3909  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Meanwhile I accidentally observed a bug, which appeared on the right-side of a clip as random color blocks/lines.
It was appearing only in 32bit float test clips and only at specific sequence of resizing.

Check and encode this script and look at the right side of the bottom right clip (32 bit float)
Code:
x=ColorBarsHD.ConvertToYUV444().Trim(0,100)
Function Resize(clip c)
{
Return c.Spline64Resize(2802,1501).\
  Spline16Resize(904,487).\
  BilinearResize(402,500).\
  Spline36Resize(200,489).\
  LanczosResize(400,300).ConvertBits(8)
}
a8 = x.ConvertBits(8).Resize()
a10 = x.ConvertBits(10).Resize()
a16 = x.ConvertBits(16).Resize()
a32 = x.ConvertBits(32).Resize()
Stackvertical(StackHorizontal(a8,a10),StackHorizontal(a16,a32))
Unfortunately it was perfect when I encoded float-only clips. The uglyness in the reproduction was that there had to be a 8-16 bit clip in the script (I usually do the tests for 8-10-16-32 bits then stack it together in 8 bits to see any difference) Obviously there had to be something that had left other patterns in memory than a 32bit float format clip.

Narrowing the problem down, it turned out that the SIMD code that handles the pixels in 4/8 units was run into some garbage at the right side, where not all the 4/8 units are visible pixels, depending of the clip width (modulo 4 or 8).

During the resizing process, the unused pixels are masked out with a zero multiplier - existing 8-16 bit code worked fine like that -, but it was not enough for 32 bit float pixels. When such a pixel is undefined, the processor would report it NaN (Not a Number), and multiplying it by 0 would still result in NaN.
Thus such pixel in the resized clip turned into undefined (garbage)

So I had to fix the float resizer code - uhh, it was old, one of my early attempts.
Since the 10-16 bit resizer parts were affected as well, they had to be touched, too.
Finally whe whole 10-16 and float resizer code got rewritten - unfortunately I couldn't see the time that the bug chasing needed, sure, with less effort I could port Zimg resizers into avs+.

Btw zimg.
Since I had to benchmark the new code if it is any better than the one in r2580, I have included the z_XXX resizers.
https://forum.doom9.org/showthread.php?t=173986

Results are interesting.
Look at the 400x2800 -> 900x400 case (16 bit)

Resizing always happens in one horizontal and one vertical pass. Or first vertical resizing, then horizontal. it depends.

In Avisynth - probably for quality reasons - there is a strategy: "// ensure that the intermediate area is maximal"

Avs resizer gave a 103 fps, while zimg had 402 fps. What?

Then it was made clear that Avs chose the 400x2800->900x2800->900x400 sequence,
while zimg chose
400x2800->400x400->900x400.

When I turned the resizing command into two resizing (first V then H), avisynth+ gave a quite comparable result of 427 fps.
First three columns (AVX2, SSE4, noSSE4 contains results of the new resizers) Code was run on an i7-7700, Avs+ x64. I built specific avs+ versions for the test to ignore AVX2..SSE4.1 CPU flags.

EDIT: this benchmark data contains the comparison of a current "under construction" version avs+ and the z-lib resizer I had access (r1a, from 2016).
They both have faster variants since then.

Code:
#32bit float           AVX2    SSE4   noSSE4   v2580  Zimg
#400x2800 -> 900x400:  103     64.3   64.3     65.9              Lanczos
#1920x1080-> 1280x720  143.1   83.7   83.2     95.7   158.4      Spline64

#16 bit                AVX2    SSE4   noSSE4   v2580  Zimg
#400x2800 -> 900x400:  103.9    84     75.8    56.2   402.6      Lanczos **see comment
#400x2800 -> 400x400->
#            900x400:  427.2   315.3  289.9    243.4  405.1      Lanczos 

#1920x1080-> 1280x720  152.2   125.0  118.9    80.6   129        Spline64
#1920x1080-> 1280x720  160.6   134.3  129.4    85.9   134.5      Lanczos
#1920x1080-> 1280x1080 240.2   190.6  185.2    113.2  191.8      Lanczos H
#1920x1080-> 1920x720  335.5   320.8  281      272.2  203.0      Lanczos V

#10 bit                AVX2    SSE4   noSSE4   v2580  Zimg
#400x2800 -> 900x400:  105.4   77.5    53.5                      Lanczos 
#1920x1080-> 1280x720  155.2   146.6  120.4    78.9   133        Spline64
#1920x1080-> 1280x720  163.6   156.2                             Lanczos

#8                     AVX2    SSE4   noSSE4   Old    Zimg
#400x2800 -> 900x400:  93.8    93.4   96.2     93.8   402       **see comment
#1920x1080-> 1280x720  104.7   105.1  106.0    105.5  120.2

# ** Avisynth - unlike zimg - always orders H/V resizers for max intermediate area!
(8 bit resizer code is untouched by me, there is no avx2 option there but I included their measurements)

Now it's to be decided that the slower H/V or V/H decision strategy should be kept or not. Is the difference really visible and when?

Last edited by pinterf; 28th January 2018 at 19:57. Reason: Comment on benchmarks, new resizers exist since then
pinterf is offline  
Old 26th January 2018, 19:11   #3910  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Were those tests based on the old zimg , or a recompiled one ? There are quite a few commits to z.lib since that avisynth version posted in Nov 2016, but I don't know offhand if any changes were made to performance (except I think he added AVX-512) or resizing strategies

If there was a noticable qualitative difference, I would imagine it should be more noticable on upscaling. But maybe with some extreme cases , where you have a few pixel width or height src e.g. a 3840x4 strip. There might be some more noticable differences. I guess we can do some tests

But end user can still separate the W, V steps manually if they needed to...



BTW, how are can you load native float formats into avs+ ? (e.g EXR) ? vapoursynth got a bunch of imagemagick updates, but I don't see equivalent method in avs+ ? ffmpeg related options load at 16bit int
poisondeathray is offline  
Old 27th January 2018, 10:30   #3911  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
@Stephen R. Savage
Have you, like pinterf, turn the resizing command into two resizing (first V then H) to ensure that AVS+ and z.Lib have the same "resize path", to make the benchmark realy accurate ?

@Pinterf
Does it mean there is a new code for resizer, so i have to update my ResizeMT...?
jpsdr is offline  
Old 27th January 2018, 10:35   #3912  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Quote:
Originally Posted by pinterf View Post
Now it's to be decided that the slower H/V or V/H decision strategy should be kept or not. Is the difference really visible and when?
You can still add a new parameter at the end, this will keep compatibility with old script, with something like this :
0 : (default) current behavior
1 : Fastest (so will probably result in opposite of current behavior)
2 : Vertical first
3 : Horizontal first
jpsdr is offline  
Old 27th January 2018, 12:29   #3913  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Anyway I will rebuild zimg and do the comparison again, at least I have a look at the code and get familiar with a possible integration.
pinterf is offline  
Old 27th January 2018, 13:10   #3914  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
Quote:
Originally Posted by pinterf View Post
Now it's to be decided that the slower h/v or v/h decision strategy should be kept or not.
Is the difference really visible and when?
Huh! I thought AVS's rationale made sense, but now that I think about it,
how is the visible result of the AVS sequence
400x2800->900x2800 // scale rows, copy columns
900x2800->900x400 // copy rows, scale columns

any different from zimg's
400x2800->400x400 // copy rows, scale columns
400x400->900x400 // scale rows, copy columns

Unless the two resize operations are not actually separable?
Does scale row n ever use pixels from the row(s) above or below? I assumed not.

(dumb question?)
raffriff42 is offline  
Old 27th January 2018, 17:56   #3915  |  Link
VS_Fan
Registered User
 
Join Date: Jan 2016
Posts: 98
Quote:
Originally Posted by pinterf View Post
... In Avisynth - probably for quality reasons - there is a strategy: "// ensure that the intermediate area is maximal" ...

Now it's to be decided that the slower H/V or V/H decision strategy should be kept or not. Is the difference really visible and when?
Quote:
Originally Posted by raffriff42 View Post
... how is the visible result of the AVS sequence ... any different from zimg's ...
One can imagine the larger intermediate resolution would keep more original information for the second stage filtering: The more resolution/information as input would yield a better output from any filter.

An “objective” test could be done to measure the quality of the results of either strategy: Produce the results of both strategies: scale the original to whatever size (up/down), then scale them back to the original size, and finally measure the quality of both strategies, with say SSIM, against the original
VS_Fan is offline  
Old 27th January 2018, 18:03   #3916  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by raffriff42 View Post
Huh! I thought AVS's rationale made sense, but now that I think about it,
how is the visible result of the AVS sequence
400x2800->900x2800 // scale rows, copy columns
900x2800->900x400 // copy rows, scale columns

any different from zimg's
400x2800->400x400 // copy rows, scale columns
400x400->900x400 // scale rows, copy columns

Unless the two resize operations are not actually separable?
Does scale row n ever use pixels from the row(s) above or below? I assumed not.

(dumb question?)




It would depend on the scaling algorithm. For example, bilinear looks at a 2x2 grid, bicubic 4x4 grid. Bicubic should be more adversely affected if you scale the lower dimension first

Computing width first vs. height or, vice versa are not bitexact, identical operations if image dimensions are not identical to begin with (square) . You can detect differences or visualize with amplified differences in the end result on any test. The question is whether or not they are significant, or under what conditions do they become significant

You can see issues on various test patterns .

(apologies, this was done in vapoursynth with zimg/z.lib, but the example should still apply to resizing order)

eg.
Test source 10bit RGB 2048x960 dpx patttern 0-1023 gradients for R,G,B,greyscale used the other vapoursynth thread .
http://www.mediafire.com/file/71x5rz..._10bit_grad.7z

Convert/resize to YUV420P8, resize (width/2 then height/2) , or (height/2 then width/2)

Code:
v = core.imwri.Read(r'F:\_Video Tests\10bit Pattern Tests\RGB_10bit_grad.dpx')

v1 = core.resize.Bicubic(v, width=1024 , height=960, format=vs.YUV420P8, matrix_s="709", range_s="full") 
v1 = core.resize.Bicubic(v1, width=1024 , height=480, format=vs.YUV420P8, matrix_s="709", range_s="full") 

v2 = core.resize.Bicubic(v, width=2048 , height=480, format=vs.YUV420P8, matrix_s="709", range_s="full") 
v2 = core.resize.Bicubic(v2, width=1024 , height=480, format=vs.YUV420P8, matrix_s="709", range_s="full") 

i = core.std.Interleave(clips=[v1,v2])
i = core.hist.Luma(i)
i.set_output()
You can use MakeDiff to see the diff but I chose histogram.luma for this exagerrated look, but you can see the issue on the native image as well if you zoom in and have good eyes. You can see the alignment is "off" when resizing height first . The height first strategy will predispose you to more "banding" in post production, encoding later etc...

Quality wise, it makes more sense to me to resize the dimension with more pixels first - better for sampling


This is a 1:1 center crop , nearest neighbor x2 to show the issue

Last edited by poisondeathray; 27th January 2018 at 18:06.
poisondeathray is offline  
Old 27th January 2018, 20:11   #3917  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Stephen R. Savage View Post
Did you consider that the "issue" goes away if you just do it normally?

Code:
core.resize.Bicubic(c, width=1024, height=480, format=vs.YUV420P8, matrix_s="709", range_s="full")
But the problem is... it does not go away

Code:
v3 = core.resize.Bicubic(v, width=1024 , height=480, format=vs.YUV420P8, matrix_s="709", range_s="full")

i = core.std.Interleave(clips=[v1,v2,v3])

Here are the full images after the histogram luma

In case it's not clear look at the intersections between the rows
http://www.mediafire.com/file/ehrjjj...calingtests.7z


again, a crop , 2x point resize

stackvertical (widthfirst, heightfirst, both)

Last edited by poisondeathray; 27th January 2018 at 20:19.
poisondeathray is offline  
Old 27th January 2018, 20:53   #3918  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Stephen R. Savage View Post
I am sure if you rotated the image, you would see the opposite "finding."
I used transpose, and it's not quite the "opposite" finding

But there are difference between all 3 - why wouldn't v2 and v3 be equivalent ?

For now, let's leave dithering out for a separate discussion. Just evaluate the order of operations on end result

Quote:
By the way, 1024x960 and 2048x480 are exactly the same "area."
Yes, good point


Quote:
See how there is no "problem" in "c"? All your observations are the artificial result of rounding.
Your "c" is the same as my "v3" . It's not the same. I even tried ffms2

So you're saying everything here is from rounding ?
poisondeathray is offline  
Old 27th January 2018, 22:02   #3919  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Stephen R. Savage View Post
It does not matter if it is the same.
I'm saying horizontal first vs. vertical first is not the same. It's just an observation

You said "See how there is no "problem" in "c"? "

In this example, there is a "problem", or at least a difference. In this specific case, the bars at the intersections don't even line up when you resize height first, or if you do it "normally" . I would expect "doing it normally" in z.lib would be = resize height first, but that isn't the case here either.

ie. there is a difference between resizing horizontal first, vs vertical resizing. They do not produce bit identical results. ie. They are not the same. In this specific case, it happens that resizing horizontal first is clearly better . That might not be true for other cases. But clearly there is a difference.

Quote:
If you use "core.register_format(vs.YUV, vs.FLOAT, 32, 1, 1)" as the intermediate, everything is bit-exact. But it doesn't matter to begin with.
No difference here. Maybe you're using a different version ?

EDIT: or did you mean just using float as the intermediate e.g. YUV444PS - in that case yes, all 3 cases are identical . Then your point about rounding errors makes more sense

Last edited by poisondeathray; 27th January 2018 at 22:10.
poisondeathray is offline  
Old 27th January 2018, 22:13   #3920  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Stephen R. Savage View Post
How can you "leave out" dithering when discussing rounding errors?
Because it's possible dithering can obscure underlying issues

Quote:
You observe discrepancies in ordering because your image is constant in one direction. After you rotate the image, vertical-first becomes the "better" direction...
resize_h->round->resize_v->round
resize_v->round->resize_h->round
resize_v->resize_h->round

These are not the same operations... You are observing a discrepancy because resizing a constant image produces a constant result. It is not the resizing but the rounding that "moves" the bars in the first pass. The reason they do not move again in the second pass is because the image has been quantized to 8 bits already.
Ok this makes sense to me now, thanks


As for the rotate/transpose test:

vertical first was slightly better in some of the patterns, but the intersections were still bad in all 3.

The premise was resizing the "larger" dimension first would be better because of more information (although as you pointed out same area) so you would expect it to be "better" to resize vertical first in the rotated version, just like you would "expect" it to be "better" the horizontal in the 1st case . But this was just 1 test case, i think it' s just happenstance here and as you say a result of the quantizing/rounding errors

Last edited by poisondeathray; 27th January 2018 at 22:19.
poisondeathray is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.