Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st August 2019, 12:32   #4821  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 638
Quote:
Originally Posted by magiblot View Post
What is the range of expected pixel values for floating-point formats? [0..1], [0..255/256], or the former for RGB and the latter for YUV?
8bit:
Full Range 0-255 / Limited TV Range: 16-235

10bit:
Full Range 0-1023 / Limited TV Range: 64-940

12bit:
Full Range 0-4080 / Limited TV Range: 256-3760

14bit:
Full Range 0-16320 / Limited TV Range: 1024-15040

16bit:
Full Range 0-65280 / Limited TV Range: 4096-60160

32bit float:
Full Range 0.0-0.99609375 / Limited TV Range: 0.0625-0.91796875


This simulates clipping a 32bit signal:

Code:
ColorBars(848, 480, pixel_type="YV12")

ConvertBits(32, truerange=true)

Limiter(min_luma=0.0625, max_luma=0.91796875)



Wrongly clipping 8bit values from a 32bit float clip:

Code:
ColorBars(848, 480, pixel_type="YV12")

ConvertBits(32, truerange=true)

Limiter(min_luma=16, max_luma=235)



I personally work with 16bit precision which I think it's enough when my output is gonna be dithered down to 8bit for SD, HD and FULL HD footages, 10bit for UHD broadcast footages and 12bit for UHD Masterfiles.
So far I haven't used 32bit precision in a real world scenario, but still, it's kinda cool to have such an high bit depth.
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 22nd August 2019 at 22:30.
FranceBB is offline   Reply With Quote
Old 21st August 2019, 13:52   #4822  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,748
Quote:
Originally Posted by FranceBB View Post
12bit:
Full Range 0-4080
14bit:
Full Range 0-16320
16bit:
Full Range 0-65280
32bit float:
Full Range 0.0-0.99609375
What...

Is this only YUV, or is it also RGB (mostly for 16bits for the last one). I didn't expect remaining gap between theorical max for full range.
__________________
My github.

Last edited by jpsdr; 21st August 2019 at 13:56.
jpsdr is offline   Reply With Quote
Old 21st August 2019, 14:51   #4823  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 1,806
Quote:
Originally Posted by jpsdr View Post
What...

Is this only YUV, or is it also RGB (mostly for 16bits for the last one). I didn't expect remaining gap between theorical max for full range.
I assumt it's because it's done as bit-shifts. 255<<4 = 4080, 255<<6 = 16320, 255<<16 = 65280.

10-bit wasn't on the list but I assume it's 0-1020.

It's possibly a way to make all up-conversions consistent/transitive (and fast), but I wonder if down-conversions are more complicated. By a simple bit-shift, only one value in 16-bit would map to 255 in 8-bit (62580), whereas 256 values (0-255) would map to 0.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 21st August 2019, 15:16   #4824  |  Link
magiblot
Eurobeat Fan
 
Join Date: Sep 2014
Posts: 97
Quote:
Originally Posted by FranceBB View Post
32bit float:
Full Range 0.0-0.99609375 / Limited TV Range: 0.0625-0.91796875
This is what theory says, as explained in http://avisynth.nl/index.php/Autoscale_parameter, http://avisynth.nl/index.php/Convert and other places:

Quote:
* The special-purpose matrices PC.601 and PC.709 keep the range unchanged, instead of converting between 0d-255d RGB and 16d-235d YUV, as is the normal practice.
Which implies that these ranges apply to RGB as well. And this makes sense to me as using multiples of 1/256 should help avoid rounding errors.

However, the question is: do filters actually expect pixel values to be in that range?

ConvertBits, for instance, seems not to:

Quote:
Conversion from and to float is always full-scale.
Practical examples show different behaviours:

Code:
BlankClip(height=240).ConvertToPlanarRGB()
Expr(String(255.0/256.0), format="RGBPS") # 255d
StackVertical( Expr("x 256 *", format="RGBP8").ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("'D notation'-compliant conversion"),
             \ ConvertToRGB32().RGBAdjust(analyze=true).Subtitle("Default conversion") )


Code:
BlankClip(pixel_type="YV24")
Expr("0.0", format="YUV444PS")
ColorYUV(levels="PC->TV") # Bug in ColorYUV? Picture turns green
ColorYUV(analyze=true) # OK: luma is 0.6250 == 16/256

Expr("1.0", format="YUV444PS")
ColorYUV(levels="PC->TV")
ColorYUV(analyze=true) # Confusing: luma is 0.91796875 == 235/256, but input expected to be 1.0 rather than 255/256

Expr(String(235.0/256.0), format="YUV444PS")
ColorYUV(levels="TV->PC")
ColorYUV(analyze=true) # Same as before: 235/256 translates into 1.0 instead of 255/256

BlankClip(color_yuv=$108080, pixel_type="YV24") # Plain black in limited range
ConvertBits(32)
ColorYUV(analyze=true) # Inconsistent: 16d represented as 16/255 == 0.062745... (periodic number)
From ColorYUV's behaviour, one could assume that [16d..235d] is used for limited range and [0..1] for full range. But, by looking at ConvertBits, one would deduce that [16/255..235/255] and [0..1] are to be used instead. Documentation, conversely, speaks of [16d..235d] and [0d..255d].

In addition, RGBAdjust and Compare are unable to analyze floating-point pixel types, which makes it more difficult to see what's going on.

Last edited by magiblot; 21st August 2019 at 15:57.
magiblot is offline   Reply With Quote
Old 21st August 2019, 16:07   #4825  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,748
Quote:
Originally Posted by wonkey_monkey View Post
10-bit wasn't on the list but I assume it's 0-1020.
10-bit wasn't in my list, but was on original post with, for this one, expected value of 1023.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 21st August 2019, 16:31   #4826  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,032
My natural inclination when using up shift would be to fill the 'nullified' least signifcant bits in destination number with the most significant bits from source number.
Code:
eg 8->10 bit,
     7654321076   src bit indexes
     9876543210   dst bit indexes

or eg 8->16
    7654321076543210
    fedcba9876543210    # hexified
But, that aint right, apparently.

EDIT: I think Knuth, TAOCP, Vol 1, Fundamental Algorithms, would suggest same for fast accurate upscale using shift only. [I left my copy on a train many years ago].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 21st August 2019 at 17:04.
StainlessS is offline   Reply With Quote
Old 21st August 2019, 17:39   #4827  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,874
Why is this horizontal line shifted ? (You can see it if you return a vs. b)

YV24 (and using ConvertToYUV444) works as expected, so is it some chroma resizing issue in 32bit? I tried different chromainplacement/chromaoutplacement settings as well

r2772 MT x64 ; or is it already fixed in one of the beta releases ?

Code:
a=colorbars(pixel_type="YV12").trim(0,-1)

a
ConvertBits(32)
ConvertToPlanarRGB(matrix="rec601", chromaresample="point")
ConvertToYUV420(matrix="rec601", chromaresample="point")
ConvertBits(8)
b=last

interleave(a,b)
poisondeathray is offline   Reply With Quote
Old 21st August 2019, 17:49   #4828  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,032
Dont know but,
Code:
a=colorbars(pixel_type="YV12")

a
ConvertBits(32)
ConvertToPlanarRGB(matrix="rec601", chromaresample="point")
ConvertToYUV420(matrix="rec601", chromaresample="point")
ConvertBits(8)
b=last

#a
#b
b.subtract(a).stackHorizontal(a)  # EDITED

ver$ r2900

EDIT: changed order from a.subtract(b) to b.subract(a) and reposted image.
So, eg where subtracted is yellow, then output is more yellow than input.
EDIT: Added in blue, b.subtract(a).stackHorizontal(a)

Dont know if as expected or not.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 21st August 2019 at 18:29.
StainlessS is offline   Reply With Quote
Old 21st August 2019, 19:15   #4829  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,874
Quote:
Originally Posted by StainlessS View Post

Dont know if as expected or not.



It's supposed to be lossless ; it is with vapoursynth

32bit float "expected" range is 0-1 , but negative values , and values than 1 should are kept. They are in other programs too. That's one of the main reasons for using float. It's commonly used with raw processing, and high end effects/compositing

You can read off the values in vapoursynth editor for different bit depths , and for float (<0 , >1 ), but I cannot "read" values in avspmod besides 8bit, I don't know what avs+ is doing

I think this issue has something to do with chroma resampling, because if you do colorbars(pixel_type="YV24"), and ConvertToYUV444 after the RGB step, it works as expected (equivalent)

If you do 8bit or 16bit (instead of 32), you got not equivalent (as expected) .

Float is required for the lossless round trip, but if it was a problem with float processing in general then the YV24 should have errors too
poisondeathray is offline   Reply With Quote
Old 21st August 2019, 22:02   #4830  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 1,806
Quote:
Originally Posted by StainlessS View Post
My natural inclination when using up shift would be to fill the 'nullified' least signifcant bits in destination number with the most significant bits from source number.

But, that aint right, apparently.
The problem with that - mathematically speaking, anyway - is that it would make 8 bit->16 bit different from 8-bit->12 bit->16 bit, for example.

Code:
10111111 -> 1011111110111111

10111111 -> 101111111011 -> 1011111110111011
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 21st August 2019, 23:46   #4831  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,032
Taken to extreme,
1 bit to 16 bit
Code:
                 1  
  1111111111111111 # same as before but with the repeated downshift of most signiicant bits (recurring stuff).

Shift only
                 1 # 1=max poss
  1000000000000000 # half max poss
It is an imperfect universe that we live in, guess we just gotta put up with it. [thank goodness that its not forever]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 22nd August 2019 at 00:08.
StainlessS is offline   Reply With Quote
Old 22nd August 2019, 10:04   #4832  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,748
@FranceBB
Is there any spec (REC-BT, ...) where these full range data are described/explained ? And especialy if they also report to RGB.
Because i'm almost sure (but maybe i'm wrong...), and mine included, the avs filters use for full range max value (2^nBits)-1 and not 255 << (nBits-8) (with exception of 10 bits).
Seems a new value for the range parameter of my filter may be needed.
__________________
My github.

Last edited by jpsdr; 23rd August 2019 at 10:01.
jpsdr is offline   Reply With Quote
Old 22nd August 2019, 14:05   #4833  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,071
http://www.vapoursynth.com/doc/functions/resize.html

See "pixel range" at the bottom
TheFluff is offline   Reply With Quote
Old 23rd August 2019, 10:02   #4834  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,748
It says that full range is (2^nBits)-1 and not 255 << (nBits-8), so, it's different from what FranceBB says.
But it's what T-REC-H.265-201802-S!!PDF-E says in page 409.
__________________
My github.

Last edited by jpsdr; 23rd August 2019 at 10:15.
jpsdr is offline   Reply With Quote
Old 23rd August 2019, 23:43   #4835  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,969
I noticed an oddity in the way negative zero variables are converted to string when they're float. The variables in the script are never negative, but I want them to display that way, unless they're zero. The upshot is....

This displays as 0
subtitle(string(float(-0), "%.0f"))

While this displays as -0
subtitle(string(-float(0), "%.0f"))

As does this
xxx = float(0)
subtitle(string(-xxx, "%.0f"))

And this
xxx = -float(0)
subtitle(string(xxx, "%.0f"))

But not for a double negative.
xxx = -float(0)
subtitle(string(-xxx, "%.0f"))

Avisynth 2.6 doesn't display the negative sign for any of them.

Cheers.
hello_hello is offline   Reply With Quote
Old 25th August 2019, 14:07   #4836  |  Link
VoodooFX
Registered User
 
Join Date: Sep 2008
Posts: 78
Continuation from this post.

This script is 71 times faster in AVS v2.6 vs AVS+ :
Quote:
clip = LWLibavVideoSource("C:\LagarithYV12.avi.lwi")
clp = clip.ConvertToRGB24
a = clp.Trim(20,-30)+clp.Trim(2000,-30)
a = a+a+a+a+a+a+a+a+a+a
a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a
return a
VoodooFX is offline   Reply With Quote
Old 25th August 2019, 17:29   #4837  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,032
See here:- https://forum.doom9.org/showthread.p...27#post1883127
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th August 2019 at 08:06.
StainlessS is offline   Reply With Quote
Old 26th August 2019, 03:09   #4838  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 782
This might be a very odd and niche feature request, but here goes...

In ConditionalReader, the OFFSET command offsets all lines below it a fixed number of frames, but it can be overwritten by another OFFSET command. My request is to add a new command, something like ADDOFFSET (or if you can think of a better name), where it doesn't overwrite previous offsets, but adds its offset to the previous one.

For example:
Code:
  # This is for frame 0, obviously
0 value0
OFFSET 2
  # This is for frame 3: 1 + offset 2
1 value1
  # This adds 4 to the previous offset of 2
ADDOFFSET 4
  # So this is now for frame 8: 2 + offsets 2+4
2 value2
  # And this will overwrite the previous offset(s), as before
OFFSET 10
  # So this is for frame 13: 3 + offset 10
3 value3

edit: Or maybe you could call it by
Code:
OFFSET+=4
  # and of course also
OFFSET-=4
  # while maybe even adding the possibility to write
OFFSET=2
edit2:
And for the motivation of this... say you have a long list of values for sequential frames, but then you need to add gaps here and there. With the current implementation of OFFSET, you can't change one gap without having to also change every OFFSET after it, and you can't directly read what the gap is without having to subtract the previous OFFSET.

Last edited by ajp_anton; 26th August 2019 at 03:29.
ajp_anton is offline   Reply With Quote
Old 26th August 2019, 03:22   #4839  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,071
Quote:
Originally Posted by VoodooFX View Post
Continuation from this post.

This script is 71 times faster in AVS v2.6 vs AVS+ :


is there any difference if you replace all the splices with loop()?
TheFluff is offline   Reply With Quote
Old 26th August 2019, 03:49   #4840  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,032
Quote:
is there any difference if you replace all the splices with loop()?
still a significant difference with blankclip.
Code:
BlankClip(length=1000000000)
return Last



EDIT: and with mostly loop()s
Code:
#clip=LSMashvideoSource("1941 Flint Michigan Parade [Low, 360p].mp4")
clip=BlankClip(length=3000)
clp = clip.ConvertToRGB24
a = clp.Trim(20,-30) + clp.Trim(2000,-30)

#a = a+a+a+a+a+a+a+a+a+a                            # *10
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20, = * 1600000 ::: 1600000 * 60 = 96,000,000

a=a.loop(1600000,0,59)

return a #.info  # 96,000,000 frames ### Max possible frames = $7FFFFFFF, ~= 2,000,000,000
avs+ capped at about same time as v2.6 std had completed, and with still lots of time yet to expend.



EDIT: Avs+ a bit faster if no ConvertToRGB24 [EDIT: Avs 2.6 std takes about 2:45.00 for same script, unlike previous, quite a bit longer than avs+]
Code:
#clip=LSMashvideoSource("1941 Flint Michigan Parade [Low, 360p].mp4")
clip=BlankClip(length=3000).Killaudio
#clp = clip.ConvertToRGB24
clp=clip
a = clp.Trim(20,-30) + clp.Trim(2000,-30)

#a = a+a+a+a+a+a+a+a+a+a                            # *10
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20
#a = a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a+a        # *20, = * 1600000 ::: 1600000 * 60 = 96,000,000

a=a.loop(1600000,0,59)

return a #.info  # 96,000,000 frames ### Max possible frames = $7FFFFFFF, ~= 2,000,000,000
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th August 2019 at 05:06.
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.