Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
10th December 2018, 21:35 | #81 | Link | |
Guest
Posts: n/a
|
Quote:
I haven't tried your work. I have some interest in programming video filters and I could do some work for you if you need. Actually, I'm interesting in formal evaluating of color space transformations. In particularly, I've tried a simple transformation - truncating the color codes' two LSB, maybe analogous to ConvertXYZ_Scale_HDRtoSDR with this script: Code:
DGSource("F:\Gladiator-2160p\Gladiator.hevc.dgi",i420=true,fulldepth=true) z_ConvertFormat(width=1920,height=1080,pixel_type="YV12",dither_type="none" Using Sony Catalyst Browse I've seen Source: Rec.2020/PQ, Working Rec.709, Preview: Rec.709. and video in the working space visually undistinguished from the source with waveform under 67 %. So, I'm interested in programming a waveform/histogram generating Avisynth function. |
|
16th August 2019, 00:39 | #84 | Link |
Registered User
Join Date: Jul 2018
Posts: 454
|
Conversion 2020->709:
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=1, OutputMode=2) Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2) Last edited by StvG; 16th August 2019 at 00:54. |
16th August 2019, 22:06 | #85 | Link |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,317
|
pColor and also pRx,pRy,pGx,pGy,pBx,pBy,pWx,pWy are used only to compute the Min/Max values of XYZ, which are not a stable [0,1] in all situations like RGB, but are White Point and R,G,B point chromaticity dependent.
With this, with integer format, 16 bit for exemple, i can affect [0]=Min(X,Y,Z) [65535]=Max(X,Y,Z) in the inputs. These previous parameters have no effect in float format. The parameter is here for optimal theorical performance, but i don't know if it will produce real visible effect between the standard BT. It will if you use different White Point (his has the biggest effect), or realy very different R,G,B point. Roughly, range is Y [0,1.00], X[0,0.96], Z[0,1.10]. Didn't check more decimal, but maybe for BT.709 max Z is 1.1001 and for BT.2020, max Z is 1.002. I think it's something like that, realy no more. Well, the previous parameters are here to be sure you have input proper range value used in integer mode. And again, of course, no effect in float mode because no needs. Described also in the Warning page 3 of ColorConversion.pdf document i've provided within the binaries package.
__________________
My github. Last edited by jpsdr; 16th August 2019 at 22:28. |
17th August 2019, 00:56 | #86 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
jpsdr,
I'm still not clear as to why I'm getting such bizarre results for XYZ conversions here. Am I doing something wrong? I read your reply in the other thread and looked at the PDF, and I can see rec.601_525 line and rec.601_625 line are different, but the question I'm trying to get a simple answer to is... why do other programs such as ColorMatrix & DitherTools only have/require one version of rec.601? I assume my understanding is wrong, but I thought once analogue PAL and NTSC are converted to digital, the colorimetry is the same, and therefore there's only one flavour of rec.601 required for colorimetry conversions. When I play an NTSC DVD encode, is the same colorimetry being used by MPC-HC to convert to RGB on playback as for a PAL DVD encode? StvG said in the other thread I should be using rec.601_625 to convert between HD and SD etc in digital-land, and I don't doubt him, but once again I'm just after a simple explanation as to why. Cheers. Last edited by hello_hello; 17th August 2019 at 01:10. |
17th August 2019, 02:14 | #87 | Link |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,317
|
Neither my dictionnary, google trad or me understand "borked"...
There is no XYZ colorspace in avisynth, so, it's tagged RGB. So, the system (avs, Windows, Video card drivers, monitor) diplays (linear) XYZ data, thinking it's non-linear RGB. Not so suprise result is odd... But basicaly, you'll see darker colors than original. Effect even worse (a lot darker) with HDR data. If you haven't, read the pdf document i've put with the filter, i've tried to explain how things work (at least, what i've understood). Maybe it will help you. I've put the two rec.601 because there was two in the spec, so, i made them both avaible. After, colorimetry is not realy my best field, so, i would suggest you to follow StvG advice. After, i'm not sure there is realy need to go through XYZ (neither Linear RGB) for 601 <-> 709. Colorimetry parameters are almost identical, this is why people always do the convertion staying in the standard non-linear space, so, just by a simple matrix translation. After, if you want perfect accuracy... For 2020 <-> 709/601, you need to go through XYZ, because colorimetry parameters are different, and so it's only through XYZ that you can do it. Only Linear RGB is not enough and will produce improper results. Described in R-REC-BT.2087-0-201510-I!!PDF-E. So, it's 3 am for me, time to bed...
__________________
My github. |
17th August 2019, 03:24 | #88 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Simple answer is convention. It's the way everyone does it 1) Matrix for bt470bg (pal, or 601-6_625), smpte170m (ntsc, or 601-6_525) are the same, 709 different 2) Transfer for bt470bg, smpte170m, and 709 are the same 3) Primaries are different for all 3, but look at footnote 2 Code:
It is recognized that a practice is now sometimes used by which, when programs produced in HDTV are release in SDTV, their HDTV pixel map is re-mapped onto the SDTV pixel map without changing the colorimetry of the original program. ie. So technically it's a HD => SD conversion with uncorrected 709 primaries . But you can do whatever you want - you control matrix, transfer, or primaries for source/destination with tools like zimg/zlib (resize in vapoursynth, avsresize in avisynth) dither tools' "newer brother" fron the same author, the vapoursynth's fmtc also give you control over these |
|
17th August 2019, 04:03 | #89 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
poisondeathray,
Thanks for the reply. Maybe I'm being thick, but I'm still not 100% clear on this. Even the section you quoted only refers to HD and SD colorimetry, implying SD is a single colorimetry. The pdf appears to me to explain the method for converting analogue to digital. Below figure 2.6 it says: Chromaticity coordinates specified are those currently used by 625-line and 525-line conventional systems. I doubt any of us are using 625 or 525 line displays any more, and it seems the colorimetry co-ordinates convert "from" those systems. Even if they're different in analogue form, it seems bizarre to use two formulas to convert them to different digital SD colorimetries rather than them ending up the same. If they're not intended to convert to the same colorimetry, why use different formulas? I must be missing the obvious here, and I won't get a chance till tomorrow, but I'll do some more reading to see if I can work out what I'm missing. In the mean time though, if there's two rec.601 colorimetries in digital form, which one do programs with only a single rec.601 option normally use? Cheers. PS. I've had the table in the ColorMatrix help file in my head, which is one reason I'm confused about it now. To me it implies there's only a single rec.601 in digital form but maybe I've been mis-interpreting it. Quote:
Last edited by hello_hello; 17th August 2019 at 04:11. |
|
17th August 2019, 04:40 | #90 | Link | |||
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
There is a 625 and 525 chart. Notice their primaries for Red, Green, Blue are different As mentioned earlier, in practice, the primaries are not adjusted for HD to SD conversion. You ignore it You need to define matrix, transfer , primaries when converting from something to another. Input and output. Thats 6 values in and out. It becomes more important when you have other formats used in production like linear exr , then the transfer becomes important . The HD <=> SD case is covered by by 1,2,3 in my post above. 1) matrix is used 2) Transfer is cancelled out 3) primaries are usually ignored Quote:
If you mean "primaries" it's usually unspecified or ignored. But some do specify it in metadata, even if it's actually ignored in conversions. (and you can verify these things , since you have control over matrix, transfer and primaries with vapoursynth resize or fmtc, or avsresize in avisynth) The way you test it, is take known colors and values such as colorbars, run it through your HD conversions to SD (or SD to HD if you want) You can specify the matrix/transfer/primaries in/out . Keep in mind - the way you or whatever program converts back to RGB is important if you are "viewing" it Keep in mind that sometimes there are conversions done in the GPU, sometimes in the renderer You can do this for different programs, editors, media players to check behaviour It takes alot of time to do these tests, but I did this a while back and my findings were primaries are mostly ignored in most programs. If you corrected for primaries (709 primaries in, 601-6_625 or 601-6_525 out) the conversion looked different than the known values you started out with in HD . The exception is vapoursynth. It reads metadata, and that metadata affects conversions. Often, you have to override the "props" or various properties if you want it to behave like other programs, very tricky if you forget (e.g. something might be flagged interlaced, though progressive content, like a PAL DVD, but vapoursynth will treat it interlaced for all operations unless you override the props) Quote:
That Colormatrix table you posted only specifies matrix, not the transfer or primaries. The matrix values for 601-6_525 and 601-6_625 end up being the same - KR = 0.299; KB = 0.114 . But 709 is different. That's why we use matrix=blah when doing those conversions HD<=>SD conversions . But no matrix conversion between "PAL" and "NTSC" because they are the same Better tables are in the more modern Rec. ITU-T H.265 standards document, which include 2020 and a bunch others along with their values and equations E3 Color Primaries E4 Transfer Characteristics E5 Matrix Coefficients Common tools like popular media players, only use matrix when converting back to RGB for display. The primaries are are not accounted for. You will get different colors than expected if you start adjusting primaries in that HD to SD or SD to HD case. Same with video editors and other tools Last edited by poisondeathray; 17th August 2019 at 05:35. |
|||
18th August 2019, 20:46 | #91 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
poisondeathray,
Thanks for the info. I also did some more reading and I think I'm now closer to an understanding. First we have this table showing the difference between NTSC and PAL color primaries (from Wikipedia). Code:
RGB color space parameters for Rec. 601 Color space White point (D65) Primary color xW yW xR yR xG yG xB yB 625 line 0.3127 0.3290 0.640 0.330 0.290 0.600 0.150 0.060 525 line 0.3127 0.3290 0.630 0.340 0.310 0.595 0.155 0.070 Red and blue for NTSC and PAL are so close they can probably be considered the same, while green is a little different. For Rec.709 the red and blue primaries are exactly the same as 625 line Rec.601, while green is roughly halfway between the two Rec.601 versions. https://www.image-engineering.de/con...6/xy-Locus.jpg Maybe now I understand why I've seen PAL DVDs that appear to have a slight green tinge. I assume they were encoded using NTSC primaries, or something.... I posted an example of that during a discussion quite a while back. Transfer Characteristics = gamma, which is the same for rec.601 and rec.709. Color Matrix is the formula for converting YUV back to RGB (or more correctly converting YCbCr to RGB). From what I understand now, there's only one formula for each, but there's constant(s) in each formula that are different for 525 line, 625 line (and rec.709) to account for the different primaries. If the wrong "constants" are used you don't convert back to exactly the same RGB you started with. Is that a correct assumption? So the formula for converting between the rec.601 and rec.709 matrices is a standard formula used by most programs, that should take into account the different "constants" for red, blue and green, but doesn't because that makes stuff-all difference compared to using the wrong matrix, and I assume in many cases the original RGB primaries might be unknown anyway. I think I finally understand what "ignoring the primaries" means.... I assume HDRTools does account for those primaries... hence the two flavours of rec.601.... so to be technically accurate, assuming I'm encoding with x264 for example, if I converted HD to SD using the rec.601_625 formula, I should specify bt470bg for the color primaries and for the color matrix (rather than rec.709 for primaries and bt470bg for matrix), and if I'm upscaling from PAL to HD it'd be bt709 for the primaries and matrix (rather than bt470bg and rec.709)? Not that I've bothered setting the color primaries for a long time, just the matrix. How am I doing? Thanks again! Last edited by hello_hello; 18th August 2019 at 21:11. |
18th August 2019, 21:32 | #92 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
These definitions possibly come across as being somewhat insulting, but that definitely wasn't my intention. What I'm failing to understand (I seem to be doing a bit of that lately) is how the intermittent XYZ colorspace matters when I shouldn't be seeing it. The screenshots were taken from the output of a script using this (2020 to 601). ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=3, pColor=1, OutputMode=2) Shouldn't I be seeing a fairly similar result to this (the expected result)? ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV(Color=3, OutputMode=2) Thanks. Last edited by hello_hello; 18th August 2019 at 21:44. |
|
19th August 2019, 00:10 | #93 | Link | |||
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Quote:
Quote:
Note that in x264, those --colorprim xxx --transfer xxx --colormatrix xxx switches are just VUI metadata. They don't actually do anything. It's up to the other recieving software how it's handled . Some pay attention to that metadata. e.g vapoursynth. So if you do that actual primaries adjustment, and flag it accordingly, you end up getting the expected colors. |
|||
19th August 2019, 06:40 | #94 | Link | ||
Registered User
Join Date: Jul 2018
Posts: 454
|
Quote:
Edit: Quote:
Last edited by StvG; 19th August 2019 at 07:03. |
||
19th August 2019, 09:29 | #95 | Link | |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,317
|
Quote:
So, when going through XYZ, you adjust primary colors, you don't when going only through linear RGB, as linear RGB integrate the primary colors of the sensor. And primary colors are different enough to produce a noticeable different result. According R-REC-BT.2087 the proper way is through XYZ for 2020 <-> 709. As i describe in my pdf doc, you have for SDR : [Scene] => [Camera sensor] => [Linear RGB] -> (Gama or OETF) -> [Non Linear RGB] -> (matrix) -> [YCbCr] The primary colors are in the [Camera sensor], so, it's only on this step that 525/625 lines has effect, not in the (matrix). Is my PDF provided with the tools so unclear ? I hoped it was clear enough to explain things, but unfortunately it seems not... (R,G,B) 2020 : (0.708,0.292) (0.170,0.797) (0.131,0.046) (R,G,B) 709 : (0.640,0.330) (0.300,0.600) (0.150,0.060) (R,G,B) 601/625 : (0.640,0.330) (0.290,0.600) (0.150,0.060) (R,G,B) 601/525 : (0.640,0.330) (0.310,0.595) (0.155,0.070) Values for 709/601 even if not identicals, are close enough to assume that only doing the [Non Linear RGB] <-> (matrix) <-> [YCbCr] is enough. OETF is identical for 2020 and 709, so, going to Linear RGB, or staying at non-linear RGB will produce the same result.
__________________
My github. Last edited by jpsdr; 19th August 2019 at 09:53. |
|
19th August 2019, 11:35 | #96 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
LOL. I used the PHP button instead of the Code button, but why not... it looks a bit pretty. PHP Code:
PHP Code:
Now back to reading Mr. Help File again to see if I can discover what "color mode of the data" means as I'm not clear on that now. Does "color mode of the data" mean "matrix"? PHP Code:
Thanks for your help. Last edited by hello_hello; 19th August 2019 at 11:51. |
|
19th August 2019, 11:50 | #97 | Link |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,317
|
Something is wrong here :
Code:
ConvertToYV24() ConvertYUVtoXYZ(Color=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertXYZtoYUV(pColor=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertToYV12() I think it will produce the same result as Code:
ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV() I think i'll have to update the doc of the plugin, because it's clearly obvious that i didn't explain properly the effect of the "p"(previous) parameters, and people misunderstand the purpose of it.
__________________
My github. Last edited by jpsdr; 19th August 2019 at 11:58. |
19th August 2019, 12:21 | #98 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
|
|
19th August 2019, 13:16 | #99 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Okay... time to accept that fact I'm just too stupid to use this program.....
I don't mind being an idiot. It's just frustrating when you're smart enough to know you're idiot. I'm off to drink alcohol until I'm too stupid to know... This was my previous silly. Is it some sort of mathematical co-incidence it produces the same result as a DitherTools rec.2020 to 709 conversion (which is why I thought I finally had it right)? Silly ConvertToYV24() ConvertYUVtoXYZ(Color=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertXYZtoYUV(pColor=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertToYV12() 2020 - 601 ConvertToYV24() Dither_convert_yuv_to_rgb(matrix="2020", slice=false) Dither_convert_rgb_to_yuv(matrix="601", output="YV12") ConvertToYV12() ConvertToYV24() ConvertYUVtoXYZ(Color=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertXYZtoYUV(Color=4, pColor=1, Rx=0.640,Ry=0.330,Gx=0.290,Gy=0.600,Bx=0.150,By=0.060,Wx=0.3127,Wy=0.3290) ConvertToYV12() 2020-709 ConvertToYV24() Dither_convert_yuv_to_rgb(matrix="2020", slice=false) Dither_convert_rgb_to_yuv(matrix="709", output="YV12") ConvertToYV12() ConvertToYV24() ConvertYUVtoXYZ(Color=1, Rx=0.708,Ry=0.292,Gx=0.170,Gy=0.797,Bx=0.131,By=0.046,Wx=0.3127,Wy=0.3290) ConvertXYZtoYUV(Color=2, pColor=1, Rx=0.640,Ry=0.330,Gx=0.300,Gy=0.600,Bx=0.150,By=0.060,Wx=0.3127,Wy=0.3290) ConvertToYV12() |
19th August 2019, 14:03 | #100 | Link |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,317
|
No needs to set the colorimetry parameters if it's for setting the same value that are in the according rec.
As you can see, "silly" provide the same result than dither. It means that dither is only doing a matrix convertion, and doesn't change the colorimetry parameters. As they, in final, do only simple matrix convertion, these 3 should produce the same result (pinterf added rec2020 recently) : "silly" Code:
ConvertToRGB(matrix="Rec2020") ConvertToYV12(matrix="Rec709") Code:
ConvertYUVToLinearRGB(Color=1) ConvertLinearRGBToYUV() I don't see any issue or surprise with these results. What is exactly your issue ? If it's difference between dither and ConvertXYZ, it's expected, as dither is only doing matrix convertion.
__________________
My github. Last edited by jpsdr; 19th August 2019 at 14:06. |
|
|