Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th September 2018, 06:12   #61  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 688
New test and new comparisons!

Original HDR Source: Link
Normal HDR to SDR: Link
Scale default: Link

Hable: Link

Code:
ConvertBits(16)

ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)

ConvertXYZ_Hable_HDRtoSDR(exposure_X=7.0)

ConvertXYZtoYUV(pColor=0)
Not bad at all, very close to the original.
Too bad for the object that Loki is holding: I got it too bright and details are lost.
I tried to play a bit with the white point, but I didn't manage to get it correct:

Hable v2: Link

Code:
ConvertBits(16)

ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)

ConvertXYZ_Hable_HDRtoSDR(exposure_X=7.0, whitescale_X=8.0)

ConvertXYZtoYUV(pColor=0)
Mobius: https://i.imgur.com/tg8V3iF.jpg

Code:
ConvertBits(16)

ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)

ConvertXYZ_Mobius_HDRtoSDR(exposure_X=17.0, peak_X=1.1)

ConvertXYZtoYUV(pColor=0)
Close, but I've got the same problem: the object is too bright and therefore I lose details...

And... here comes the star... here comes the good one... here comes the one that keeps details... here comes my favorite one:
Reinhard: Link

Code:
ConvertBits(16)

ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)

ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.5, contrast_X=0.9)

ConvertXYZtoYUV(pColor=0)
Just perfect, as I expected from Reinhard.
The exposure is exactly the same that I have on the HDR content and details of the object are not lost as I'm able to adjust the contrast accordingly. In my former chart I gave Reinhard an "A" and I'm gonna confirm my grade again.

Updated chart:

- ConvertXYZ_Scale_HDRtoSDR:

Grade: F

I gave it an F before because it didn't give me the correct values at all and I still gotta say that it's not fit for the job. Yes, it can be useful to test values and see how much you can scale down, but the result will be unwatchable anyway.

- ConvertXYZ_Hable_HDRtoSDR

Grade: C+

It's better than the "Scale" function, but I still think that it tries to "exasperate" the colors and give a sort of explosive fake HDR effect to the viewer at the expense of details which, for me, it's an absolute "no go".

- ConvertXYZ_Mobius_HDRtoSDR

Grade: B

It still tries to accentuate the "fake HDR feeling" by bringing colours a bit too saturated, but some people may like it. Overall, it's watchable for many people, but unfortunately, it loses some highlights. The kind of people that like these kind of grading probably doesn't care and actually prefers something like this, but it's simply not me.

- ConvertXYZ_Reinhard_HDRtoSDR

Grade: A

Reinhard gets the job done and it does it well, period.
It was my favorite one and it still is as today confirmed my thoughts once again.
Thanks to the contrast parameter and the exposure parameter it's possible to tweak it to get a very good result.
Yes, it's not gonna have extremely saturated colors and they will be perfectly legit for the BT709 range, the black level is gonna be pretty much perfect, details are gonna be kept and highlights are gonna be preserved.
Overall, it's my recommended way to do HDR to SDR.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP

Last edited by FranceBB; 30th September 2018 at 17:54.
FranceBB is offline   Reply With Quote
Old 30th September 2018, 10:50   #62  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
In the readme :
Code:
   pColor -
      Color mode used in YUV/RGBtoXYZ.
         0 : BT2100
         1 : BT2020
         2 : BT709
         3 : BT601_525
         4 : BT601_625

       Default: 2 if Color=0, 0 otherwise.
So, as you've used :
Code:
ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)
You have to use :
Code:
ConvertXYZtoYUV(pColor=0)
Otherwise, thanks for your tests and report.
__________________
My github.

Last edited by jpsdr; 30th September 2018 at 10:52.
jpsdr is offline   Reply With Quote
Old 30th September 2018, 17:57   #63  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 688
@jpsdr... yes, of course. Tests are legit with pColor=0, but I forgot to include the script while I was testing, so I did it later while I was already writing the post here and I pasted the wrong one. xD
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP
FranceBB is offline   Reply With Quote
Old 1st October 2018, 09:46   #64  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
You can eventualy try/test to be more accurate if you have the SEI mastering parameters of your HDR stream. You don't have them by any chance ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 13th October 2018, 21:16   #65  |  Link
Soulnight
Registered User
 
Join Date: Jan 2017
Location: Germany
Posts: 21
Hello!

thank you for creating this tool.

I am trying to do a small experiment with hdr rec2020 content to display in a certain color the pixels having colors outside REC709 colorspace.

I thought I would have to:

1) first convert from YUV to RGB.
2) Then convert non linear RGB to Linear RGB
3) Multiply by the matrix to go from REC2020 to REC709 colorspace

At 3), I though, I should then tell what to do with RGB values outsides the legal range and specify their value.

Like:

Code:
= decodeTransferFunction(sourcePixel); float3 rgb709 = mul(rgb2020, float3x3(...)); float3 result; if ((rgb709.r < 0.0) || (rgb709.r > 1.0) || (rgb709.g < 0.0) || (rgb709.g > 1.0) || (rgb709.b < 0.0) || (rgb709.b > 1.0)) result = 1.0; else result = 0.0;
But I don't know how to do that in Avisynth.

I currently have that code:

Code:
loadplugin("C:\Users\flole\Downloads\ManualColorMatrix\ManualColorMatrix_26.dll")
loadplugin("C:\Users\flole\Videos\AviSynth\HDRTools\x86\Release_W7\HDRTools.dll")
Directshowsource("F:\Kodi\Demo 4k HDR\The World In HDR 4K Demo.mkv")
ConvertYUVtoLinearRGB(Color=1)
#isRGB()
ManualColorMatrix(1, 1.66049, -0.58764, -0.07285, -0.12455, 1.13290, -0.00835, -0.01815, -0.10058, 1.11873)
Does the function you have written directly combined step 1 and 2?
ConvertYUVtoLinearRGB(Color=1)

Thank you, Merci beaucoup.
Florian

ps:
Final goal will be to visualize what color are "extended" and outside the REC709 color space.
And then analyse a full movie to get a % of pixels and frames showing colors outside the REC709 range.
And then if possible, even say how far and saturated are those colors for different Hue (RED, green, etc).

Last edited by Soulnight; 13th October 2018 at 21:20.
Soulnight is offline   Reply With Quote
Old 14th October 2018, 10:29   #66  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
If you check the some example code provided in the first post (and/or in the Readme), you'll see exactly the script needed for what you want to do.
Indeed it's possible to do this from linear RGB step with a transformation matrix combining the two RGB -> XYZ / XYZ -> RGB matrix, but as you can see in the examples, you can directly do that with my tools, but going by the XYZ step, and not the LinearRGB step.
Also, read the pdf document i've provided with the plugin, i've tried to explain things, so maybe it will help you to understand exactly what you have in each step of the process of transformation signal from the natural real world light to the YUV data we are working on on our PC.
__________________
My github.

Last edited by jpsdr; 14th October 2018 at 10:34.
jpsdr is offline   Reply With Quote
Old 14th October 2018, 12:24   #67  |  Link
Soulnight
Registered User
 
Join Date: Jan 2017
Location: Germany
Posts: 21
Thanks for the answer.
I did did read and I tried to understand the examples as well.

The thing is, I need to sort the "good" from the "bad" pixels afterwards.

And I understand how to define what is an illegal pixel in RGB (in 8bits, bigger than 256 or smaller than 0). With XTZ, it seems more complicated.

Can you confirm that:

ConvertYUVtoLinearRGB(Color=1)

not only convert YUV to RGB but then also convert non linear RGB to linear RGB?
Soulnight is offline   Reply With Quote
Old 15th October 2018, 11:12   #68  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
My bad, i didn't notice your PS. I thought you wanted to make convertion, didn't see you wanted to make some analysis...
Yes, i confirm.
All the functions are indeed made to not produce bad pixels, so any value outside range will be clipped, so the functions for the second conversion step can't be used for your analysis.
__________________
My github.

Last edited by jpsdr; 15th October 2018 at 11:29.
jpsdr is offline   Reply With Quote
Old 15th October 2018, 11:22   #69  |  Link
Soulnight
Registered User
 
Join Date: Jan 2017
Location: Germany
Posts: 21
Thanks! I want to preserve the illegal values for the color outside rec709 so I can color them to black or white for example (using something like a if statement. If pixel r or g or b value outside the 0 to 255 range, then r=g=b=0)

Do you know how to do that?

Last edited by Soulnight; 15th October 2018 at 11:29.
Soulnight is offline   Reply With Quote
Old 15th October 2018, 11:40   #70  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
Unfortunately, not.
But for sure, you'll need filters working on float values to do this.
So, first, set OutputMode=2 on the ConvertYUVtoLinearRGB.
If ManualColorMatrix worked on float values, it would have been exactly what you need just after.
Your coeff matrix on your previous post, you get them reverting the following 709 -> 2020 matrix provided in rec2087 ?
0.6274 0.3293 0.0433
0.0691 0.9195 0.0114
0.0164 0.0880 0.8956
__________________
My github.
jpsdr is offline   Reply With Quote
Old 15th October 2018, 11:57   #71  |  Link
Soulnight
Registered User
 
Join Date: Jan 2017
Location: Germany
Posts: 21
Ok. Too bad.

I will attach this evening my excel spreadsheet where I gave I came up with all the conversion matrix.

It's a multi step process and you have to know your colorspace coordinates.

In the future, I will also do those studies for specific "triangle gamut " that I will have measured for display measuring like 90% dci.

Goal is to see how often do we have information outside of the rec709 colorspace, or outside the real colorspace covered by a tv/projector.

Any idea to do matrix multiplication in float?
Soulnight is offline   Reply With Quote
Old 20th October 2018, 11:58   #72  |  Link
-QfG-
QfG Group Germany
 
-QfG-'s Avatar
 
Join Date: Oct 2018
Location: Germany
Posts: 36
First, this an awesome filter for avisynth, and second , sorry for my bad english, i'm German . Ok i have a small Problem, i will show it here. I post the previe pictures in previemode, click for fullsize...

Beginning, this is the picture with MPC-BE madVR 150 Nits. I will this result in my target file:

Original 4K with madVR 150 Nits @1080p


Try with HLG
Code:
Exposure=2
Whitescale=11.2
ConvertYUVtoXYZ(Color=0, OutputMode=0, HDRMode=2, fullrange=false)
ConvertXYZ_Hable_HDRtoSDR(exposure_X=Exposure, exposure_Y=Exposure, exposure_Z=Exposure, whitescale_X=Whitescale, whitescale_Y=Whitescale, whitescale_Z=Whitescale)
ConvertXYZtoYUV(pColor=0, OutputMode=2)


Ok i like the Details of the HLG , but the problem is, where are my Colors? I need more Colors...

Try with PQ
Code:
Exposure=2
Whitescale=66
ConvertYUVtoXYZ(Color=0, OutputMode=0, HDRMode=0, fullrange=false)
ConvertXYZ_Hable_HDRtoSDR(exposure_X=Exposure, exposure_Y=Exposure, exposure_Z=Exposure, whitescale_X=Whitescale, whitescale_Y=Whitescale, whitescale_Z=Whitescale)
ConvertXYZtoYUV(pColor=0, OutputMode=2)


The result with the standard whitescale=11.2 to dark. With Whitescale=66 the brightness and colors perfect, but too much white (show the skyline in the upper left corner).

Conclusion:

I need more colors in HLG, because i love this detailed picture. For PQ conversion i can use DGTonemap, this is ok. Is there any Solution for more colors and fewer brightness in HLG mode?

Last edited by -QfG-; 20th October 2018 at 12:01.
-QfG- is offline   Reply With Quote
Old 20th October 2018, 13:37   #73  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
First, you have to use the proper mode. If your file is HDR PQ, use PQ, if your file is HDR HLG, use HLG, so, my first question would be : what mode is your file ? Can you post mediainfo result of it ?
According tests made by FranceBB (post #61), for him reinhard produde the best results. Have you tried it ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 20th October 2018, 14:21   #74  |  Link
-QfG-
QfG Group Germany
 
-QfG-'s Avatar
 
Join Date: Oct 2018
Location: Germany
Posts: 36
thanks for response, file transfer is PQ.

Code:
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format_Commercial_IfAny                  : HDR10
Format-Profil                            : Main 10@L5.1@High
Codec-ID                                 : V_MPEGH/ISO/HEVC
Dauer                                    : 2 h 2 min
Bitrate                                  : 35,0 Mb/s
Breite                                   : 3 840 Pixel
Höhe                                     : 2 160 Pixel
Bildseitenverhältnis                     : 16:9
Modus der Bildwiederholungsrate          : konstant
Bildwiederholungsrate                    : 23,976 (24000/1001) FPS
ColorSpace                               : YUV
ChromaSubsampling/String                 : 4:2:0 (Type 2)
BitDepth/String                          : 10 bits
Bits/(Pixel*Frame)                       : 0.176
Stream-Größe                             : 29,8 GiB (79%)
verwendete Encoder-Bibliothek            : ATEME Titan File 3.8.3 (4.8.3.0)        
Default                                  : Nein
Forced                                   : Nein
colour_range                             : Limited
colour_primaries                         : BT.2020
transfer_characteristics                 : PQ
matrix_coefficients                      : BT.2020 non-constant
MasteringDisplay_ColorPrimaries          : Display P3
MasteringDisplay_Luminance               : min: 0.0050 cd/m2, max: 1000 cd/m2
I tried Reinhard, but Reinhard has "warmer" colors as Hable. Hable is perfect, without this white Bloom effect. But now, it's looks much better then before, playing with the switches ...

-QfG- is offline   Reply With Quote
Old 21st October 2018, 11:11   #75  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
... "Display P3" is not of big help, i need the values (not your fault)...
Do you use DGIndexNV by any chance ? If yes, what are the SEI parameters value ?
__________________
My github.
jpsdr is offline   Reply With Quote
Old 21st October 2018, 11:25   #76  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,494
Quote:
MasteringDisplay_ColorPrimaries : Display P3
is the same as
Quote:
Mastering display color primar : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Also called "P3 D65".
https://en.wikipedia.org/wiki/DCI-P3#System_colorimetry
sneaker_ger is offline   Reply With Quote
Old 21st October 2018, 12:52   #77  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
Thanks.
In that case, to have correct result, you should do :
Code:
ConvertYUVtoXYZ(Color=0,Rx=0.680,Ry=0.320,Gx=0.265,Gy=0.690,Bx=0.150,By=0.060)
#Whatever XYZ HDR to SDR (pColor=0,pRx=0.680,pRy=0.320,pGx=0.265,pGy=0.690,pBx=0.150,pBy=0.060)
ConvertXYZtoYUV(Color=2,pColor=0,pRx=0.680,pRy=0.320,pGx=0.265,pGy=0.690,pBx=0.150,pBy=0.060)
__________________
My github.

Last edited by jpsdr; 21st October 2018 at 12:57.
jpsdr is offline   Reply With Quote
Old 21st October 2018, 13:34   #78  |  Link
-QfG-
QfG Group Germany
 
-QfG-'s Avatar
 
Join Date: Oct 2018
Location: Germany
Posts: 36
Quote:
Originally Posted by jpsdr View Post
Thanks.
In that case, to have correct result, you should do :
Code:
ConvertYUVtoXYZ(Color=0,Rx=0.680,Ry=0.320,Gx=0.265,Gy=0.690,Bx=0.150,By=0.060)
#Whatever XYZ HDR to SDR (pColor=0,pRx=0.680,pRy=0.320,pGx=0.265,pGy=0.690,pBx=0.150,pBy=0.060)
ConvertXYZtoYUV(Color=2,pColor=0,pRx=0.680,pRy=0.320,pGx=0.265,pGy=0.690,pBx=0.150,pBy=0.060)
thx, i will try it . Here my DGIndex Log:

Code:
DGHEVCIndexFileNV20 DGIndexNV 2053.0.0.159 X64

DEVICE 0
DECODE_MODES 0,0,0,0,0
STREAM 2 0
CLIP 0 0 0 0
RANGE 0 0 18446744073709551615 0
DEMUX 2,3,4,5,6,7
DEPTH 10
ASPECT 16 9
COLORIMETRY 9 16 9

MASTERING 13250 34500 7500 3000 34000 16000 15635 16450 10000000 50
But i'm to stupid to check, how i must calculate the MASTERING values for HDRTools .
-QfG- is offline   Reply With Quote
Old 21st October 2018, 13:53   #79  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
It's explained in the ReadMe :
Code:
Another way to get them, it's if you're using DGIndexNV, on HEVC HDR stream.
In that case, you'll have in the DGI file this line (for exemple) :

MASTERING 13250 34500 7500 3000 34000 16000 15635 16450 40000000 50

The values are the following:

MASTERING GreenX GreenY BlueX BlueY RedX RedY WhiteX WhiteY MaxMasteringLevel MinMasteringLevel

The GreenX GreenY BlueX BlueY RedX RedY WhiteX WhiteY are by 0.00002 steps.
The MaxMasteringLevel MinMasteringLevel are by 0.0001 steps.

In our exemple, it will result :
Gx: 0.265
Gy: 0.690
Bx: 0.150
By: 0.060
Rx: 0.680
Ry: 0.320
Wx: 0.3127
Wy: 0.3290
Max: 4000
Min: 0.05
__________________
My github.
jpsdr is offline   Reply With Quote
Old 21st October 2018, 14:40   #80  |  Link
-QfG-
QfG Group Germany
 
-QfG-'s Avatar
 
Join Date: Oct 2018
Location: Germany
Posts: 36
Yes i have seen it, but don't understand.

But know i think i find my mistake:

GreenX in DGIndex = 13250. Steps = 0.00002

So 13250/0.00002 = 662500000

I must: 13250x0.00002 = 0,265

Now, i check it

EDIT:

So i'm happy and i like it Changed PQ to HLG
http://screenshotcomparison.com/comparison/122892

thx for this awasome plugin

Last edited by -QfG-; 22nd October 2018 at 22:11.
-QfG- is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.