yCMS - Color Management System

[madshi]

yCMS does its internal calcuations in linear light RGB, AFAIK, just like the Lumagen Radiance. However, yCMS calculates in 64bit floating point. I don't think the Radiance is that exact.

[cyberlolo]

Originally Posted by yesgrey

madshi answered it really well. I'll just add that the Gamut_Measurements command was created also having in mind the gamut correction for multiple saturations (not yet supported), and that would not fit in the Output_Primaries command.
With only one saturation allowed there is an overlapping between the two commands, but we decided to still introduce this command now because it's more user friendly than the Output_Primaries. A big part of the users would not need (or dare ) to go past the Warning, so I must say that you are one of the braves...
The "Gamut_Measurements" command should always be used when the display's coordinates are measured, no matter what the source is. The source should be reflected only in the Input commands, and in none of the output commands. The commands relative to measures are all output commands, because you are measuring the output (display).

The Output_Primaries command should be used only when a user cannot measure the display's primaries but he has an idea of what videoStandard his display might be. Then, he should use this command to set it accordingly.

Ok, I think I got it now. I won't use Output_Primaries at all, and will use Gamut_Measurements with both SD and HD sources. Thanks for explaining!

And hey, I'm one of the braves because I need to use that awesome Input/Output_Range to set my black and white points at 16 and 240, as I asked you some time ago. By the way, it works like a charm (no banding), so thank you again!!

Originally Posted by madshi

No, that's wrong. As I said in my previous comment, you should simply forget all commands in the "defining standards" section. They are for people who want to reencode (not play/watch) videos, e.g. Blu-Ray -> x264 MKV.

All commands you'll ever need for madVR and video playback are in the first 3 sections of the manual (mandatory, optional, calibration). Please simple *forget* the commands in the "defining standards" section.

Thank you too for your help, madshi. But I think there's at least one command in the "Standards definition" section that it's really useful for watching/playing videos: that is the "Input/Output_Range", which allows to set black and white levels of the video data, defining the valid range (all video data outside of it is clipped). This command solves the problem that I posted in your madVR thread some time ago, as gives the users the chance to set top white level at any point between 235 and 254, without introducing banding (which ffdshow's levels filter does). So with this, anyone can decide if to discard all WTW info (white level @ 235), only a part of it (let's say setting white at 245), or if preserve all WTW up to 254. And to me, this is a really important feature. Don't you think?

[cyberbeing]

There appears to be a bug where if you don't use Input_Format & Output_Format and instead use the "Advanced settings" for everything, yCMS complains that there is no Compression_Method value. Since it is supposedly optional, it should be defaulting to 0 when no value is specified.

Also, could you please explain what Compression_Method does? Is it currently just a placeholder at this point?

Other than "0" what would be an example value? None are listed in the manual.

[yesgrey]

Originally Posted by cyberbeing

How can I use the new IRE function and keep my Blacks, Black?

I've found the bug. It will be corrected in next version.

Originally Posted by madshi

So you're aware of a noise problem in dark areas and there's hope for an improvement?

Yes. I simply forgot to include it in the first post as "Known issues". I improved it compared to cr3dlut, but the definitive solution would take me a little more time...

Originally Posted by FoLLgoTT

I don't quite understand that. Luma (Y) should be independant from xy.

In a perfect display yes, but I think there isn't any in the market yet... I only will know for sure when I have it working. It might be possible to have it all correct.

Originally Posted by FoLLgoTT

Maybe I'm wrong, but from my understanding of color management you have to correct all mixed colors if the primaries/secondaries get corrected.

That's the idea: correct all colors. The primaries/secondaries are only the reference values.

Originally Posted by FoLLgoTT

So I'm wondering how yCMS interpolate the mixed colors when correcting gamut.

There is no interpolation involved in yCMS. All colors are corrected in linear RGB color space using 64 bit FP for each component. Then, it's applied gamma encoding and the result is converted into the output format bit depth selection.
The only interpolation involved is in madVR, which interpolates 3DLUT input/output using tri-linear interpolation for finding more accurate values for its 32 bit FP calculated values.

Originally Posted by cyberbeing

There appears to be a bug...

Thanks. It will be corrected in next version.

Originally Posted by cyberbeing

Also, could you please explain what Compression_Method does? Is it currently just a placeholder at this point?

Yes.
When we defined the 3DLUT file format specification we considered the possibility of compressing the files, due to its big size and high loading time, so this option is to be used when the 3DLUT files could be compressed.

madshi and I are currently working on adding compression to the 3DLUT files.

[Blue_MiSfit]

It seems to really take advantage of this, one must have good calibration equipment, correct?

~MiSfit

[Mark_A_W]

Get a HCFR Blue_misfit.


But I don't think it's ready yet. I don't think there is a clear path to go from measurements to the 3DLUTs yet.

[yesgrey]

yCMS v1.1 released

http://yesgrey.com/ycms.html

   - Increased Grayscale_Measurements maximum to 11.
   - Compression_Method was not being considered as optional.
   - Fixed incorrect black point when Grayscale_Measurements ire 0 was not set.

[cyberbeing]

Originally Posted by yesgrey

   - Fixed incorrect black point when Grayscale_Measurements ire 0 was not set.

Well blacks are now black, but near black shadows are crushed into bright solid tones, which makes the problem orders of magnitude worse.


| Regular 3DLUT on LEFT | 3DLUT with IRE on RIGHT |

All of the below give horrible results. Try it with a dark scene and see for yourself.

# Set input format
Input_Format   HD   YCbCr    8

# Set output format
Output_Format  HD   RGB_PC  16

# Grayscale IRE Measurements
Grayscale_Measurements
 10    0.486085    0.472363    0.497613
 20    2.855507    2.751755    2.748106
 30    7.040504    6.968139    7.044012
 40   13.197106   13.020103   13.015621
 50   21.422743   21.428576   21.564960
 60   31.921806   31.852612   31.998531
 70   45.422878   45.317829   45.406380
 80   60.435123   60.174709   60.314659
 90   78.802399   78.381996   78.620575
 100  97.811592   98.124657   98.652870

# Display Gamut Measurements
Gamut_Measurements 0.626 0.343 0.294 0.607 0.149 0.078 0.3127 0.3290

# Set input format
Input_Format   HD   YCbCr    8

# Set output format
Output_Format  HD   RGB_PC  16

# Grayscale IRE Measurements
Grayscale_Measurements
 0     0.000000
 10    0.486085    0.472363    0.497613
 20    2.855507    2.751755    2.748106
 30    7.040504    6.968139    7.044012
 40   13.197106   13.020103   13.015621
 50   21.422743   21.428576   21.564960
 60   31.921806   31.852612   31.998531
 70   45.422878   45.317829   45.406380
 80   60.435123   60.174709   60.314659
 90   78.802399   78.381996   78.620575
 100  97.811592   98.124657   98.652870

# Display Gamut Measurements
Gamut_Measurements 0.626 0.343 0.294 0.607 0.149 0.078 0.3127 0.3290

# Set input format
Input_Format   HD   YCbCr    8

# Set output format
Output_Format  HD   RGB_PC  16

# Grayscale IRE Measurements
Grayscale_Measurements
 0     0.000000    0.000000    0.000000
 10    0.486085    0.472363    0.497613
 20    2.855507    2.751755    2.748106
 30    7.040504    6.968139    7.044012
 40   13.197106   13.020103   13.015621
 50   21.422743   21.428576   21.564960
 60   31.921806   31.852612   31.998531
 70   45.422878   45.317829   45.406380
 80   60.435123   60.174709   60.314659
 90   78.802399   78.381996   78.620575
 100  97.811592   98.124657   98.652870

# Display Gamut Measurements
Gamut_Measurements 0.626 0.343 0.294 0.607 0.149 0.078 0.3127 0.3290

Do you think you are running into a problem because the BT.709 curve (red line in the above image) crushes low luminance blacks. It appears yCMS may be adjusting those crushed (non-existant) BT.709 transfer curve blacks into actual visible colors when you shouldn't be.

[Blue_MiSfit]

I have a Pantone Huey Pro. It doesn't look like HCFR supports it, but we'll see...

[kostik]

I have the same issue that cyberbeing reported. The shadows turn into red with or withoust the 0 IRE evrything but this issue works well, hope it will be fixed soon.
Thanks for your work

[yesgrey]

Originally Posted by kostik

I have the same issue that cyberbeing reported. The shadows turn into red with or withoust the 0 IRE evrything but this issue works well, hope it will be fixed soon.
Thanks for your work

I'm working on it...

[BeNooL]

What are the value measurement expected by yCMS for the Grayscale_Measurements option?

In the manual, the values given as example are quite large (more that 450 at 100IRE) while the one shown by cyberbeing are more in the 0 to 100 range (same for me).
So just wondering if the proper data is given for the computation of this correction?

[yesgrey]

Originally Posted by BeNooL

What are the value measurement expected by yCMS for the Grayscale_Measurements option?

Like said in the manual: >= 0.0

Originally Posted by BeNooL

In the manual, the values given as example are quite large (more that 450 at 100IRE) while the one shown by cyberbeing are more in the 0 to 100 range (same for me).

That was intentional to let people know they could use any values.
The units are not important, because internally all values are divided by the maximum value.

[BeNooL]

Okay so let me rephrase the question: in HCFR Colorimeter which of the grayscale measurement has to be copied over to yCMS config file?
X, Y, Z, R, G, B, x, y etc?

PS: thank you very much for your development effort on this project.

[yesgrey]

Originally Posted by BeNooL

in HCFR Colorimeter which of the grayscale measurement has to be copied over to yCMS config file?
X, Y, Z, R, G, B, x, y etc?

I don't have HCFR installed, but I think it should be Y (Luminance)...

[cyberbeing]

Originally Posted by yesgrey

I don't have HCFR installed, but I think it should be Y (Luminance)...

Y (Overall Luminance) if you are only inputting a single value.

R (Red Luminance), G (Green Luminance), B (Blue Luminance) if you are inputting multiple values.

[Thunderbolt8]

a question regarding 1.1 (and other upcoming versions): I only need those 4 3dlut files for madVR, 2xHD and 2xSD as described in your example. I created these files with 1.0, should I now recreate those files with v1.1 and all other upcoming versions?

[BeNooL]

Thanks for the clarifications Yesgrey and cyberbeing.

Thunderbolt8, yes you have to regenerate the 3Dlut files with each new version of yCMS.

[yesgrey]

Originally Posted by Thunderbolt8

a question regarding 1.1 (and other upcoming versions): I only need those 4 3dlut files for madVR, 2xHD and 2xSD as described in your example. I created these files with 1.0, should I now recreate those files with v1.1 and all other upcoming versions?

If the new version does not correct any bugs shown in your current 3DLUT files there is no need to.
If you haven't used the Grayscale_Measurements you're should be ok, but I strongly advise to recreate the files whenever any bugs are fixed. It would be safer.
You can create a simple batch file that calls each of the input files and creates all the 4 files, which would take you only a few seconds. I don't think that's too much work...

[janos666]

Nice progress. The Grayscale_Measurements command gives me better result than I had with the Output_Primaries command and it is nearly usable (for me) now.

But there is a huge bug with the out of gamut colors (I think) in this version. Take a look at the blue area on the upper left corner: off - on
I made these screenshots with enabled/disabled 3dlut which was built with these settings:

# Source video format
Input_Format       HD YCbCr 8

#3DLUT output format
Output_Format      HD RGB_PC 16

# Gamut correction - measured native x;y coordinates
Gamut_Measurements 0.657073 0.329413 0.231911 0.690324 0.140940 0.090552 0.311880 0.329555

I do not want to guess about the real problem (output bitrate, my perception, poor quality video source, poor quality display) but I think that gamma correction sometimes kills the shadow details. I can hear as shadows are screaming and dieing here: off - on (Sorry but I was a Lexx fan. )
I used the lines above with these:

#Gamma correction - measured grayscale RGB luminances
Grayscale_Measurements
0 0.151752 0.146392 0.138733
10 1.564805 1.551411 1.491997
20 4.668003 4.810841 4.791349
30 10.406371 10.604137 10.320132
40 19.016638 18.991844 18.722298
50 30.179890 30.871260 30.149662
60 44.580005 45.085168 44.541965
70 63.151944 64.109925 63.294323
80 84.184549 85.582548 84.539725
90 110.365791 112.272175 111.200465
100 140.120883 142.500616 141.767822