Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.


Go Back   Doom9's Forum > Hardware & Software > Software players

Thread Tools Search this Thread Display Modes
Old 25th November 2012, 12:02   #1  |  Link
Registered Developer
Join Date: Sep 2006
Posts: 9,140
developer discussion: custom pixel shaders -> color space, black/white definition

Hello devs,

now we have multiple renderers (VMR9, EVR-CP, madVR) which support custom pixel shaders. I think it's time to explicitly define a few things which are currently not clearly defined:

(A) Which colorspace should custom pixel shaders run in?

1. R'G'B' (gamma corrected RGB)
2. RGB (linear light RGB)
3. BT.709 Y'CbCr (gamma corrected YCbCr)
4. BT.709 YCbCr (linear light YCbCr)
5. something else

Currently VMR9 and EVR seem to use R'G'B'. Should we stick to that? That would be fine with me. But we should clearly define this.

(B) Where is black and where is white?

1. black 0.0, white 1.0
2. black 16/255, white 235/255
3. black (16 * 256)/65535, white (235 * 256)/65535
4. something else

Currently VMR9 and EVR seem to run pixel shaders with black at 0.0 and white at 1.0. Should we stick to that?

Black at 0.0 has some clear advantages. One key advantage is that the typical RGB <-> YCbCr matrixes simply work as expected. With black at 16/255 the usual matrixes would not work correcty, AFAIK. So if a shader needs to convert RGB to YCbCr, having black at 0.0 makes things much easier. The same applies when a shader wants to convert R'G'B' to RGB because the proper conversion is done with "pow" when black is at 0.0.

However, there's one big disadvantage with having black at 0.0: When using cardinal texture formats for temp storage, BTB and WTW are clipped. Which is what currently actually happens with VMR9 and EVR. Having black at 16/255 and white at 235/255 would preserve BTB and WTW even when using cardinal textures.


I'm not really sure what my preferred solution is. I would vote for either R'G'B' with black at 0.0, or for R'G'B' with black at 16/255. In the first case I'd have to use float textures to preserve BTB/WTW which would probably hurt performance a bit. In the 2nd case I could stick to cardinal textures, however life would be more difficult for shader writers.

Comments / votes, anyone?
madshi is offline   Reply With Quote
Old 25th November 2012, 14:33   #2  |  Link
Registered User
Underground78's Avatar
Join Date: Oct 2004
Location: France
Posts: 567

How big would be the performance impact? I tend to think that it would be great to avoid adding a new complexity when writing shaders. Also, knowing that basically all shaders currently around are for R'G'B' with black at 0.0, it would require to update all those existing shaders to work with the new definition.
Underground78 is offline   Reply With Quote
Old 25th November 2012, 15:07   #3  |  Link
Registered Developer
Join Date: Sep 2006
Posts: 9,140
To be honest, I'm not fully sure how big the performance impact would be. Probably it depends on the GPU generation. If the shader math is complex enough, memory performance might not matter at all. However, for simple shaders or for texture intensive shaders probably the memory performance will be the bottleneck. I guess if performance is important, float16 could be used. Personally, I guess if we decide on black = 0.0, I'll probably go float32 for madVR custom pixel shaders.

If we do stay with black = 0.0, then probably VMR9 and EVR should be updated to use at least float16, so BTB and WTW are not lost.
madshi is offline   Reply With Quote
Old 25th November 2012, 15:15   #4  |  Link
Registered Developer
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,254
If the VMR/EVR problem with cut-off BTB/WTW is the only reason against black = 0.0, and using R'G'B' is the preferred solution as well, i would vote for simply staying with that. No need to change existing shaders, and if they may be simpler to write, as you say, another reason to stick to it.
If VMR/EVR developers care, they could as well change to float16/float32 textures, but they have been running like this for years without many complaints...
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 25th November 2012, 19:43   #5  |  Link
Ver Greeneyes
Registered User
Join Date: May 2012
Posts: 447
Most shaders probably assume that the format is RGB or R'G'B', and some might even do a conversion to YUV themselves for color difference evaluation. Perceptually linear J'a'b' (CAM02-UCS) is probably too big of a performance hit even for recent cards, and I was never able to solve its problems near black (though I'll get back to messing with it soon if only so I can have something to show for my efforts).

Edit: actually got somewhere with this, though there are still some odd problems to solve. I don't mean to hijack this topic though, just a heads up that getting this working at all isn't necessarily an issue.

Last edited by Ver Greeneyes; 26th November 2012 at 06:04.
Ver Greeneyes is offline   Reply With Quote

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +1. The time now is 19:28.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.