I’m afraid so, but this is representative of 97% of the programmers that work in the computer graphics area. For most computer people “RGB” is “RGB”.
Part of the problem is that it’s not clear what the scope of Wayland is. A lot of the Waland developers view it as fairly low level nuts and bolts, and so it is not the place for application specific concerns and complexity such as Color Management. But on the other hand they have completely disrupted the established UNIX/Linux graphics API (X11), without fully replacing all it’s key functionality (such as CM support, VideoLUT access, or network transparency etc.). This is understandable from the development point of view of taking one bite at a time, as well as the anarchic nature of open source development, but from the point of view of the overall health of the Linux/Free operating system desktop it is rather myopic, and worsens the Linux desktop ABI/API fragmentation problem that is the main thing that holds it back.
They have the (quite reasonable) view that in time all the missing bits will be restored, but my main concern is that because Wayland was architected without any consideration CM, that they may have made fundamental architectural decisions that make it impractical or even impossible.
I’m a bit at a loss. Only one developer on the Wayland dev list seemed color savvy (Niels Ole Salscheider), and a vocal number of them seem overly sure they know everything they need to know about computer graphics.
One of the problems is that in programming, social form follows technical function (and visa-versa), and Wayland changes the boundaries within the application GUI stack. Prior to Wayland, applications could use the X11 API directly, although most use a GUI library front end for ease and speed of development. With Wayland, it’s not really practical for an application to be written directly to Wayland, at least not without some sort of rendering library to do the hard work. So even more than before, application writers will be dealing with the vendor of GUI library (Qt, GTK etc.), and it is they who deal with Wayland. So the process of providing user feedback is quite long: End User → Application Developers → GUI toolkit Developers → Wayland developers. Since the GUI toolkit developers have made the switch of their back end from X11 to Wayland as transparent as possible, many End Users and Application Developers are not yet aware of any problems, at least until they fire up a new Linux distro running Wayaland + XWayland, and notice their CM or CM tools has stopped working. (And if you have one screen and are using Gnome/colord, you may not notice until you want to calibrate & profile). And given this apparently smooth changeover, it’s understandable that most Wayland developers don’t think there’s a problem.
I’m still working my way through it all. Of course to really understand it means compiling it and running it, and then poking around its insides, a project I can’t find time for at the moment
There is a way to tell all of those people about color. We need a color talk an give it at all FOSS conferences. You need to teach users and developers why they want color management. It is not only important for graphics designers or developers, also for all the rest that UIs look the way they have been designed. And if you enjoy great photos you want that it look correct and not green, magenta or blueish.
Create a talk and visit conferences, then the pressure on wayland to implement it will get bigger.
This can’t be stressed enough. If the benefits aren’t presented from the perspective of those who one wants to convince, it’ll be a very difficult if not impossible challenge. I’m sure that simply pushing the rationale from this side’s perspective alone would soon degenerate to an unproductive pissing match between head-strong parties, each claiming that “mine’s bigger than yours” or “my logic is more logical than yours” - if they even agree to listen to each other in the first place.
Message
Develop a condensed bullet list of the issues. Support each bullet with detailed rationale.
Target
Who are the key Players and Decision Makers? Identify them. Lurk on the Developer’s List. Get help from the Inside: Attempt to develop and cultivate a relationship with anyone who has shown willingness to discuss the issues.
Empathy
Scour their posts/blogs to see what makes them tick. Tweak the general message in order to develop customised versions which hits the “sweet spot” of the identified parties.
Presentation
Diplomatically participate on the Developer’s list to feed only non-confrontational pre-defined boiler-plate “suggestions” whenever possible.
Identify conferences, submit abstracts, present.
Allies
Make the public aware of the issues so that they too become advocates for the Cause.
Identify the leading media Influencers. Help them to do their jobs by providing them with content. Such content should consist of a non-confrontational summary of the Message.
.
. IMPLEMENTATION
I just spent quality time with this post, and I think it describes the ‘best bet’ path. The important thing is to get a Wayland change committed that incorporates communication of a color profile along with the pixel map, for the ‘surface’. And yes, if they already grok such communication in the DPI parameters, that’s prior thinking in that same paradigm.
I think educating the Wayland team on the head-hurter that is color management is needed, but primarily from a use-case perspective, as it applies to the end-user communities. The more compelling ‘education’ will be to present patches that incorporate CM in a way that doesn’t glom up their fundamental thinking about how compositing window management architectures should be implemented.
Education, intermingling and collaboration are all good ideas but these things might not change people’s minds. Speaking from experience (unfortunately, a lifelong issue of mine), the problem with people who don’t listen or care, or who see you as irrelevant or inferior, is that they don’t think you have anything to contribute. Worse, they might consider you a troublemaker. The only way around that is to forge ahead yourself. Don’t mind playing the fooling. Do stuff that you know is healthy regardless of what they say or do.
The concrete thing to do is to build stuff that works like extensions and shims. Might not be ideal but it is a start. Perhaps, once these things mature, colour management might be considered important by the Wayland folks. Perhaps not. Perhaps they might even be implemented. Perhaps not. But at the very least, we creators would have something to use. Also, there is still the possibility of forking Wayland, although I am sure that would be a big responsibility.
That’s my tentative conclusion as well. The thing is that implementing the analogous thing in CM is more complex - rather than the Compositor simply scaling pixels (something they have been doing a long time for “Chrome”), it needs to do colorspace conversions. Added complexity is that current CMMs (like lcms) don’t have the tweaks needed for mixing HDR and normal displays (I’m guessing some extra ‘intent’ params will be needed.) Successful heuristics for HiDPI mightn’t have analogs for CM. (If the surface is being displayed on more than one output the client can simply render to the output with the highest DPI and the compositor will scale the pixels down to other outputs. For CM, which is your ‘high quality’ output and which ones are not ? i.e. a mirrored display setup etc.) And it’s one thing to have an extension protocol spec., it’s quite another to get implementations to follow, especially when they are possibly using a lot of different tech. for compositing, none of which currently supports CM color conversions out-of-the-box.
I suspect this is the main technical issue. I’m gathering that the whole technical premise of the Wayland (and possibly other) compositors is that you can copy application RGB to any output with gay abandon, when we know that in reality this is not possible without wrecking color. Adding in the necessary color conversions will really crimp the style of how the compositor currently operates.
Yep - that’s the open source way. You can’t get “other” people to do stuff, you have to do it if you want it done. Either that or they have to suddenly be inspired to help
I suppose that we can’t all be optimists! Yet if we were all pessimists, our ancestors would never have embraced the world beyond the maw of their caves. Balance is good.
Regardless of how daunting it may seem, this is not a futile adventure, predestined to fail. Rather, it’s pretty basic stuff which happens all of the time. For example, think of the folks out in the streets tasked with winning contracts for the products or services offered by your organisation. If they shirked away from similar obstacles whenever anybody vehemently says “no!”, you and your colleagues (and families) would soon go hungry. But you’re not hungry, are you? That’s because these competent professionals, acting on your behalf, are constantly overcoming objections and converting Luddites by employing proven basic techniques in a properly managed manner.
It’s easy. Anybody can do it. You just have to put aside fear brought on by baseless preconceptions. And then get on with it…
Sorry, they will be examples regarded as of little relevance, since they primarily involve printing, and we’re talking about displays. A better start is something similar to the Android O Color Management talk referred to previously.
Er… I still believe that my suggestions are valid
I did not intend someone to bring “printed products” – but in the work flows of today’s ad agencies, design shops &c there are lots of displays. And if it isn’t right there…
I just created an account here to join the discussion. I’m personally coming more from the wayland/hardware side of things and while I think I have a reasonable understanding of color I do not know about the practical side of color management at all.
Bringing good color management to wayland is a great goal to have and I’m sure every party is interested in getting there.
Having said that I also think that especially @gwgill is pushing really hard to maintain the X11 model for color management and isn’t open to work in the constrains that wayland has. A more open mind for new ideas and new workflows could really help.
Another gripe I have with this discussion is the focus on solutions and implementation rather than use-cases and concepts. In particular I’m never sure what part of color management would not work (well) with a particular concept.
I think it would be useful if further discussion would ignore performance, what kind of hardware is used to perform a calculation and other technical details and instead focus on what kind of pipeline (what data is required and which calculations have to be performed at what stage) would be required.
So, with having said all of that I have a few questions:
What exactly is required for a color to be shown correctly? My current understanding is that the output image has to be in the same color space as the display (which includes primaries, white point and EOTF) and the (measured) color correction function (or LUT or whatever, let’s just simplify and call everything a function) has to be applied to every pixel. Is this correct?
In particular which workflows/use-cases do not work with the current proposal in the wayland-devel mailing list?
Furthermore a few opinions:
You should ignore color measurement for wayland. Realistically it requires full control over the display you want to measure. Wayland (and Xorg) can lease out full access to a display temporarily (this is mainly used by VR) and a color measurement application would then talk to the display using KMS. This problem doesn’t have to involve wayland at all.
A color management system will be required in the compositor, period. Arguing that it is “too complex” or “to big” is not helping and not really true given that we already have huge GUI toolkits, OpenGL drivers which include a JIT compiler and often even javascript runtimes in those processes.
Compromising on security is a no-go. Someone mentioned that wayland has had a security/permission/privilege protocol proposed but that never gained traction. Most of the time it made sense to not use wayland at all and have a dbus interface (like for screen recording, monitor configuration, etc) instead. I suspect that users will configure color management like that, too (iirc colord already does it that way).
I hope this doesn’t come across too combative. I’m just trying to learn more about color management and at the same time try to convey the viewpoint and constrains of the wayland side.
I’m a Krita developer that actually implements HDR color management support for Krita on Windows atm [0]. Just wanted to add my points:
The idea of offloading color management from the compositor to the apps is surely a failing approach. Windows does it and it is the reason why it is hated by most of the graphics people. The pain starts as soon as you buy a wide-gamut display: the system GUI and all non-managed apps will start looking awful. Just google “windows wide gamut display colors” and see tons of questions without answers: Windows just doesn’t support color management. Only a few clever graphical apps will show you a correct color. All other apps will have acid-looking colors. And that is the reason why Apple has become a de-facto standard for people in graphics (I’m not a fan of Apple myself, though).
HDR is another reason why compositor must do color management. On a single screen, there might be different types of surfaces, like legacy SDR window might be painted next to an HDR video surface. If your display is HDR capable, then SDR surface should be converted from sRGB to Rec2020-PQ (which is the color space of the display); if the display is not HDR capable, then, in reverse, HDR surface should be converted to sRGB. This conversion cannot be done by the app itself, just because one app cannot influence the buffers of other apps. It can only be managed by the compositor.
And the most tricky thing: applying display (ICC) profiling curves should happen after(!) this conversion! When an app writes to a Rec2020-PQ buffer, it has no idea about the fact that this buffer will be converted to a non-HDR sRGB-like (or AdobeRGB-like) display, so it just physically cannot prepare the data accordingly.
The last point is exactly the thing that Windows’ DXGI “color management” API fails to do. To do proper color management on Windows, the developer should basically disable this API by marking a buffer as “sRGB” (even though the display is wide gamut) and prepare all the data himself.
[technical comment on the original requirements proposal from @dutch_wolf] The compositing should not happen in Rec2020-linear always. Most of the time it is useless (and eats resources). Composition should happen in the actual color space of the display in use. If one uses an sRGB-8bit display, there is no reason in doing the composition in Rec2020. Just compose it in sRGB-8bit or (if really needed for some effect reasons) in Rec709-linear-16 bit.
thanks for taking the time to show up here. as a user space program developer, let me add my two cents which might partly answer your questions.
i’m very happy to ignore colour management implementation details, hardware or implementation constraints. in fact if that “just worked” so i wouldn’t have to worry about it that would be great. some of the more annoying details have been mentioned above:
multi screen environments and managing profiles per screen (i don’t want to do it if the compositor can handle it)
loading icc profiles, supporting other sources of colour space conversions, etc
about your questions:
re: 1. we can provide profiled buffers in tristimulus. let’s say that is a floating point buffer with unbounded CIE XYZ (illuminant E whitepoint) data. to convert that into display RGB you don’t need a lut. going from tristimulus to display RGB can be expressed in a matrix, this side of things is linear (camera RGB to XYZ not so much, because the original input is spectral and not three dimensional). you may need a shaper curve after the matrix, but yeah, this is by no means complex or difficult, or “too big”. it does become a lut if you do perceptual gamut mapping to compress wide gamut colours to sRGB output for instance. for my part i’m not a huge fan of this, i like correct colours.
all you need to start managing colour is any buffer in any precisely defined tristimulus space. the linear ones are all a matrix multiply away from each other, given that the input is not clamped (one of the reasons i’d prefer a way to hand over a floating point buffer). if the colour space is a tag on the input buffer, or everything has to be rec2020 or XYZ… i’ll leave that up to you. even linear rec709 (sRGB without tone curve) is fine if you don’t clamp at [0,1].
re 2. can’t speak to that, have to read the thread first.
Replying to your call for usecases, I can tell what Krita would actually expect from a compositor:
Ideal case:
Display color space can be: sRGB-like, AdobeRGB-like, p2020-pq. First two can be defined by an ICC, the last one should use so special treatment.
The entire window is painted on an sRGB-8bit surface that is converted by the compositor into the color space of the display.
In the middle of the window there is an overlay pass-though surface for the canvas. This canvas is explicitly color-corrected into the display color space by Krita itself. Compositor just passes it through to the display.
We don’t need any opacity support for this pass-though surface. So there is no need to delinearize the colors for compositing. It can be blit directly in 8/10 bits.
The compositor should provide Krita information about the actual color space of the display (ICC or a tag for PQ-style color spaces, not representable in ICC terms), so we could prepare colors for such pass-though surfaces.
What we do right now on Windows for HDR:
We create a p2020-pq surface for the entire window.
We patch Qt to render its sRGB GUI elements correctly on this surface.
We draw the canvas directly on the window surface.
Since Windows does not support any color management, this approach is not properly color managed for wide-gamut displays. If the user has a wide-gamut, but not HDR display, our p2020-pq will not be able to treat it. Windows’ compositor will automatically convert it into sRGB-8bit and will pass this data directly to the display, basically, ruining the colors.
For an HDR case teh topic of “color management” is a bit fuzzy. Theoretically, display manufacturers should treat is somehow in hardware, but I don’t know any real details about that.
What we do right now on Windows for non-HDR:
We create an 8-bit sRGB surface for the entire window. Basically, it disables any so-called “color management” from Windows’ side.
We have a profile for the display, so we just color-correct canvas data before passing it to the surface.
Technically, it means that our canvas is color managed, but the GUI is not. GUI will still have acid-looking colors on wide-gamut displays.
