Wayland color management

[gwgill (Graeme W. Gill)]

@rasimo

I totally get if you or anyone else gets fed up entertaining my curiosity, given I am only trying to understand the current state of things and my only contribution (at best) is reproducing that understanding in a way that’s (hopefully, maybe?) accessible to someone that’s not followed all of the past discussions on this. I am still continuing hoping to keep the constructive discourse going, but obviously am not demanding for anything (how could I?).

Sorry - I don’t currently have a lot of time (or energy) to replay past discussions on this and other lists at any length. One of the key problems it seems, is that a lot of the statements relating to this subject have little or no meaning to someone lacking the requisite life experience.

I still feel like I am missing something here: For all I see the wayland color management proposal fulfills your requirements (or they are explicitly out-of-scope for the proposal) and the summary of your protocol sketch pretty much matches wayland’s.

Then it seems to me that you are not reading what I wrote

Key point is that you can’t make a problem go away by saying they’re out of scope.

Anyone who has actually used a color managed system in anger knows that being able to color profile it and keep it in calibration is essential. It is not an option that can be waved away. The fact that all existing desktop application environments support this (and hence all existing applications that depend on color management expect this, and users expect this) is a hint as to the reality of the situation.

And while it’s perfectly reasonable to expect that implementation of a new facility will be incremental, anyone who has architected and participated in standardizing moderately complex systems knows the issues that result by not looking far enough ahead when architecting something. Too often you end up with a mess when you realize that early assumptions and simplifications are wrong, and that foundations need to be ripped up and redone, and/or horrible hacks have to be lived with because of momentum and sunk cost. If this was brand new territory, and everyone was on a learning curve, then a stumbling path to “perfection” is how it has to be, but this is not the case.

The requirements about being able to set the display color profile is not covered by wayland’s protocol. It explicitly states that it does not cover that process, but assumes display profiles are already in place. To me that doesn’t mean there’s no way to set display profiles, just that this proposal doesn’t define how that happens. Basically an implementation is free to handle that however they want respectively that may get standardized in a separate proposal.

And that is a big problem. You haven’t got a modern Color Management system if there is no standard way of doing Color Management things. (Converting pixels from one colorspace to another is not a Color Management system.) Saying “do it some other way” is not going to happen. As the author of the only profiling system readily available on Linux, I’m 100% sure that I’m never going to hack into my code, distro or compositor specific code to load profiles or attempt calibration and profiling, even if there were a way to do so. Just like any other application author, I want a well thought out standard API to do these things, so that it can be written once and will work on all Wayland based systems. I get that on all the existing operating systems I support.

As to your sketch: 1) is not covered, at least I didn’t see any provision in wayland for a compositor not doing color management. However that is basically the current, undesired “baseline” behaviour, right?

Wrong. I could digress and point out that the major difference between Wayland and X11 is that X11 does the rendering on an applications behalf - that is the way it worked well as a remote display protocol, and that in contrast Wayland assigns that responsibility to the Application, instead providing a smart “dumb frame buffer” to the application. So logically it would follow that the Application should have the responsibility for Color Managing the pixels, since that is part and parcel of the rendering process !

But most Wayland Color Management proposals have included a Compositor pixel color transform capability for two reasons:

1. It’s highly desirable in terms of making it easy for non color sensitive applications to be written that will work across the modern range of wide gamut and/or HDR etc. displays, without the authors having to be experts on, and clutter their code with extensive color management.

2. It’s forced by the wrong assumptions made in the initial architecture of Wayland, that pixels on different displays are interchangeable, and that the application can be kept ignorant of which display they are rendering to. This was the very basis of how Wayland handles compositing, so it’s not easy to fix/work around. Wayland can move an applications window around between displays without the applications involvement, so it is a problem when the application is doing/has to do the pixel color conversion.

But let’s come back to why the application has to be able to manage pixel color conversion itself. And once again, this is something that becomes evident when you have written color managed applications, such as a Postscript RIP, or soft proofing application etc., but may not mean anything to you if you haven’t:

1. The format of source color specifications is open ended. There are many standard ways of specifying a colorspace, but there are an infinite number of ways that aren’t standardized, and people keep inventing more. As I have previously mentioned, for a taste of this, go read the Postscript or PDF specifications related to color. Even something like ICC profiles allows for up to 15 channels of color information, so to support the subset of ICC based color specifications within Wayland would mean that Wayland has to have a raster format capable of handling floating point rasters with up to 15 channels, named color “rasters” etc.

2. The details of converting from one colorspace to another is open ended. There are some commonly used ways of doing this conversion, but when it comes to color sensitive applications, it can be very application specific, and people are constantly inventing new ways to do this. No clever technical tricks can hide this problem. In general, color conversions involve dealing with gamut differences between colorspaces, and the conversion needs to know the details of the source and destination colorspaces to setup specific flavors of conversions.

So a modern Color Management architecture has to allow applications to do their own pixel color conversion if it is to be comparable to existing systems. Making this possible while allowing Wayland to do the Compositing is one of the challenges. If Wayland had been architected with Color Management in mind, a lot of problems may have been avoided. (And Color Management was a well understood thing at the point in time that Wayland was created.)

On 2. My understanding is that the compositor needs to be able to convert input of various sources into a common working spaces for blending, thus it needs to be able to handle the input color space and the protocol lists a few of those that a compositor should be able to handle.

See above. You can’t assume that the Compositor can handle the pixel color conversion except in very carefuly controlled situations. And once source colors have been converted to a particular displays colorspace they can be blended as they always have been. It does offer the possibility of improving the blending operation by allowing it to compose in a linear light mixing space - knowledge of the EOTF would allow simple per channel curves to convert to a linear light mixing space and back, improving transparency behavior and anti-aliasing. I guess that may imply a change to Wayland implementations, if they are currently doing blending before windowing into particular displays, but then it’s hard to take advantage of display hardware acceleration if they were doing that.

For those who are a bit lost in the nuances of color, a strong parallel in the Wayland world is how it is coming to grips with displays with widely different DPI’s. When the application is doing the rendering, trying to hide which display the application is rendering for becomes difficult when different displays have different DPI’s. So some of the “pure vision” that Wayland started out with, about how applications didn’t have to know what screen they were rendering to, and “every pixel is perfect” have had to give way to reality. Many of the same compromises are needed for Color Management. People don’t need to measure and re-measure their screens DPI though, since it doesn’t drift or change, and it’s relatively straightforward to measure.