Just saw the following on the wayland developers mailinglist HDR support in Wayland/Weston and that reminded me that I always wanted to make a topic here to discuss color management on wayland, especially since the proposal from the wayland devs itself[1] is lacking in a few aspects, namely
It is missing profiling/calibrating support which in contrast to what the wayland dev thinks is not an orthogonal problem (IMHO)
Not all applications actually use ICC (anything that uses OCIO for example) so not everything can use devicelink profiles to tell the compositor what to do
The ICC spec is rather large and complex so libraries/programs that can read devicelink profiles suddenly have a larger attack surface, this is especially a problem for compositors since those are the root of trust in the wayland world
For this and other reasons I think it would be mandatory that people from the image editing/creation software are involved in this discussion. To frame this discussion I will try to put down what I think such a protocol should look like.
Proposal
There are 4 general cases here:
legacy/sRGB
Basic applications (browsers, basic viewers, video players, games, basically anything that a normal user would use to view images/videos)
Advanced applications (image/video/3d editors, advanced viewers, so anything that needs more control over how the colors are actually rendered)
Profiling/Calibration
In my proposal the compositor internally should composite in linear/scene referred color space with the rec.2020 primaries, legacy/sRGB would be converted to this color space, while basic applications would render directly to this color space. Then the compositor would use a shader to render this composite down to the screen color space and in this space would composite in the advanced applications. In this case profiling can just use the advanced application route and only for the calibration we need to add a way to change the calibration LUT[2] from the wayland protocol.
I hope this made sense since for me it is currently late in the evening so will see if I can add a diagram tomorrow.
[1] [RFC wayland-protocols v2 0/1] Color management protocol
[2] With is usually loaded into the graphics card LUT, but in advanced monitors could also be in the monitor. Theoretically this could also be a shader that would be applied at the end.
As much as I love to see any improvements of colour management and HDR display support in Wayland I think here is the wrong place. So please move all the discussion/communication to the Wayland mailinglist.
I disagree this is the wrong place since a) the only developer in the opensource graphics community that even follows the wayland dev list (that I am aware of) is @gwgill and b) since we first need the requirements a wider discussion among people aware what color management is, is important; after that the real protocol can be hashed out on the wayland dev list.
On top of that this post is (also) meant to raise awareness of the issue so that more people will join the discussion, if that ends up being here on the wayland devel mailing list is less of a concern.
As much as I love to see any improvements of colour management and HDR display support in Wayland I think here is the wrong place. So please move all the discussion/communication to the Wayland mailinglist.
Really ? Wow! Can you explain why you think it is the wrong place ?
Iāll explain why I think it is critical that there be discussion about Wayland and Color Management on a forum exactly like Pixls.
As best I can tell (from the long and unsatisfactory thread I pursed on the Wayland Developers list some little time ago), the core issue with Wayland and Color is social, not technical:
none of the key developers have the faintest idea what Color or Color Management is, and (worse):
they have no idea why it would be at all important.
The takeaway from this is that, as and when the free software/Linux world migrates from X11 to Wayland, all the applications that get discussed on Pixls will be in trouble, because there are no tools to setup Color Management on Wayland, and no facilities for applications to employ Color Management on Wayland, and there seem to be very serous technical and social obstacles in the way of ever doing so. In order to get any traction on this, the key Wayland developers need to be convinced to revisit some of their most dearly held goals and past technical decisions. This is not going to be easy, but the very first step is to convince them that they were missing an absolutely must-have, mission critical feature from their list of goals for Wayland: support for modern Color Management. Whatās needed to convince them, is all the developers and users of color critical applications telling them that Color Managemet is vitally important, and explaining why, and what the stakes are. Currently the key Wayland developers think that Iām joking when I tell them that users will be forced to stick with X11 or (gasp) even switch to MSWindows or Apple if Color Management is not possible on Wayland. Either that or they dismiss such users as ānot their target audienceā.
So I think it is very much worthwhile having discussions here, to first raise awareness about the looming problems for graphical applications on Linux, and then to develop some ideas about how to get Wayland developers to change their priorities, and get them focused on facilitating Color Management as a vitally important goal for their project.
@gwgill for a user such as myself that can offer nothing in terms of technical contributions/code, but can offer my usecase, what do you think is the best path forward to convince wayland developers that color is important? e.g., what should I do to help?
I sustain. Linux intrinsically sucks at color management, because itās made by geeks for geeks. Where are the artists ? On Mac. Is that going to change anytime soon ? ā¦
You know, ideally there would be a co-hosted X.Org Developerās Conference and Libre Graphics Meeting, and this would provide the opportunity for graphics application writers and the Open Source graphics system providers to talk to each other. Something like a āColor - why it is so importantā keynote presentation would be a great way of getting in front of the Wayland developers. I could imagine someone like Chris Murphy presenting an introduction to color management, what it is and why it is important. Someone like (say) Florian Hoch giving an outline of how Color Management is setup on current (X11) systems, and one of the developers of an open source application that implements color management outlining how application use the CM setup and implement CM at an application level, and in particular giving an outline of how multi-monitor CM works.
Quite how something like this could be arranged and such a keynote sponsored, I have no idea.
I agree something like that would be the best thing but since I donāt know how to arrange that either I think the best option is to hash out a set of requirements that all of us can get behind and present that to the wayland devs. Such a list would roughly look like this I think
Graphics applications need to know which monitor they are running on (or at least the color space of said monitor
Need access to the video card LUT (or equivalent since we might decide to not touch that anymore and use shaders in stead to for example have a per HW plane LUT)
If a graphics application pushes pixels those pixels shouldnāt be touched/be altered (color wise) by the compositor at all
I think Wayland should do some sort of unified color management:
applications send pixels buffers with a colorspace signature embedded, but not color-corrected,
the compositor does on-demand color-corrections from buffer colorspace to display colorspace only for visible frames (saves a lot of perf), possibly on GPU (for now, AFAIK, graphic libs are not multi-threaded and super slow).
That way, your whole desktop is color-managed with the same predictable algo.
The problems of on-app color management are:
most devs are color-clueless
in a multi-screen setup, when you start an app on one screen and move it on another one after, what is the actual profile used ? (i.e. does the app know itās not on the same display anymore ? are the pixel buffers refreshed ?)
are you even sure the original display profile was the right one in this multi-display setup ?
there are zillions of funky DE and counting. You canāt ask to app devs to maintain them all.
Although I agree that we will want this in the near future for basic stuff (large gamut monitors are becoming more mainstream) but it wonāt work for more advanced stuff (profiling among other things) you want to have full control over how the colors get mapped to the output, so for those cases and alternative must be provided.
Doubly so for compositor creators and the wayland devs (just see the mailinglist discussion @gwgill had with them)
In X11 we could just ask the X11 server about this and do the right thingā¢, currently this is not possible under wayland so we need a protocol.
That why you need external profiling in calibration and protocols to communicate the right settings
Which is why we need a standardized protocol to talk to the DE/compositor so the DE just implements the protocol and we just talk it, no need to implement something different for each DE/compositor/window manager
On top of that most/all applications currently caring about color management already implement it in app (see for example this phoronix message from Boudewijn Rempt maintaner of Krita ). They do that most of the time by just linking to LCMS and/or OCIO and let all the colormanagement things be done by those libraries.
I personally do not like such approach, for at least one good reason: if one has to convert pixels from a linear wide-gamut working colorspace (like Rec.2020 or ACEScg) to the display, youād want to do that on floating-point values, hence before sending pixels to the compositor. Unless Wayland supports floating-point buffersā¦
Another is that in general you want to have control over rendering intent[1] (yes even in an OCIO based app although there it is often baked into the config so not user selectable)
EDIT: I do believe wayland can support FP buffers not sure of anything currently takes advantage of that tough
[1] In ICC terms, if you want Relative/Absolute Colirmeteric(sp?), Perceptual or Saturation. Also maybe black point compensation
Drew a daigram with how I think it should work (roughly), double arrows means the compositor needs to communicate information back to the application, single arrow is just image data. @gwgill does this make sense to you or am I missing something?
Iād be happy to try and facilitate this, but LGM this year is only 4 months away. Would that be enough time to get the necessary people committed to coming?
I could at least get a āBirds or a Featherā meeting set up. Itās a few hours of dedicated space for discussion of topics separate from the main presentation track.
I donāt know about funding for attendance though.
Based on the discussion here, I personally think only one of the two will make it happen:
A āguerillaā dev makes it into the Wayland cabal and produces patches that are accepted. Recent example: @anon41087856 and filmic, pushing forward the understanding and acceptance of scene-referredā¦
Some time passes, and it becomes evident that Wayland-based distros donāt gain āmarket shareā (for immediate lack of a better term), and the reason is realized that image applications donāt work right. Might be an article somewhere interviewing a Pixar exec: āWe wanted to use %$#@-buntu, but Wayland does not support our workflow toolsā
Yes, a bit cynical, but Iām growing old and irritatedā¦
Iām afraid that kind of āfull controlā is rather a chain of trust between all the CM layers in place now in Xorg. There are too many layers, no quality control. App devs wire their pixels buffers to some magic CM lib and expect to get magically cleaned crap at the end. The color corrections done by the CM or the video card are kind of opaque. I say put all that in one place, MacOS style, stop fragmenting and layering crap.
Sure, but that makes one team and one project to educate instead of thousands of them.
??? You can have calibration, but as long as you canāt assert what profile is in use for that particular app, and double-check the color pipe, you live in faith. Thatās what I hate about ICC wizardry on Linux : as soon as people see ICCs, they are happy because their app is color-managed somehow. Nobody worries about what they actually see on screen. And when you want to assert it, you discover opaque stuff and undocumented features.
And most of the time, they suck (cc Firefox). Also LCMS performance is a disgrace: it does not do multithreading or SIMD, let alone GPU offloading. So much for FPS. Is that 2019 or what ? Besides, if I open the same picture in darktable, Gnome viewer and Firefox, I see 3 different results. Which one is the right ? I donāt knowā¦ What quality insurance tools do I have to answer this question ? None ! Trust the devs ! They know what they are doing.
I think 2019 is not the time to be clever about color management. Figure out what Apple does, and do the same. These guys have invented GUIs, their OS is the de-facto standard for the graphic industry, they have been doing it for a long time and are good at it.
If apps want to be clever, they can always send pixel buffers with a ādo-not-touchā colorspace tag.
Not rely on apps to perform the color management on single-threaded CPU tasks. Centralize/build-in the CM into Wayland, close to the GPU, so color transforms could be offloaded directly on it without fancy libs deprecated every once in a while.
