Wayland color management

You see, the “elephant in the room” is that X is basically only on life support. Their own project leader says this.

So, Wayland is our path to the future. But it doesn’t sound like they care about supporting us.

No matter the history, no matter the stakes involved, an “us-vs-them”-rhetoric/mentality doesn’t help anyone. It just pushes people away and doesn’t lead to solutions. Wayland is open-source, so you have all the freedoms: Typical options are engaging current efforts in good faith and trying to help that reach a state that is usable to you, or alternatively propose something that you deem good yourself (proposing as in doing, not just suggesting what someone else could be doing, that’s the previous option). Telling people working on it that “they don’t care about us” won’t help. From personal experience maintaining an open-source project, this is the fastest way for me to stop listening - because I do care and there’s plenty of people to help that believe me that I do care.

1 Like

If someone flat-out rejects a proposal, it sounds like a likely waste of time to implement the proposal to have the implementation rejected as well.

2 Likes

Up-front: I have not followed this, I have zero clue and don’t care if any of the statements are true. Assuming for the argument they are:

Again, two options: 1. Try to find better ways to explain yourself, to convince the people in charge. 2. Do it on your own fork, call it wayland-color-managed. As that then has your proper color management, and wayland doesn’t, and they are otherwise feature equivalent, that’s going to be adopted.
Usually 1. is far superior, as forking has a huge cost. And also in most cases, the problem is really communication and/or getting the right supporters. You, understanding your own proposal, see why it’s superiour. Others not understanding apparently need a different explanation. Or if you are truly sure you exhausted all angles you could take at explaining it, rally support. Either finding a few key people or a majority of people involved in windowing systems, graphic design software, … to support your proposal, they might reconsider their position.

So instead of schooling people about how they should behave with the Wayland team, perhaps with all your experience you could, after learning about the issue, act as a mediator.
I fear that the Wayland team suffers from a profound NIH syndrome.

1 Like

No. Just browse the dev mailing list of Wayland.

Wayland has been started to fix the security issues of Xorg, mostly harming people connecting to a graphic session remotely through a network. It has been designed to be a hardened graphic server for geeks.

The last time I had to connect remotely to a graphic server was in engineering school for an assignment. The last time I had to calibrate my screen and install a new color profile was last week. Says the guy pissing C code for a living.

For years, they have repeated they don’t care about color management, that color management is a threat to their perfect little world of hardcore security, and refused input and requests from the senior color management devs in the opensource world. Well, nothing is more secure than a product that doesn’t work, is it ?

You don’t try to reason with people who fail to see that any graphic server will ultimately display colors, and that color management is… well not an option but a core feature. You call them stupid again and again until they break. Because stupid is what it is, and when devs are not paid, there is no way to fire them, so harassment it is. The matter is pressing enough as Wayland is now the default graphic server for most Linux distros, and has put the whole graphic community in deep shit, even more now than Xorg is tagged abandonware.

Xorg will remain the only sane option for a long time, for everyone using a display to display colors. But I fear it will soon be deprecated in many distros. And that is unacceptable.

Graphic artists rely on color management to do their job. Users rely on color management to see what the graphic artists intended for them to see. The sRGB standard comes from 1999. Everybody knows that since at least 1999, there are no excuses. Wayland has been actively ignoring that.

I don’t care if it’s a hobby project from unpaid devs: you do something, you do it right or you do nothing and let someone competent do it. Because, again, that hobby project is now the default server in many distros.

So, kudos to Pekka for bringing some sense back in that project, but color management being an after-thought in a design that is already a decade old, there are good chances that’s it’s already too late.

7 Likes

I am looking forward to testing that C-filmic-window-server prototype :stuck_out_tongue:

I agree on many many things in your post, but this…

…is of-course going to devaluate every sound argument that you make. You just pulverized any chance of getting your (actually reasonable) arguments heard. Which is a shame, because I do think your argument is logical and makes sense.

I also don’t think that @rasimo was schooling people. He merely pointed out that flingig feces is not getting anyone closer to their goal. A captain obvious statement? Not really if harassment is seen as valuable tool.

love and peace! :heart:

2 Likes

And, to be very clear about the non-sense that is wayland color management, “color management” is only a big word to say “multiply your RGB pixels by a 3×3 matrix and apply a 3D LUT”.

It’s really nothing difficult. Getting the values of that 3×3 matrix out of an ICC profile is actually more code, but whatever CMS can be used as a parser for that.

I live in a world of results, not a world of wishes nor kindness. Whatever it takes to get results should be done. There are people out there that rely on the results coming out of their computer for a living, not everyone using a computer is a 50-something IT guy turning into a vocational artist on week-ends, who can tolerate a lot as long as it’s “libre”.

2 Likes

Sorry I have to but in here but you are wrong, yes what you say is true on an application level but not on a window compositor level where you might have multiple applications all using different color spaces, some of which might need to directly render their color to the screens while others might not care.

For example, lets say we have at least three applications one of which is in sRGB, on is in adobe RGB but is fine with very slightly off rendering (e.g. a simple viewer viewing an image) and the third is a graphic application rendering directly to the screen (e.g. something trying to show a printproof of a CMYK file). How to best handle that provided alpha blending, and multiple outputs might be involved? Well for the first two it is simple just render them to some nice scene linear intermediate do the blending there and then output to screen (of course after the scene linear to screen transform), the third app will need some more effort, sure everything that doesn’t need alpha blending can be blit directly over the previous composite, and we can take advantage of that colors in the alpha blended areas are probably not going to be accurate anyway so go through the scene linear stuff as well.

As you can see from that I need a lot more text to explain myself that it is not just a a 3x3 matrix + 3dlut, if that all it was it would have been a lot easier. Also you forgot CMYK and the fact that ICC can (theoretically) handle up to 16 channels of color space, do you want to deal with buffers of 4+ channels while also needing to blend and blit multiple different color spaces? (oh sure you can convert all of that to a nice scene linear and do it all there but then applications that need to be able to render directly to the screen won’t be able to do so anymore)

So please stop calling people stupid when you clear don’t understand the scope of the thing they are trying to do. Yes I agree that Wayland should have this from the start and yes I also agree that it took way to long and way to much effort to get the Wayland devs to understand all this but good luck moving forward with your attitude.

2 Likes

Soooo you just tag your window pixel buffer with a color space ?

  1. If the app is already color-managed internally and outputs display RGB, you tag it with display RGB, and your server-side color management becomes a no-op.
  2. If the app has non-color-managed output but writes its GUI colors in some space (say DCI-P3), you tag it with that space and the server-side color management will deal with the conversion to display space.
  3. If the app doesn’t care and doesn’t provide a tag, the server will tag it sRGB and deal with it as in 2.

Anyway, if you want to do alpha compositing on UI windows, you will have to remove all the OETF of the RGB buffers before the blending, convert all buffers to the same RGB space, apply your Porter-Duff alpha in linear, and redo the OETF after. But nobody does that, so the alpha transparencies produce fringes that nobody sees because guys lack color training and see no problem compositing sRGB buffers with display ones.

You’re making up problems. Nobody in his right mind uses CMYK to paint apps and who is using 16 channels again ? Get something basic to work, 3×3 matrix and 3D LUTs cover 99% of the real-life use cases, and having only them to work would be a huge improvement. Nobody said you had to support the whole range of ICC madness trying to be future-proof by making shit up.

I have got results, me.

1 Like

I hope you do see the mountain of problems associated with this statement.

1 Like

While you clearly don’t see the part saying “people relying on accurate colors for a job upon which their living depends”. We are past pet projets and hobby-coding. Especially in the current context of artists losing assignments and income from all the confinements and travel restrictions. There are people out there who don’t play photographer, or 3D designer, or whatever. It’s their job, it’s a tough one, and they need it. Right now, we are are forcing them to buy expensive Mac that can’t be repaired and trap them into a golden jail. That’s a lot of money for overpriced goods. All that because Linux is color-disabled.

There is a community of creative people that struggle to join both ends right now. Opensource could have been an opportunity to help them by reducing some costs. But no. We failed.

5 Likes

The above is exactly what we are trying to do but it does require communication since from the app (the color tag) and to the app (the display profile). With some extra caveats to deal with output to multiple outputs.

And wouldn’t be nice and correct if we could get rid of those fringes?

You do know that without my interference this was one of the plans for Wayland, right? (using ICC link profiles), and sure you don’t paint in CMYK but applications like Scribus (and other desktop publishing) would still like it to work. As pointed out you definitely don’t want to do that in your compositor, so an app needs to go directly to screen internally (and thus tag with the display profile)

As do I since without this thread and some prodding on the mailinglist we would be much further behind. And if this works we will have color management on par with MacOS. Sure you do amazing stuff for Darktable and I thank you for that but for the rest I just see you shouting in the void without actually contributing anything

2 Likes

colord does that already, you don’t need to go to screen, you need to fetch the profile and run the CYMK conversion to RGB internally.

I promise when darktable will stop taking an average of 50/h week on my time, I will start something else. Meanwhile, I’ll refrain from scattering.

1 Like

Yes but the way colord works won’t work on Wayland, also in the future we will probably have multiple profiles for when the screen switches between modes (e.g. SDR to HDR or vice-versa but not limited to that), with Wayland events that is pretty easy to support. AFAICT not so much with colord.

All in all colord is nice but it is in the end just a patch job around missing functionality in X11.

and I sincerely wish you the best of luck

1 Like

What a fun conversation. I had no idea X11 was supposed to do shit like fetching the right color profile for a printer with a certain combination of paper and inks. And X11 people never implemented this? The loonies! :smiley:

And, really, I had no idea Scribus’s window would need CMYK color space tagging, seeing how the whole canvas of Scribus is based on so-very-much-not-CMYK-capable Cairo.

I’m afraid none of this is bringing us closer to answering the question: are Wayland people going to listen to Graeme and Flo this time?

3 Likes

Part of the problem is that I mostly agree with you, and yet you go on a rant of what I have not seen…I mean…how often do I have to state that you make a valid point?

I criticise you for using harassment as a tool and justifiying it afterwards with medieval ‘whatever it takes must be done’ logic. If you deal with people, who basically argue for you, like that, I wish you best of luck.

2 Likes

Seems pretty unlikely at this point. In my recent experience many projects/forums become a “club”, and there is an inevitable level of hazing and “prove you should be a member of this club” type thing that goes on to any newcomer. Plus the pecking order stuff.
[ And to be fair, my code development style is not suited to the “small incremental changes” norm that open source shared projects prefer. ]

So the possible outcomes of Wayland acquiring color management capability that I see are:

  1. An implementation by people that have a foot in the club that (I fear) will be partial, or made without a rounded understanding of the range of things that color management should be able to do. It could provide a path to good color management in Wayland eventually, but it will be slow, and may involve a lot of API churn, and arguments about whether API’s should be improved when they are found to be not ideal, etc. There will be a great temptation to settle for what is implemented, and not be bothered to fix things for applications that “few people use”, particularly if poor architectural choices are made in initial implementations, because those doing the implementation are on a learning curve.

  2. A fork. Do it once and do it right - rocket Wayland into the 21st Century. This was the direction I was heading in, so at least there would be a clear example of what is possible, and a distillation of what I have learned about color management systems. The fork would only survive in the long term if adopted by Distro’s, or if the code was folded back into mainline under pressure of Users/Application writers/Distro’s. [ Unfortunately my personal circumstances mean I don’t have time to work on this project in the foreseeable future. It would take something like the sponsorship of a Company to change that. ]

  3. Trojan horse. Someone who hasn’t got the mainline Wayland developers backs up could implement 2) without them being so immediately hostile to it.

9 Likes

But you gotta admire that passion tho. It’s almost like he feels somehow personally responsible, tortured even for the state of FOSS creative software.
I think he just went with the feeling he had generally about the last few posts in this thread and wrote it down. And well, that medieval logic went quite nicely with that tragedy theme.
We might not have phrased it like that but I totally get it and you do too xD

One should really make a movie about a dev that feels responsible for foss but can’t make everything himself and doesn’t succeed, a tragedy film or a play where we all go trough catharsis and realize that FOSS is the way and the dev was right all along :sob:

I wouldn’t feel personally attacked if I were you. The reply really wasn’t about your post I think, even tho he quoted you xD.

edit: sorry for going off topic, oops.

2 Likes