Problem when using XYZ LUT monitor profile with GIMP-2.10, GIMP-2.99

[Elle (Elle Stone)]

Currently by default GIMP-2.10 and GIMP-2.99 can’t correctly use ArgyllCMS XYZ LUT monitor profiles.

The problem is that these monitor profiles have an embedded matrix. By default this matrix uses the sRGB color space primaries, but with swapped RGB channels. The point of the embedded matrix with the swapped channels is to alert people to the fact that software is failing to actually use the XYZ LUT.

Indeed GIMP - or rather babl - is failing to use the XYZ LUT. Somehow babl code is detecting and using the matrix instead of using the LookUp Tables. So colors are swapped. Red becomes green and so on. Bug reports have been filed (see below).

I’m under the impression that DisplayCAL, which uses ArgyllCMS to make monitor profiles, does make an XYZ monitor profile by default. And perhaps GNOME color manager or related software also does. So even if you didn’t deliberately ask to have this type of monitor profile made, it might be what you are using.

So if you are using GIMP-2.10 or 2.99, and you see very strange colors when trying to use your monitor profile, check to see what type of profile it is. If it’s an XYZ LUT monitor profile, there are two workarounds:

• Start GIMP by specifying to disable the babl transforms. I don’t know how one would do this on Windows, but on Linux the command looks like this:

  GIMP_COLOR_TRANSFORM_DISABLE_BABL=yes /path/to/gimp-executable/gimp-2.xx

  where of course “gimp-2.xx” is either gimp-2.10 or gimp-2.99.

• Use a different type of monitor profile. A shaper matrix profile is a good type of profile to use for many monitors.

For details on how babl currently deals with XYZ LUT profiles, go here:

https://gitlab.gnome.org/GNOME/gimp/issues/2025

and here:

https://gitlab.gnome.org/GNOME/babl/issues/21

Based on the (currently as I write) most recent comments in the bug reports, it’s not clear:

• whether the way babl handles XYZ LUT profiles with embedded swapped color matrix profiles will ever change to avoid producing totally wrong colors . . .

• or whether GIMP users instead will have to avoid using XYZ LUT profiles made using ArgyllCMS . . .

• or whether perhaps ArgyllCMS could add an option to not embed the matrix, which option people making XYZ LUT monitor profiles for use with GIMP would have to use . . .

A related question is whether DisplayCAL or GNOME color manager-related software should make XYZ LUT profiles by default, if indeed they do. GIMP is not the only software that can’t use this type of profile, and often people who use GUI software to make ICC profiles rely on default settings.

[Elle (Elle Stone)]

In case you are curious, here’s the ArgyllCMS colprof documentation on XYZ LUT monitor profiles:

http://argyllcms.com/doc/colprof.html#a

A cLUT base table profile using a PCS of XYZ can be created if -ax is used, and this may have the advantage of better accuracy for additive type devices (displays, scanners, cameras etc.), may avoid clipping for displays with a colorant chromaticity that can’t be encoded in Lab* PCS space, and may give a more accurate white point for input devices by avoiding clipping of values above the white point that can occur in Lab* based cLUT input profiles. A disadvantage of this type of profile is that it can be a lot less robust if given a test patch set that is sparse, or too unevenly spaced. By default cLUT XYZ PCS Display profiles will also have a set of dummy matrix tags included in them, for better compatibility with other systems. The dummy matrix deliberately interchanges Red, Green and Blue channels, so that it is obvious if the cLUT tables are not being used. If it is important for both the cLUT and matrix be accurate, use -aX, which will create shaper/matrix tags.

[Morgan_Hardwood]

Hello

I feel I’m missing a vital piece of information in order to understand the situation and Pippin’s reply.

GIMP asks babl if it can provide a conversion for a given intent and a given pair of profiles. Babl only claims to be able to do so when both profiles have are matrix+trc profiles and relative colorimetric is requested and lets GIMP defer to lcms2 for profiles where this requirement is not met.

The profiles used here are broken and contain incorrect matric+trc profiles this is like encoding color space information in PNG chunk, exif and ICC profile and expecting exactly one of them to be picked by an image viewer and all others to always be ignored since they are configured “out of gamut” to indicate incorrectness, IMHO such ICC profiles with mismatching information should only be used for debugging.

1. When you write “XYZ LUT”, you actually mean “XYZ LUT + swapped matrix”, correct?
2. My DisplayCAL 3.5.3 defaults to “XYZ LUT + matrix”, but it also has an “XYZ LUT + swapped matrix” option.
3. Pippin writes about “matrix+trc”. How does that relate to XYZ LUT + swapped matrix?
4. When software encounters an XYZ LUT + swapped matrix profile, it should know that it should ignore the (swapped) matrix and use just the LUT, correct?
5. GIMP for some reason fails to ignore the (swapped) matrix when reading an XYZ LUT + swapped matrix profile, while other software does not fail to do so. The suggestion seems to be to not generate LUT profiles with a matrix. Why not fix the detection instead, since other software can detect it correctly and Do The Right Thing?

[Elle (Elle Stone)]

@Morgan_Hardwood

When you write “XYZ LUT”, you actually mean “XYZ LUT + swapped matrix”, correct?

Yes. Or rather I mean XYZ LUT monitor profile with any embedded matrix, but only the swapped matrix makes the problem immediately obvious. I read through the ArgyllCMS colprof documentation on making “-ax” type profiles several times, and didn’t see any way to simply not embed a matrix profile at all, but maybe I didn’t read closely enough. Is there such an option? All I see is the default option to embed with a swapped matrix that uses sRGB primaries, and a second option to embed a shaper matrix profile that actually describes the monitor.

@Morgan_Hardwood

My DisplayCAL 3.5.3 defaults to “XYZ LUT + matrix”, but it also has an “XYZ LUT + swapped matrix” option.

OK, so DisplayCAL embeds the “actually describes the monitor shaper matrix” by default in the XYZ LUT monitor profile when made by DisplayCAL? This would mean that GIMP users using “by default” DisplayCAL-made XYZ LUT monitor profiles might not even know that in GIMP they really aren’t using the monitor profile they thought they were using.

@Morgan_Hardwood

Pippin writes about “matrix+trc”. How does that relate to XYZ LUT + swapped matrix?

Hmm, some background information: babl now has code that provides ICC profile color space transforms without using LCMS. The rationale is that babl can do these color space transforms considerably faster than LCMS. Personally I haven’t been able to confirm or disconfirm “faster”, but some people have reported that it really is faster.

But currently babl is only set up to handle the specific use case of a monitor profile that is a matrix profile, and apparently also only if the user requests Relative Colorimetric intent.

For LAB LUT monitor profiles (which don’t have any embedded matrix profile), babl automatically hands the requested profile conversion to the monitor over to LCMS, or rather to GIMP, which hands the transform over to LCMS.

For XYZ LUT monitor profiles with embedded matrix profiles, and if relative colorimetric intent is chosen (which most people ideally should be using most of the time), then babl uses the embedded matrix profile by default. Hence the problem.

@Morgan_Hardwood

When software encounters an XYZ LUT + swapped matrix profile, it should know that it should ignore the (swapped) matrix and use just the LUT, correct?

Well, on the one hand, Pippin’s point seems to be that if a matrix profile is embedded in what is ostensibly an XYZ LUT profile, how is software even supposed to know whether to use the LUT tables or the matrix? I didn’t look very hard (it’s a very long document), but I didn’t see any provision in the ICC V4 specs for embedding a matrix in an n-channel ICC profile, and I don’t recall ever seeing mention of such a thing in my past perusals of the ICC specs.

On the other hand, it seems obvious that if a user selects a monitor profile with LUT tables, use the LUT tables. LCMS doesn’t have this problem of course, and LCMS used to handle all GIMP color space transforms.

@Morgan_Hardwood

GIMP for some reason fails to ignore the (swapped) matrix when reading an XYZ LUT + swapped matrix profile, while other software does not fail to do so. The suggestion seems to be to not generate LUT profiles with a matrix. Why not fix the detection instead, since other software can detect it correctly and Do The Right Thing?

In all fairness not all software ignores the matrix - last I checked Chrome used the matrix and produced swapped colors.

Regarding “Doing the right thing” I agree with you 100%. To me, this is a bug. Worse, it’s a regression from something that used to work before. Fortunately there’s a workaround but users shouldn’t have to start GIMP from the command line using an environmental variable just to get GIMP to use the actual profile the user chose in “Preferences/Color Management”.

[gwgill (Graeme W. Gill)]

Hi Elle,
note that the swapped matrix is added to profiles that are intended to be XYZ cLUT profiles (colprof -ax option). Prior versions of ArgyllCMS didn’t add a matrix to such a profile, but certain software then refused to load the profiles at all, and it seemed better to let them load and diagnose the problem (that they don’t handle cLUT profiles) more positively by recognizing the swapped matrix channels. Currently there’s no option to generate a display profile with just the XYZ cLUT. You can create a display profile with just a Lab* cLUT though. You can create a profile with accurate XYZ cLUT and Matrix tags using the -aX option, but you probably won’t know from looking at the color which tags a CMM/Application is actually using.

Typically an application not handling cLUT profiles is because the authors haven’t implemented full ICCV2 support, i.e. haven’t implemented the cLUT execution code. (There is little excuse for this when lcms is freely available though, even if that means that the cLUT path is not as fast as the matrix path. GNU licensed software could also use Argyll’s imdi code, which has faster cLUT code in most cases when running on a general purpose CPU.)

[gwgill (Graeme W. Gill)]

@Elle

Well, on the one hand, Pippin’s point seems to be that if a matrix profile is embedded in what is ostensibly an XYZ LUT profile, how is software even supposed to know whether to use the LUT tables or the matrix?

ICC profiles don’t have any way of saying what “type” of color transform algorithm they use. They are just a bunch of tags that have to meet at least one minimum set, but there are many combinations that are legal and usable (and even more if you use ICCV4 or ICCMAX tags).

It’s up to the CMM to choose a priority of which tags to use, and how to fall back if tags are missing. I’m not sure that the ICC spec. defined any priority (I couldn’t find such a statement in a quick scan), but my assumption was that the priority would naturally be from most detailed to least detailed, i.e. that cLUT would have priority over Matrix by default, since if the user has gone to the trouble of creating a more detailed cLUT based profile, the implication is that they want to make use of it. (ArgyllCMS tools have the -o option to reverse priority). I think lcms uses the same priority order.

So I’d recommend GIMP handing the color conversion of profiles that contain cLUT tags over to lcms, rather than using the Matrix tags if they are present (i.e. invert their decision logic).

[Elle (Elle Stone)]

@gwgill -thanks so much! for casting light on this issue of what tags to use if matrix and LUT tags are both present.

I’m assuming that of course PhotoShop uses the LUT tags, but as I don’t have access to PhotoShop it’s better to ask than assume . So:

Does anyone know for sure whether PhotoShop uses the LUT tags in an XYZ LUT monitor profile made using ArgyllCMS (or DisplayCAL which uses ArgyllCMS)? To be absolutely sure you’d need an XYZ LUT monitor profile with the default matrix with the swapped channels.

In case you have access to PhotoShop, but don’t have such a monitor profile, here’s one to try:

nec-20171223-g22-t5200-R255-G239-B250-qm-cal-qh-ax.icc (531.3 KB)

It won’t matter that this profile was made for a monitor that’s not your own monitor, because the colors should look really, really wrong if PhotoShop is using the matrix instead of the LUT, for example red becomes green.

If PhotoShop is like it used to be, the only way to change the display monitor profile is through a Windows utility (used to be right-click on the screen, but I forget what to choose from the resulting menu), and then you have to restart PhotoShop.

[afre (Alan)]

I don’t have PS nor have I set up colour management on my laptop. I think Windows takes care of colour management.

It isn’t clear how Windows interacts with PS. Adobe’s help doc: Color settings in Photoshop

The doc doesn’t have images. For those, see: Photoshop Essential Color Settings

[gwgill (Graeme W. Gill)]

Hi Elle,

I’m assuming that of course PhotoShop uses the LUT tags, but as I don’t have access to PhotoShop it’s better to ask than assume . So:

I’m not sure what a current version of PhotoShop does, since I don’t have a copy.
The old version I do have (V5.5) uses the cLUT rather than matrix when both are
present when loading files, and also for its display profile. (i.e. the assumption about cLUT vs. Matrix tag priority is widespread.)

If PhotoShop is like it used to be, the only way to change the display monitor profile
is through a Windows utility (used to be right-click on the screen, but I forget what
to choose from the resulting menu), and then you have to restart PhotoShop.

Yep.

I’m not sure I’m going to persuade pippin. If not, then I’m tempted to remove
the swapped channel matrix tags from the XYZ cLUT profile as a default.
Maybe leave that in there under a different option.

– Graeme.

[Elle (Elle Stone)]

As an update, code has been added to babl that now allows users to use ArgyllCMS XYZ LUT profiles, and actually use the cLUTs instead of the matrix.

I don’t know how long it will take these changes will percolate down to various precompiled versions of babl/GEGL/GIMP. But if you compile babl/GEGL/GIMP-2.10/2.99 yourself from code pulled from git, and you update the babl code, the change is already made.

Many thanks to @gwgill for helping to clarify the issues and to babl developer Øyvind “pippin” Kolås for making the changes to the babl code.

[gwgill (Graeme W. Gill)]

So the next release of ArgyllCMS will change the colprof -ax for display profiles to just have the XYZ cLUT tags and no shaper/matrix tags. That way the application/CMM will be forced to use the cLUT or nothing. -aX is still there to create a Display XYZ cLUT with fallback shaper/matrix tags, and a new option -aY will create a Display XYZ cLUT with diagnostic channel swapped matrix tag.
(Beta test is in the usual place for people who can compile the source).

[Morgan_Hardwood]

Hi @gwgill
I wanted to know what version the next release will be and when the latest release was, so I went to the ArgyllCMS website, saw that the latest release was “Current Version 2.0.1”, but failed to find a release date. As ArgyllCMS doesn’t have a GitHub/GitLab page, could you please add the release dates to the versions?
Thank you

[Claes]

Hi @Morgan_Hardwood,
ArgyllCMS version 2.0.1 was put together on July 9, 2018,
if I am able to read the signs properly.

Have fun!
Claes in Lund, Sweden

[Elle (Elle Stone)]

Another note about how babl and GIMP does now and eventually will (or might - looking at the babl bug report linked to above, it’s stil unclear) handle monitor profiles. This note also includes matrix monitor profiles such as single-channel and three-channel shaper matrix profiles:

If you look in GIMP Preferences under Color Management, there are already two options for conversions to the monitor profile, and also the same options for conversions to the soft-proofing profile. The options are:

1. Precision / Color fidelity
2. Speed

Here’s a screenshot, where I’ve chosen “Precision / Color fidelity”:

If you use a matrix monitor profile, and you choose “Speed”, the babl matrix speed-ups are already being used. This code has been in place for awhile now. If you choose “Precision / Color fidelity”, then currently the babl matrix speed-ups are not used.

The relevant babl and GIMP code for XYZ LUT monitor profiles isn’t yet fully implemented even in git. But in the future, at least as currently planned:

• If the user has chosen an XYZ LUT monitor profile, and the user chooses “Speed” and also there is a shaper matrix in the XYZ LUT monitor profile, then the matrix is used all the time and the babl matrix performance speedups are enabled. As already notes, now there is code that detects the swapped channel matrix, and this matrix is not used.

• If the user has chosen an XYZ LUT monitor profile, and the user chooses “Precision / Color fidelity” and also there is a shaper matrix in the XYZ LUT monitor profile, then the matrix will still be used but only for generating quick previews, after which the cLUTs are used to generate a more accurate display of the image.

Given that @gwgill will be adding an option for making XYZ LUT monitor profiles that will allow not embedding any matrix at all, the new colprof options will have implications for users making such monitor profiles, and perhaps implications for the default behaviors of “gui” profile-making software such as DisplayCAL and software that’s part of gnome color management.

In other words, GIMP users need to think carefully about what options they use to make monitor profiles, regardless of whether they use the command line or use DisplayCAL or gnome color management software or whatever.

I’m hoping that once the full set of babl/GIMP code changes are actually implemented, people on this list can help test whether, how much, and under what circumstances operations in GIMP really are speeded up by choosing “Speed” vs “Precision/ Color fidelting” in GIMP Preferences.

Again, currently the babl speed-ups are only applied if you also choose “Relative colorimetric intent”.

[Morgan_Hardwood]

Hi @Claes

The intention of my post was not to ask when v2.0.1 was released, but to raise the issue that this information is missing from the website. A date in the header, “Last updated 2018/7/9”, typically applies to when the text on the page was last edited, not when a specific release listed further down on the page was tagged. The problem could be easily solved by adding a date under the version line.

There are other issues too, for example no explanation on how to report a bug (ctrl+f for “bug” and “report”), so one must resort to email.

All of these issues could be resolved by using a web-based source-code and issue-tracking system like GitLab, GitHub, etc. (both of which support email, but also so much more).

[gwgill (Graeme W. Gill)]

The intention of my post was not to ask when v2.0.1 was released, but to raise the issue that this information is missing from the website.

It will be released when it is released. There is no schedule, so there is no date to be missing from the website. Beta source is available for download. Sometimes I will make beta binaries available for those not in a position to compile their own, but this is ad-hock.

There are other issues too, for example no explanation on how to report a bug (ctrl+f for “bug” and “report”), so one must resort to email.

The website makes it clear that bugs should be reported to the ArgyllCMS mailing list or directly to me.

All of these issues could be resolved by using a web-based source-code and issue-tracking system like GitLab, GitHub, etc. (both of which support email, but also so much more).

ArgyllCMS is a personal project, and the number of bug reports is small enough that I don’t think that maintaining a formal bug tracking system would improve my life.
[ If it were a multi-person project, then public version control and bug tracking would be much more desirable. ]

[darix]

@gwgill

ArgyllCMS is a personal project, and the number of bug reports is small enough that I don’t think that maintaining a formal bug tracking system would improve my life.
[ If it were a multi-person project, then public version control and bug tracking would be much more desirable. ]

You are wrong here IMHO. Visibility for the user is also an important part. Be it end user or people like me who package the software. Being on e.g. github/gitlab also reduces the hurdle for contribution (be it code or bug reporting), as many people might have an account there already, or in the case of gitlab can reuse existing accounts.

As a side note: would you accept patches for meson as a build system for argyllcms?

[gwgill (Graeme W. Gill)]

As a side note: would you accept patches for meson as a build system for argyllcms?

No. I’d prefer to spend time working on new features and new projects rather than re-working a build system with which I happy enough. Given I use it 100 times as much as anyone else, that’s what counts. (In any case meson doesn’t work the way I prefer, since it doesn’t permit in-source builds. Depending on a large project like Python is not thrilling either - a thing I have striven for in ArgyllCMS is for it to be self contained, and not a project that is dependency hell.)
[ Of course you are free to create your own build scripts (and maintain them) for your own situation, if you so desire. ]

[Morgan_Hardwood]

Hi @gwgill

@gwgill

It will be released when it is released. There is no schedule, so there is no date to be missing from the website.

Sure, but I did not ask for a future release date. I pointed out that there is no date given for the latest release (and no release history with dates, which also comes in useful when you’re writing articles, when you’re a package maintainer for various distributions, etc.).

@gwgill

The website makes it clear that bugs should be reported to the ArgyllCMS mailing list or directly to me.

The website is a wall of text, and ctrl+f for “bug” and “report” turned up nothing useful, so I disagree that it makes that clear.

I agree with @darix

You are of course free to handle the project as you wish, but there are benefits to using a git/bug tracker even if it’s a one-man thing. In fact let me list a few off the top of my head and in no particular order:

• It exposes hundreds of thousands of existing GitHub/Lab programmers to your project without the hurdle of requiring to register for a new account. It encourages contribution.
• Clear release history with dates.
• Clear visibility of what changes from version to version, branch to branch, commit to commit.
• No overhead for you, assuming you already use git.
• You can still use GitLab/Hub via email if you prefer that.
• Clear list of bugs open and closed, your thoughts on them, and their resolution or workaround.
• Insight into the humans behind the project, their willingness to accept patches, their anti-patterns, etc.
• Not forcing others to use a mailing list to get insight into the above.

I hope that makes it worth considering.

[darix]

@gwgill

No. I’d prefer to spend time working on new features and new projects rather than re-working a build system with which I happy enough. Given I use it 100 times as much as anyone else, that’s what counts. (In any case meson doesn’t work the way I prefer, since it doesn’t permit in-source builds. Depending on a large project like Python is not thrilling either - a thing I have striven for in ArgyllCMS is for it to be self contained, and not a project that is dependency hell.)
[ Of course you are free to create your own build scripts (and maintain them) for your own situation, if you so desire. ]

And you are probably the only one still using that build system which makes contributing to it hard. I picked meson because it can handle windows/mac/linux/bsd in one build system. my POC only does linux for now but it builds part of it already. and for the dependencies … it is relatively lightweight. basically just python and ninja (though this might be optional).

The reason why I started hacking on the build system in first place that Jam did claim not to find some of the libraries which were installed though. with meson I could just query pkgconfig and be done.