Confusion on color profiles

Software darktable
66

This is exactly the thing I was setting up yesterday… I have run the following code snippet, with the display profile set to some wide-gamut colorspace in the preferences:
02


  CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  if( !colorSpace ) {
    std::cout<<"Cannot get CGColorSpaceCreateDeviceRGB()"<<std::endl;
  } else {
    std::cout<<"Display profile: "<<CFStringGetCStringPtr(CGColorSpaceCopyName(colorSpace),kCFStringEncodingASCII)<<std::endl;
    CFDataRef data = CGColorSpaceCopyICCProfile(colorSpace);
    if( !data ) {
      std::cout<<"Display profile: cannot get CGColorSpaceCopyICCProfile()"<<std::endl;
    } else {
      CFIndex icc_length = CFDataGetLength(data);
      const UInt8* icc_data = CFDataGetBytePtr(data);
      cmsHPROFILE icc_profile = cmsOpenProfileFromMem( icc_data, icc_length );
      char tstr[1024];
      cmsGetProfileInfoASCII(icc_profile, cmsInfoDescription, "en", "US", tstr, 1024);
      std::cout<<"Display profile: "<<tstr<<std::endl;
    }
  }

  colorSpace = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);
  if( !colorSpace ) {
    std::cout<<"Cannot get CGColorSpaceCreateWithName(kCGColorSpaceSRGB)"<<std::endl;
  } else {
    std::cout<<"sRGB profile: "<<CFStringGetCStringPtr(CGColorSpaceCopyName(colorSpace),kCFStringEncodingASCII)<<std::endl;
    CFDataRef data = CGColorSpaceCopyICCProfile(colorSpace);
    if( !data ) {
      std::cout<<"Cannot get CGColorSpaceCopyICCProfile()"<<std::endl;
    } else {
      CFIndex icc_length = CFDataGetLength(data);
      const UInt8* icc_data = CFDataGetBytePtr(data);
      cmsHPROFILE icc_profile = cmsOpenProfileFromMem( icc_data, icc_length );
      char tstr[1024];
      cmsGetProfileInfoASCII(icc_profile, cmsInfoDescription, "en", "US", tstr, 1024);
      std::cout<<"sRGB profile: ICC name = "<<tstr<<std::endl;
    }
  }

The terminal output from the above code is quite interesting:

Display profile: kCGColorSpaceDeviceRGB
Display profile: cannot get CGColorSpaceCopyICCProfile()
sRGB profile: kCGColorSpaceSRGB
sRGB profile: ICC name = sRGB IEC61966-2.1

To cut it short, the profile retrieved with CGColorSpaceCreateDeviceRGB() DOES NOT HAVE any associated ICC data, while the one retrieved with CGColorSpaceCreateWithName(kCGColorSpaceSRGB) corresponds to a standard sRGB ICC profile.

Also, quoting the Apple documentation:

Device color spaces are primarily used by iOS applications because other options 
are not available. In most cases, a Mac OS X application should use a generic 
color space instead of creating a device color space. 
However, some Quartz routines expect images with a device color space. 
For example, if you call CGImageCreateWithMask and specify an image as the mask, 
the image must be defined with the device gray color space.

Nowhere I see mentioned that CGColorSpaceCreateDeviceRGB() should return the display profile, and from what I observe it seems to be a very bad choice for handling color management stuff…

67

It’s not clear to me that Apple have broken their API so much as moved the ground under it. Rather than being able to set the native space using colorWithDeviceRGB etc., the null transform trick is now needed for non-primary displays. (See ArgyllCMS/spectro/dispwin.c variable nscs for how I’m handling it.)

68

Just to clarify, if the only processing steps done with darktable before exporting the image to disk are these:

  • demosaic
  • hot pixels
  • chromatic aberrations
  • white balance
  • raw black/white point

and if the input and output profile are the same (in my own case, a custom camera input profile) . . .

. . . then is there any forced conversion to Lab or to an RGB working space not selected by the user?

darktable passes my “very yellow flower” raw file test - there doesn’t appear to be any clipping in the blue channel from any forced color space conversions that might one way or another impose gamut clipping. Of course this doesn’t mean there aren’t forced non-gamut-clipping conversions to Lab or some internal RGB working space.

Maybe I should have said darktable doesn’t do any automatic non-user-selected conversions to internal working spaces? I’m guessing darktable doesn’t convert to Lab unless/until the user selects an operation that’s done in the Lab color space?

69

Yes, the Lab conversion is always done. That might change in the future but don’t count on it.

70

Sorry but I cannot stop thinking how funny is that I got into this forum as a new DT user with a “simple” doubt on profiles, and look at what this thread has generated.

Now, if at some point you could “translate” all the (undoubtedly necessary) technicalities would be nice, so that “simple” DT users can benefit of eventual upgrades in terms of color management and similar.

Thanks to you all.

3 Likes
71

Hi,

TL;DR, as I understand it (again, if I’m wrong I’m sure someone will correct me): use “system profile”, and you will see the “proper” colours, albeit clipped to the sRGB gamut. Importantly, this clipping only affects what you see when viewing the image from inside darktable. No clipping will occur on the output image. So, if you open the output image in another program that doesn’t suffer from the Cairo limitation on OSX (Photoshop should work, I suppose), you should see all the colours that your monitor is capable of displaying.

When the situation changes, you will definitely see this in the release notes of darktable and all other image processors that are affected by this.

2 Likes
72

Thanks.
I think I understood that. In fact as mentioned, at least in my case, “system profile” and “sRGB” are totally identical. Makes sense.
However, I dislike the idea of not getting the most out of my monitor, because it means falling back to sRGB. And I am sure many, many others of OSX with wide gamut display will feel exactly the same.

See you soon then.

73

Yes, it will be mentioned. And even then the system display profile setting should still be the right thing to use.

74

I’ve been using ArgyllCMS to “accomplish this” and “test that” for as long as I’ve been using Linux (since somewhere around 2007), and so far ArgyllCMS is the one and only color-management-related software that seems to me to be 100% reliable. And so I have a huge respect for anything @gwgill says about anything related to color management.

Color management on X works. I don’t mean that non-color-managed desktop widgets look good on a wide gamut display. I don’t mean that multi-monitor setups are easy to use. I don’t mean color management happens “automagically and correctly” without the user having some knowledge of what they are doing. I only mean two things:

  • When using ICC profile color-managed editing applications (GIMP/Krita/free-libre raw processors/etc), on X the images are shown with correct colors, at least on the main monitor, assuming of course the user knows how to calibrate and profile their monitor, how to load the vcgt tag if required, and how to tell the software to use the monitor profile.

  • When using a proper OCIO lut made from one’s properly made monitor profile and the profile of whatever working space one wants to use, on X the images shown by Blender and presumably other OCIO-color-managed applications have correct colors.

In case it might be useful to members of the pixls.us forum, here are links to various discussions of color management over on the Wayland development mailing list, more or less in order by time:

This page has a box for searching the wayland-devel archives:
https://lists.freedesktop.org/archives/wayland-devel/

A lot of the Wayland discussions go right over my head. But if the people developing Wayland don’t actually understand (or care about?) color management as it pertains to color-managed image editing, that’s a huge problem.

75

As the conversion to Lab doesn’t seem to clip colors, then of course that’s not a problem. From curiosity, does darktable use LCMS to make the conversion?

76

It depends. When the input/output profile is a matrix profile we use our own fast path, otherwise lcms2 is used. You can force darktable to always use lcms2 in the preferences though.

77

As promised, I will summarize here my findings after experimenting a bit with the color profiles in OSX.

First of all, here is some explanation of the logic being used by Cairo under OSX.
When painting pixels on the screen, Cairo uses the Quartz 2D backend. Internally, the pixel data is associated to a CGImage structure, and is transferred on screen by drawing the image into a CGContext associated with the application window.

The Quartz 2D graphics library is intrinsically color managed. The destination graphics context is associated with the display profile, and the input CGImage must also define the profile needed to interpret the pixel data. The library takes care automatically of the colorspace conversions when the image is drawn into the graphics context.

From the point of view of a photo editing application that wants to keep control of the colorspace conversions, this is of course not convenient, particularly because the Apple API does not provide any way to disable Quartz internal color management (probably they consider the software developers not smart enough to take care of color management :wink: ). The only way I could find to “bypass” the color management is to assign the current display to the CGImage structure, so that Quartz applies an identity transform (it seems that in this case Quartz does not apply any transform).

As far as I understand, the “identity transform trick” was the original goal of the Cairo developers as well, but was accomplished in a way that is not working anymore in recent OSX versions. For this, they associate to the CGImage a colorspace obtained with the CGColorSpaceCreateDeviceRGB() function. However, from my experimentation as well as from some Qt bug report it looks like DeviceRGB is treated internally as sRGB by Quartz, hence resulting is an unwanted color transform.

The best working solution I could find is to replace CGColorSpaceCreateDeviceRGB() by the following function in CairoQuartzCreateCGImage():

static CGColorSpaceRef CGColorSpaceCreateDisplayRGB()
{
  CGColorSpaceRef result = NULL;
  result = CGDisplayCopyColorSpace(CGMainDisplayID());
  if (!result) {
    result = CGColorSpaceCreateDeviceRGB();
  }
  return result;
}

The retrieved monitor profile corresponds to the “main display”. Hence, this works correctly in a single-display setup as well as if the application window is on the main display in a multi-display configuration.

In order to support displays other that the main one, one would need a mechanism for passing the display ID to the underlying Cairo library. Unfortunately, the Quartz API does not provide any interface for retrieving the color profile associated to the destination CGContext…

I am currently preparing a PhotoFlow package with this patch included, so that eventually other OSX users with wide-gamut displays can give it a try and see if they get the expected result.

2 Likes
78

Oh my, I actually understood this. No wonder my wife’s eyes just glaze over when I use the word ‘color’ in a sentence… :smile:

3 Likes
79

@Matteo_Bertolino @Morgan_Hardwood @houz - I have prepared a new version of my PhotoFlow editor that is linked with a Cairo library patched as I suggested above. You might use it to test the visualization of wide-gamut images on your wide-gamut display, without the limitations imposed by the standard Cairo code. For this, you have to set by hand your display profile in PhotoFlow’s preferences:

58
581224×872 38.9 KB

The updated photoflow package is here.

The reason why I am proposing to use PhF here is simply because I have full control over the build process, and I can easily do some testing. This exercise should serve as a proof-of-principle for a possible patch to be proposed to the Cairo developers, so that the sRGB limitation under OSX might be solved for all GTK-based image editors…

Let me know what do you think.

4 Likes
80

Hi.
Well, first of all, many thanks for your work. That’s good news indeed.
Now, I am not sure I will be the best person to run tests with…anyway, I’ll give it a try.
First I need to install Photoflow, which I come to know the existence of today. Once installed I’ll get back here, probably for further guidance.
I’m sure the others will be able to help out a lot more.

81

ok. installed.
Now, which profile shall I select from the .icc folder? My own, as produced by the calibration process? Actually in that folder I only have my own created .icc profiles. No generic AdobeRGB98. I wonder where all the others have gone.

82

@Matteo_Bertolino - Yes, you have to select the ICC profile you created for your display. Other profiles can be found in /Library/ColorSync/Profiles and the relative sub-folders, as well as /System/Library/ColorSync/Profiles.

Thanks for checking!

83

@Carmelo_DrRaw great to see that there might be a solution at least for the common single-monitor setup in macOS!

However I don’t have macOS myself (I visited a company in Stockholm a few years ago to help them with their macOS color-managed setup and ran into the macOS sRGB wall) and I no longer have the Eizo, though I do profile and color-manage my (very good ASUS UX305CA) laptop screen. Would you like me to test anything or was your call for testing only for macOS users with wide-gamut screens?

84

Hi Morgan.

Just a question: why do you refer only to “single-monitor-set-up”?
Do things change if I have a a second monitor plugged in? In my case my main iMac monitor is broken (yet active, but useless) and I use only a second monitor (Eizo), as main monitor (calibrated via i1 Display Pro).
And, what do you mean by “ran into macOS sRGB wall”?

Many thanks.
M

85

@Matteo_Bertolino

So it sounds like you could expect correct colors on a multi-monitor setup provided the program’s window is on that monitor, if you use @Carmelo_DrRaw’s photoflow package and the code works.

By “the macOS sRGB wall” I mean the problem that GTK+ applications in macOS are clamped to sRGB even if you use a wide-gamut monitor profile. That is what @Carmelo_DrRaw’s code above attempts to work around. More info in this thread: Wide-gamut preview in macOS

1 Like