Color Management in Raw Processing


#62

The darktable manual says:

  1. up to demosaic – Image is in raw data format with only latent colors
  2. between demosaic and input color profile – Image is in RGB format within the color space of the specific camera or input file.
  3. between input color profile and output color profile – Image is in Lab format.
  4. after output color profile – Image is in RGB format as defined by the selected display or output ICC profile.

Also, it talks about two profile conversion steps (rather than an assignment and a conversion): “This process implies that the pixelpipe has two color conversion steps: input color profile and output color profile.”


(Ingo Weyrich) #63

Iirc correctly, in RT the dcp is applied in two steps. Both steps are after demosaic on RGB data. The secons step applies the dcp tone curve and the dcp lookup table, if enabled, a bit later in toolchain…


(Glenn Butcher) #64

No mention of a working profile.

Note that previous discussion here was about color space transforms between RGB spaces; I think RGB -> LAB would be another transform, but for a different concern…


#65

In darktable most modules work in the Lab space (which is what @aurelienpierre criticized not long ago).


(Hevii Guy) #66

:rofl::sweat_smile::laughing: Stop it, Glenn. Stop it. You’re killin’ me!


#67

Mathematically speaking you don’t need the white point since you can always calculate it (set R=G=B=1 and do a matrix times vector and that should give you the white point) but having the white-point makes things quite a bit easier when since it becomes unnecessary to calculate it. Which is rather handy since there are quite a few color conversion involve the white-point.


(Glenn Butcher) #68

Really? And I was wrapping my head around the white point triplet being the thing that captured my white balance in a no-whitebalance target shot for a white balance compensating camera profile. Makes sense though, just working backward from the objective…

In addition to being a color neophyte, I’m not a math person. Three college degrees, FOUR math courses total. I found a way to do computer science without the pain that is undergraduate ‘analog’ math, or calc/diffie-q. Matrix math and its applications just hurts my head, and that is one thing that I encounter in both photography and my day job…


#69

Never too late to whip out your handy dandy notebook and pen down those matrices for sport. When you are done, MathJax or asciimath it up for us so we that we may see the elegant colour math at work.


(Glenn Butcher) #70

For those who probably winced at my choice of Powerpoint to cobble together the initial graphic, I have just converted it to LibreOffice Impress with no discernible differences. I am more impressed (oh, no pun intended) with LibreOffice with every release.


(Glenn Butcher) #71

I recently wrote some C++ routines to do matrix math, in part looking for an intuitive way to express matrices in C++ std::vectors, in part to figure out David Coffin’s dcraw C elegance, in part to do sport. Well, and I’m also noodling with how to write colorspace transforms to implement DCP profile handling. Oh, the std::vector thing turned out to be easy:

typedef std::vector<float> floatvector;
typedef std::vector<floatvector> floatmatrix;

floatmatrix bradford_D65_to_D50 =
 {
      { 1.0478112, 0.0228866, -0.0501270},
      { 0.0295424, 0.9904844, -0.0170491},
      {-0.0092345, 0.0150436,  0.7521316}
 };

and then I wrote routines to do dot_product(), multiply() and convenience things like printmatrix(), isSquare(), etc.

Keeps me out of the bars…


(Gord) #72

The horror! Quit math immediately!


#73

Yes the matrix vector product describes a linear combination of 3 vectors, in this case the 3 vectors describe the XYZ coordinates of respectively the Red, Green and Blue components. So the Matrix Vector product effectively becomes

\overrightarrow{XYZ}_R * R+ \overrightarrow{XYZ}_G * G + \overrightarrow{XYZ}_B * B = \overrightarrow{XYZ}

and when R=G=B=1 that should give you your white point.

EDIT: Fixed math, thanks @agriggio!


#74

Is it possible to make the arrow span the length of XYZ? It is too short for my tastes. :stuck_out_tongue:


#75

If I knew how I would


(Alberto) #76

@dutch_wolf there you go

\overrightarrow{XYZ}_R * R+ \overrightarrow{XYZ}_G * G + \overrightarrow{XYZ}_B * B = \overrightarrow{XYZ}


(Edgardo Hoszowski) #77

Sorry if this is off topic, I’m just learning about this things and this is really helping.
In darktable the input color profile module provides several options for the profile (standard color matrix, custom icc, etc) and uses an in memory Lab profile to do the actual transform. Is that Lab profile the working profile you refer to? (even if is not RGB)


(Mica) #78

No, your working profile should be something like ProPhoto or some other wide gamut profile, or Rec 702 or Rec 2020 if you want a linear profile.


(Glenn Butcher) #79

I’m going to leave it to one of the darktable folks to give the definitive answer.

If I had to guess, there’s probably a separate conversion from whatever RGB space is the working profile to LAB, so that tone manipulation can be done without dorking up the color. If an image is in LAB colorspace, if you apply a curve to the L channel, you’re modifying Luminosity, independent of the other two channels, which describe the color. There’s differing opinion on whether this is a good idea.

It can be a bit confusing because LAB is also one of the intermediate colorspaces that can be used to do transforms, at least per the ICC spec. So profiles don’t need to know about each other, transforms are done through an intermediate color space called the Profile Connection Space, or PCS (never too many acronyms…) So, when the software transforms your image from the input camera profile to whatever working profile, it’s actually done inputprofile -> PCS -> outputprofile. XYZ and LAB are the two spaces the ICC calls out for use as PCSs; XYZ is the most-often-used one, in my limited experience.


#80

Darktable doesn’t really have a RGB working profile it does (almost) everything in LAB so LAB is the working color space for darktable. At the time this seemed like a good idea due to LAB covering the full human visible gamut (so anything the camera can see will be covered) and the inherit separation of chroma from luma, alas you run into issue since LAB is perceptual uniform which isn’t the greatest way to edit[1]. The other issue is that LAB is ‘here be dragons’ territory with regards to editing HDR[2].

TL;DR for now the working ‘profile’ of darktable is LAB (some people are working to change this but it is a slow process since they want to preserve older edits)


[1] For historic reasons a lot of editing was done perceptual uniform since it could be done with only 8 bits per channel but now that we can do 32bit float editing in a linear space gives a lot less artifacts
[2] There has been done some research but what I could find is that although L* is surprising accurate to well above 100 is is unknown how accurate a* and b* are at those L* values.


(Edgardo Hoszowski) #81

I used darktable as example because I’m familiar with it and (some of) the code, but my goal was to understand what the working profile is, and is more clear now, thanks.