According to the reference doc, colors are essentially clipped to the boundary of the gamut. There is an example on page 109 of the API doc pdf file. In the doc search for “gamut mapping”.
If a color lies outside the gamut, the colorfulness (“C” of the LCh) is reduced such that the new C lies right on the gamut boundary. This is done using constant h (hue angle), and L is also unchanged. Since L is unchanged, I guess you can think of it as a slice through the LCh cylinder space perpendicular to the L axis. So in that slice, each point on the boundary of the gamut is described by rectangular coordinates ‘a’ and ‘b’. In that slice, as the hue angle (h) changes, the gamut boundary has corresponding a and b coordinates defined and the max C for any given h angle is the hypotenuse of the triangle formed by a and b for that particular h.
So mapping is achieved by reducing C until it matches the same C value you get at the gamut boundary.
What isn’t described there is how C gets “reduced”. Do they desaturate by adding white? Some other method? Off hand I’m not sure why it would not escape the challenge we see with v6.
I did read that, but didn’t bring it up because flannelhead implied that LittleCMS was used for the gamut check and not the gamut mapping part.
It would be desastrous to use LCMS gamut mapping, both Flannelhead and Aurelien would have spottet that almost certainly.
for the sunflower set your histogram to srgb and you can easily tell the highlights are too high. With the histogram set to the other colorspaces you will think you have more headroom to go before the highlight compression kicks in.
Need to test this on my amoled display that comes with an adobergb and p3 profile to see if darktable allows me to use that extra headroom when setting the output to one of those.
just noticed that windows photos is showing my images incorrectly. In photoshop, krita, and even microsoft paint it shows it correct.
filmic v6 maxrgb , colorbalancergb (would increase the shadows a bit if I did it again)
The histogram output is the processed pipeline…I don’t understand the before HLR kicks in…its one of the earliest steps… could you clarify…from your logic is sounds like you are suggesting that people stay in sRGB the whole way through?? Again I could be wrong… When you move from a higher gamut color space to a lower one there are out of gamut colors going in to the conversion but they are mapped to the new space and depending on the settings and the software using absolute perceptual or relative mapping so I am not 100% sure what point you are trying to make…for sure I don’t see any raw clipping in this image either???
Thanks for taking a stab at this. Be careful with the histogram settings, as it is a known confusion point for DT users. The recommended approach is to set the histogram profile to your working profile. Additionally I’d suggest using the waveform view. The upper dotted line will correspond to 100% relative to the display-referred profile, which lets you easily see the point at which true clipping occurs in the output profile. But having said that, even a small amount of clipping is not a disaster. What I can see in your edit is what I see a lot when using max RGB. Local color contrast gets degraded and you end up with flattened, almost pastel effect. I’m not slamming max RGB totally, but I think it’s better suited for portraits or other content where local color contrast or detail is not as important as smoothed out color gradients. My starting point is always “no” color preservation. If I see things I don’t like, then I take a look at the other preservation modes. No absolute best here, just sharing what i do.
When I have my histogram set to lets say rec2020 or adobergb (can’t remember what’s darktables default. I might have changed this months ago) I might think my highlights are not clipping yet but If switch the histogram to srgb I will see that the highlights are already clipping and instead of pushing them more like I would if I think I still have headroom, I will push the midtones instead.
If darktable’s default histogram profile is srgb that means I changed it some months ago, but I’m sure I’m not the only one using a diff profile for it.
@bnpndxtr correct, testing with no preserve does get rid of the flat look. My working profile is rec2020 and setting the profile to it leads me to make diff choices when editing because it makes me think I have more reach than my output.
edit:
took another crack at this and the results are decent. used various preserver chromi , but I also used the extreme luminace saturation slider to nudge the colors off the border of the vectorscope (srgb setting). Definitely needed to use the tone eq
learned a lot from this test. For the sunflower I couldn’t get the energy how I wanted , so I carried it as far as I could with filmic and then added a colorbalancergb post filmic to nudge it (the brilliance) did the same for the girl in the hat. I’m not against any new options, just wanted to see if I could tackle the issues.
This is an excellent question and I’m sorry my reply is very late. But I think I found an excellent image to illustrate this on.
This is an edit with sigmoid from an experimental build of mine. As of currently, sigmoid doesn’t have a similar gamut mapping as filmic v6 does. Sigmoid does its processing in Rec.2020 primaries which have a wider footprint than most displays (which are usually closer to sRGB / Rec.709 primaries).
I went pretty over the top with the contrast to illustrate these things better.
Notice the rose seems really bright and very much like the monitor primary red. There are two kinds of clipping going on here:
When the Rec.2020 RGB values are projected to Rec.709, there might be negative RGB values introduced. The display can’t represent those, so they get clipped of. If we have a red slightly bending towards magenta but it’s out of the destination gamut, the RGB triplet might be something like RGB = (0.6, -0.1, 0.1). The negative green component is decreasing the luminance (but that’s correct since luminance stays invariant in the projection). If it’s clipped to zero, the luminance increases. Hence the rose looks so bright. Clipping of negatives like this also causes a skew towards the primaries: red, green and blue.
There clearly is also a clipped area in the foremost petals where the reds are too bright to be displayed. The triplet might be something like RGB = (1.1, 0.0, 0.2). In the output stage, the value above 1 gets clipped to 1, so we have RGB = (1.0, 0.0, 0.2). Notice that the red and blue component got closer to each other, so the color is skewed towards magenta. This is exactly what I see going on in this image. In general this “upper side” clipping causes skews to the secondaries: yellow, magenta and cyan.
Here I added proper handling of negative destination gamut RGB values such that the luminance is preserved.
Notice that the rose got quite a bit darker and IMO looks a bit more realistic now. The “upper side” clipping issue in the foremost petals is still present. There’s also a horrible skew of the background greens toward yellow.
Here the “higher side” compensation is implemented to avoid the skew to magenta. I didn’t implement it exactly like in filmic which does it at constant luminance but aimed a bit lower.
The front petal doesn’t look magenta anymore and there’s now something that resembles its shape despite me trying hard to destroy it by cranking the contrast so high.
When one bypasses the gamut_check_RGB() call in filmic, similar things might happen. The issue causing too much desaturation (and the resulting perceived skew toward salmon) might be because the chromaticity angle preservation is so strict, while people might actually expect to see a slight skew toward yellow in these areas. So a complete removal of the gamut check isn’t a good solution in the long term, but perhaps aiming at relaxing the gamut mapping a bit would do.
(Ps. @jandren some gamut mapping experiments with sigmoid here. A bit simpler than in filmic. Works well with experimental custom primaries.)
@flannelhead Thank you for a great rundown the cause and effect here. It all makes sense. I think one reason I don’t see too much of this with my hack is that I’m really scrutinizing the results for loss of texture and detail, so I kind of see when the “magic marker” color fill effect starts to become too apparent. But you are absolutely right that I’ve pulled out a failsafe for the higher side.
I think the best thing to come out of all of this discussion, here and on other threads, is the awareness about effect. Perhaps as you say there are some ways to provide some relief so that the real world use cases can be accommodated. I think that what we are seeing in everyone’s comments on this and with similar series of photos is that these photo subjects are not really corner cases for photography. These cases are awesome photo-ops that are very popular and arguably even common for photography.
Is the flat look unmodified filmic max-rgb no easily fixed by raising the display output (and letting people clip things if they want to that way to get punch back ?).
Or does that also skew the gamut mapping?
@flannelhead if you’re thinking about ‘relaxing’ it… Isn’t it an idea to add a slider to the advanced filmic settings with how much gamut mapping to apply , between 0% and 100%?
Or is 'yet another complicated slider ’ something that is unwanted ?
The reason i didn’t want to do it this way in my own experiment if possible was because although raising the display output fools filmic by artificially raising the value of display_white, in addition to making the gamut_check_RGB issue go away it also reduces or removes tone mapping of the high end. Simply bypassing RGB gamut check function instead allows colors to avoid the perceptual salmon looking desat without giving up the benefits of normal filmic tone mapping. This comes with risk of out of gamut colors in the high end that must be assessed manually, but in my case this was preferred to the otherwise aggressive desat.
As it’s easy for me to forget while I’m in there actually using the tools, it’s also good to keep in mind that, although they can be related, gamut clipping and histogram clipping are not the same thing. Gamut clipping can happen at individual RGB channel values well below 100%. Raising the value of display_white progressively makes the histogram clipping go away, and affects gamut clipping too because (I think…) it makes the gamut checker think that a given RGB triplet is further inside the gamut rather than near the edge.
What I have done previously with v6, which is kind of similar to what you describe, is to get filmic set the way I want it, initially ignoring the desat effect in the more brilliant colors. Then using an instance of color balance RGB with a luminance mask, I reduce the brilliance of just those “highlights” that are experiencing the desat due to gamut_check_RGB until the desat goes away. Then I add another instance of color balance RGB post-filmic with a raster mask from the previously mentioned instance of color balance RGB, and simply boost the brilliance of those same highlights by the same amount used previously to reduce their brilliance. This method retains the tone mapping benefits of filmic for everything except the areas I manually manipulated. But to me this is a hassle to do, and I wanted to avoid it if I could come up with some other way, even if as a temporary bandaid until something better is available.
However I see part of your question is about maxRGB color preservation mode. I almost never use this mode as long a I can get “no” to look good. maxRGB seems to clobber local contrast and color contrast IMO and seems to really only be good for subjects where surface details are not that important as it tends to give everything a matte look. So I think part of the issue you describe is just a property of that preservation mode. If you are trying to manage smooth color transitions or maybe doing portraits then maybe maxRGB would give better results to the other modes. I have never understood why maxRGB is the default selection, as to me it almost always gives inferior results (which admittedly is a partially subjective call on my part).
I took a look at some of these. You have to be careful because some of the modes make the more brilliant colors look better (like “hue” blend mode) but they essentially defeat most of what filmic is doing (like basic tone mapping!). “lightness” didn’t seem to do much. “HSV value” had more effect overall, but made it look just like what maxRGB would do to the textures and color contrasts.
“darken” mode yields results that look surprisingly similar to bypassing gamut_check_RGB in the sunflower image, because it masks out the gamut mapping desat method from functioning at all in the upper end of the curve (adding white would lighten, so it is prevented) however a sneaky side effect of this is that it will also defeat any contrast adjustment that would have normally been accomplished by brightening. So imagine the upper half of the filmic s-curve (or even a traditional s-curve) becoming flattened back to “nominal”, while the lower half of the s-cure was allowed to remain “s-looking”. It may be that this doesn’t really visually matter on a particular image, so it could be useful.
But these blend modes are interesting and also present different ways to incrementally work around the core issue, so thanks for reminding me of those. Even though I use a few of the blend modes a lot in my own editing, I always like when I stumble onto what various blend modes can add to the toolset.
Those sound like the display blend modes …do I am going to check the scene ones… I did a few in lightness and they didn’t look too bad but it was like one or two images…
What is interesting here with images like this…is for example you can see that it would seem the UCS gets invoked with rgb color balance. I edited this image either with sigmoid or no sigmoid and just channel wise auto levels… With a very basic edit just enable rgb colorbalance with no parameter altered ie the no-op module and you will see the perceputal space expands the blues and makes the yellows paler… so maybe in a good way wrt gamut but it looks washed out…then in the masks tab change it to JzAzBz…the impact is less. So its not just the tone mappers playing with the color and the potential loss of contrast or apparent desaturation you also need to be mindful what rbg colorbalance is doing…
EDIT “gamut but it looks washed out” this might be a bit exaggerated by me using levels…
Interesting to compare the blend modes in sce vs display reference, as I’ve never messed with that setting before in other modules. It looks like the scene referred modules default to the scene referred blend modes, which makes sense.
But anyway with filmic the scene referred lightness mode looks very close to my edit. Just very subtle differences (visually anyway) here and there. I tried this on some other images and they look approach my other edits except in cases were filmic highlight reconstruction is used. The lightness mode seems to goof with or break filmic’s highlight reconstruction in the case I had, and I guess it makes some sense. But I’m going to try some other examples too.
I took a quick look at this, as it’s another setting I never change. So I couldn’t any difference between the color saturation science types until I actually started changing saturation, etc. But once I made any changes to the settings, then the color saturation method made a big difference in how it looked. In general I liked the UCS version better, but I didn’t experiment very much with it.
They (scene blending) have the fulcrum which you can use to modify the blended result and brighten or darken the image in that module so it can be useful to use them to sort of reset or adjust the middle gray if needed… From the manual
Classic blending modes, designed for display-referred RGB (constrained to 0-100%), implicitly define a fulcrum at 50% (gray) or 100% (white) in their algorithms, depending on the blend mode. Because scene-referred is not subject to these restrictions, this fulcrum needs to be explicitly defined by the user when performing blending operations in the “RGB (scene)” color space. The additional blend fulcrum parameter will be presented to the user when using one of these blend modes in this color space. The effect depends on the operator used. For example, values above the fulcrum might be brightened and values below darkened, or vice versa.
As for the change might be because I was using levels on the individual channels by itself and with sigmoid…this may have exaggerated the effect… The perceptual space nicely spreads color but in flat images that can be like using maxrgb in filmic and you lose contrast so you need to add it back more… I think most of the time UCS looks nice but you can also use the alternate as a contrast booster …
