Is it worthwhile struggling with input profiles?

It looks to me to be a color difference between the first image and the other two, but between the second and third images it well could be a tone difference. I’m in and out of a few home projects right now, otherwise I’d load each in software to look at the histograms.

Please note that, within the confines of profiles, you have two distinct operations possibly going on, a color conversion and/or a tone conversion. Those two need to be teased apart to understand the real difference between the effect of each profile. And then, we mess with tone again in places like filmic or basecurve, or automatch. And then, aggressive tone manipulations start to work their way into affecting color. @Morgan_Hardwood’s advice is good; using input profiles to mess with the ‘look’ is challenging, think twice about using them if you’re not going to dig into their weeds…

I like the default one all right, except for the sky, which looks a bit bland to me (too much magenta and not saturated enough). But the faraway mountain haze (esp. in the shadows) is the most neutral of three, which I like. The second one has a typical Nikon blue sky – looks a bit fake to me (too much green and too saturated), but that’s really personal aesthetics – many people like their skies like that. The shadowy haze is the coldest of the three. I’d pick the dcp version of the sky (maybe back off the saturation slightly), and maybe open up the shadows in the foreground bush/trees (it looks the most contrasty of the three in that area).

This is darktable 2.6.2 with minimal processing:

  • turn Base Curve off
  • set Exposure to +1.2 stops (guessed at by assuming the hill in the centre should be about a mid-tone)
  • no ICC profile, just using the built in matrix
  • keep the white balance on “Camera”

You get the same (I think) results in RawTherapee using the Neutral processing profile and a +1.2 exposure adjustment.

Hi’ Everyone

Thank you for your feedback on “A closer look on the sky and the Sibillini Mountains”.

@ggbutcher I’m always looking forward to read your input and I would love to study the result of your examination of the histograms if and when you get the possibility to have a closer look.

Based on yours and others input I think that we can conclude the discussion as follows:
“No, it is not worthwhile struggling with input profiles. Go default and take it from there!”
Correct?

The above statement seems to be very close to:

“No, it is not worthwhile struggling with input profiles. Go neutral and take it from there!”
By neutral I mean: RT – neutral processing profile, DT – base curve turned off.

If you go neutral then your starting point is further away from the dcp-result, which would you be my ambition for the editing result. This is a disadvantage, is it not?

@sankos Awesome comment: “this is a typical Nikon blue sky” and yes, I like the sky but I prefer the dcp sky.
In fact I liked the default result very much until I had a closer look on the sky. I agree that the mountain haze looks natural in the default version. How to transform the default sky without changing the mountain haze using RT? I would apply a mask using DT.

@paulmiller I like your simplistic approach. I have a tendency to think that I must be missing something not using the advanced tools. This thread has shown that this is not always the case.

No wonder that you have to increase the exposure when you turn off the base curve. The photo was deliberately under-exposed and the base curve (or the profiles) lightens the whole image and boosts the highlights.
It’s even more simplistic not to turn the base curve off, then you just have to open the photo in DT. I like the result even better. What do you think?

I’ve always been a “nuts and bolts” sort of person, wanting to understand the specifics of whatever mechanism is placed in front of me. In their efforts to make the rather ungainly mechanism of digital photography accessible to most folk, a variety of insulating layers have been developed to facilitate such. I think sometimes these layers work at cross purposes, and as a result confound our understanding.

To attempt to lay-flat the mechanisms we’ve been discussing, here’s what I’d call the baseline processing needed to get to what is probably neutral in most of our understanding:


Of note is that this is a 14-bit raw, a somewhat technical tidbit we’ll talk about later.

So, if you look at the commands pane at the top left, the first thing in the chain is a group that contains all of the processing needed to get to a RGB image:

colorspace:camera,assign The first thing I do to any raw is to assign its color profile. One would think, oh no tone manipulation going on here, but indeed, what this assignment will be used for later is to convert the image to a gamut and tone suitable for display or output. Note the “CMS:display” at the bottom right, in the status bar; in my software this means that whatever tool you select for display will be converted to the display gamut AND tone, which is approximately a gamma curve of 2.2. More on this in a bit.
subtract:camera This is the simple subtraction of the camera-supplied value from each and every R, G, and B value of each pixel. Not all cameras have such a value in their metadata, this one’s is 1008.
whitebalance:camera Here’s the first place we really mess with the original measurements of light, each R, G, and B value in each pixel is multiplied by a camera-provided number, for this image those multipliers are 1.203125, 1.000000, and 2.121094 respectively. These numbers are what some previous mechanism determined it would take to make all three channels of a neutral pixel “neutral”, that is, R=G=B. Not strictly germane to the current tone discussion, but now the original measurements (well R and B, G was not really changed with 1.0) have lost their relationship to the presented energy.
demosaic:half This is my default demosaic for proofing. Normally I’ll change it to something better, but for this I’m leaving it as this particular demosaic does not change the R, G, and B values, it just uses them as-is in adjacent pixels.

Okay, now to the crux of the illustration. We’ve talked extensively about tone curves here, but not about the fundamental reason for doing them. That would be to put the camera’s notion of white at the value where the display or output will render it as such. The raw data, as 14-bit values, doesn’t align to any of these boundaries in current technology (most displays until recently have been 8-bit, as is the prevalent JPEG encoding), so, some kind of “normalization” is required. I usually do this here in the chain with a blackwhitepoint operator built for this purpose, but in this case I replaced it with a curve to graphically show the transform:

curve:rgb,0.0,0.0,53.0,255.0

Not “curvy” at all, this straight-line transform simply scales the data from its camera-defined limit to the upper end of internal data container in the software, which is used to anchor “display white”. The exposure tool in other softwares would do a similar thing, but you’d have to play with it a bit to get the highlights to just push up against “display white”. Interestingly, this operation doesn’t change what i think most consider the data’s “linearity”, as the relationships of the numbers are still such that one is “twice the size” or “half the size” of another, reflecting their original relationship with respect to the light energy.

So, to what we see here, how many manipulations of tone have happened? Well, in the toolchain, really none, ignoring the abomination that is white balance. At the conclusion of the curve, the data is scaled but still “linear” But, there’s one non-linear tone manipulation applied to get the displayed image, that being the camera → display profile transform. Here’s a graphic of such a tone curve:

tonecurve2

This is not a “correct” gamma curve, I normalize it to top-off at white, but you should get this idea - the lower values of the image are lifted, and that lift tapers off for the upper values. After such a curve is applied to “linear” data, it’s no longer “linear”.

Okay, now to the crux of the matter, “Is it worthwhile struggling with input profiles?” Well, first off, they’re needed in order to assign the proper color characteristics to the camera’s raw data. But a profile has two sets of data, one for color, and one for tone. In the above illustration, the assigned camera profile says the data is “linear”. effectively has an “identity” tone transform, that is, one that doesn’t change the data. So all it does is provide the color information for later mapping gamut to the display or output. Later, that “linear” assertion is used to do the display transform. So, if you’re going to use such a camera input profile to change tone, you’re actually telling that later display/output transform that the data is something other than what it really is, which is “linear”. Not a very intuitive way to work with the data.

So, I think, most “camera look profiles” are actually meant to be applied after the original color/tone assignment. But, if you were to use a camera profile to also change the tone of the resulting image, you need to do so considering whatever other tone curves you might decide to apply later, such as automatch, filmic, or even a manual curve. To my mind, I’d rather do only one such tone curve, one that I can control through its entire range. And, I really only want to do one because the data analyst in me doesn’t like whipsawing data back and forth with multiple such operations.

The other thing to point out is that profile-based tone curves can also be LUTs, which don’t have to express a “nice” transform. That is, a LUT can do dis-continuous things between adjacent values. Heaven help us there…

Soooo, @obe, I haven’t done the histogram inspection of your images yet, will get to those later. I have been thinking of such an illustration of the raw processing chain for some time now, and it just seemed like this thread was ready for me to inflict it upon… :smile:

Edit: sometimes we write things before we finish our coffee, note the strikeout…

5 Likes

I am, too, but I need to spend most of my time on non-photographic problems; I learn so much from your comments. I hope you write a book someday to go along with version 1.0 of your program…

Just to add one more data point: I like to process my raws from a “neutral” starting point, but I want the colors to be accurate. So, I use the “muted” or “natural” versions of my camera’s DCP from Adobe, but I only use the base and look tables, not the tone curve. This gives me something close to RT’s “neutral” profile, but with more accurate colors.

1 Like

I often use this Firefox extension to quickly check an image’s histogram: https://addons.mozilla.org/en-US/firefox/addon/histogram-viewer/

1 Like

Hi’ @ggbutcher and @mbs

Thanks once again for another amazing feedback!

Mee too! And I might add that I sign up for your next class covering color and tone……:blush:!

Interesting, I will try it out in RT. Unfortunately DT doesn’t accept dcp profiles.

The ability to turn off the dcp tone curve in RT is a great feature and as you say, allows you to get more accurate colours. That, plus the ability to use the Exposure module tone curves in perceptual mode gives the best result colour-wise of anything I have tried to date.

1 Like

Rt’s neutral profile is not supposed to bring accurate colors. For that you need a camera profile (ICC or DCP).
In RT, ICC camera profile is a legacy.
Camera profile (DCP) is automatically selected if one is shipped with RT for your camera.
In other case, you can select a standard DCP from Adobe or if you are not satisfied, create one (see http://rawpedia.rawtherapee.com/How_to_create_DCP_color_profiles and https://www.ludd.ltu.se/~torger/photography/camera-profiling.html for all arcane technical details).

note: The DCP tone curve should not be selected.

Use of DCP is very simple and required to get accurate color at the beginning of processing. ( you need also white balance which is more tricky)
The display of those colors is an other story.

That is an artistic choice

You are right, but for the starting point I personally prefer to use the perceptual mode. We are fortunate to have the choice.

It’s not only an artistic choice. The perceptual tone curve is much slower than the non perceptual tone curve. That may be an issue when processing a large amount of files in queue.

1 Like

I have an Adobe DCP landscape profile originating from the DNG converter. I used the profile in some of my examples in this thread when using RT.

The DCP menu contains a number of check boxes: Tone curve, Base table (greyed out), Look table and Base line exposure (checked in my examples).

Checking Tone curve and Look table on/off in results in the following in combination with the RT neutral processing profile:

dcp%20examples

What do you think? Comments and “explanations” will be much appreciated……

RT 5.7 allows you to apply Hue and Luma masks in its Colour Toning module. Here’s a screenshot showing the effect of putting blue to the lightest areas of the photo with a luma mask:

If you apply the Adobe DCP, it is probably for the look it offers (you prefer it to the default colours). The Adobe tone curve is pretty universal too, so I usually leave these two on, but if I need aggressive highlight recovery I might disable the DCP tone curve and apply my custom tone curve which suits the effect I’m striving for.

Woah there, cowboy…

@Morgan_Hardwood I’d appreciate it if you could enlighten me…

I’ve found that if I don’t use my camera’s DCP, then often the colors I get are different both from my recollection of the scene and the camera’s JPG. Even worse, they often look “wrong” – green becomes yellowish, red becomes purple, etc, especially for colors that were very intense (saturated?) in the live scene.

When using a DCP, I run into these problems much less frequently.

Furthermore, I like to start from a low contrast, low saturation image and add contrast and saturation to my taste. That’s why my default profile is “neutral”, with the addition of the DCP (but without the tone curve), and a few other things thrown in there, such as sharpening.

So far I’ve been getting good results with this profile, but if it could be improved, or if I’m doing something suboptimally, please let me know…

I feel there is still a basic misunderstanding here.

Do not confuse “processing profiles” with “input profiles”. RawTherapee’s default processing profile for raw photos, as well as the “neutral” processing profile, both use an input profile - a simple color matrix at the very least, and all the other stuff a full-fledged DCP contains (ColorMatrix, ForwardMatrix, HueSatMap, LookTable, ToneCurve) if one exists.

Also worth bringing up, to throw a monkey wrench in the works and complicate things more, as anyone who’s used tone curves in RawTherapee knows, the same curve shape can lead to drastically different results based on the tone reproduction operator, the “curve mode”. Accurate (in what sense?) vs aesthetically pleasing. So there’s also that to consider.

1 Like

Thanks! I think I understand that, and I know that at the very least a simple matrix from dcraw will be applied.

But, I wanted to reply with some actual examples of color differences with and without Adobe DCP files for my camera, and, to my surprise, things are different from they were three years ago – whether it’s dcraw, RT, and/or something else I don’t know. I revisited some old files where I recalled having color troubles, but I couldn’t reproduce them. Here are a couple of examples where, three years ago, the colors without DCP would have been wrong (both “not as pleasant” and “not as I remember them”). Left-hand side is “camera standard”, and right-hand side is with Adobe DCP Standard for Olympus EPL-5. Nothing has been done to the images except applying the Neutral profile and selecting the DCP (look and base only).

So, I’m about to backtrack: for my camera on RT 5.7, I actually prefer RT’s standard color matrix. The DNG seems to produce slightly yellower greens and bluer reds and purples.