Camera color profiles

Hello everyone !

I just bought a Nikon D780 and am starting to develop my .NEF files with RawTherapee 5.9. The color rendering seems correct and I get very good results, similar to what I got with the .NEF files of the Nikon D700 that I was developing, at the time, with RawTherapee 4.2.

I would like to ask your opinion about the color profiles of the cameras. Several years ago, when I developed my .NEF photos with RawTherapee 4.2, I got a decent color result. By the way, I will tell you that I much preferred the color rendering of RawTherapee to that provided by Lightroom ! If the RawTherapee at the time (4.2) developed my .NEF files so well, it was because it had the color profile of the camera, namely the Nikon D700, in the dcpprofiles folder: Nikon D700.dcp. There was also a color profile file in the /iccprofiles/input folder: Nikon D700.icc. By the way, I never understood why two color profile files were needed, if someone can explain to me…

When I got my Nikon D780, the first thing I saw was that my old version of RawTherapee was unable to develop .NEF files from this camera. And for good reason: the Nikon D780 did not exist at the time of this old version, and therefore the color profiles of this device did not exist in the color profile folders of the cameras either.

So I installed the latest version of RawTherapee: 5.9. The first thing I did was look in the dcpprofiles folder to see if my Nikon D780’s color profile was there. Disappointment: no, it is not there. Misunderstanding too, since the Nikon D780 was released in 2020, and so the designers of RawTherapee have had plenty of time since then to create a color profile for this device. There is NIKON D700.dcp, NIKON D750.dcp, NIKON D800.dcp, but there is no NIKON D780.dcp profile. In the folder /iccprofiles/input it does not appear there either !

I was expecting the latest RawTherapee 5.9 not to be able to develop the raws of my new device, but it is! .NEF files are recognized, and development gives correct colors.

Can someone explain this mystery to me ? How can RawTherapee 5.9 correctly develop raw files from a device that it does not have color profiles ?

Thank you for your answers !!!

Someone has to photograph a color chart and submit the profile, and it seems nobody has done that.

You can borrow a dcp from adobe dng converter, which generally has them.

1 Like

The default camera profile in RawTherapee isn’t a file, it’s an entry from camconst.json. Well, maybe, I think the old dcraw adobe_coeff table may be the source of last resort. Here’s a link to the camconst.json entry for the 780:

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json#L1892

Oh, welcome to the forum!

Yes, I went to Adobe to see the .dcp profiles of the Nikon D780. But which one to choose ? Here is the list of what they have in the following folder :

C:\ProgramData\Adobe\CameraRaw\CameraProfiles\Adobe Standard

=> Nikon D780 Adobe Standard.dcp

And then there are also other color profiles in the folder

C:\ProgramData\Adobe\CameraRaw\CameraProfiles\Camera\Nikon D780

Nikon D780 Camera Flat.dcp
Nikon D780 Camera Landscape.dcp
Nikon D780 Camera Monochrome (Green Filter).dcp
Nikon D780 Camera Monochrome (Orange Filter).dcp
Nikon D780 Camera Monochrome (Red Filter).dcp
Nikon D780 Camera Monochrome (Yellow Filter).dcp
Nikon D780 Camera Monochrome.dcp
Nikon D780 Camera Neutral.dcp
Nikon D780 Camera Portrait.dcp
Nikon D780 Camera Standard.dcp
Nikon D780 Camera Vivid.dcp

So which one to choose for RawTherapee ?

Thank you for your explanation ! However, I don’t understand it ! If camera profiles are hard-coded in a json file, then what are the .dcp profiles in the dcpprofiles folder for ? My computing skills are limited, and I have no idea how RawTherapee works internally.

welcome @preuthier

I suggest to start with the neutral ā€œNikon D780 Camera Neutral.dcpā€. Are you aware of the video of @Andy_Astbury1 where he is giving the explanation how and why:

FREE DCP Camera Profiles for Raw Therapee - Linux Users TOO! Plus Colour Correction Regions - YouTube (at about 10.30)

1 Like

There’s a lot to know about all this, but I’m going to answer your specific questions, see where that goes…

Firstoff, the JSON file is simply a convenient way to collect the data for a very large number of cameras. If you had a file for each camera RawTherapee can support, you’d have a whole bunch of files. If you want to see that in action, download and install the Adobe DNG Converter; in its ProgramData directory (Windows, IIRC), you’ll find literally thousands of camera profile files, all DCPs.

Now, the essential reason to have DCP files for your camera in addition to the matrix in camconst.json is that DCP files support the color conversion a little differently. The nine camconst numbers are for a specific white point, D65, roughly corresponding to a cloudless day at noon. So strictly speaking, those numbers are tuned to convert colors in images shot in daylight. However, in practice using those numbers for images recorded in other light does not yield bad results. But, if you’re interested in specific color for things like reproduction, DCP profiles contain a mechanism that’ll produce a matrix tuned to a specific color temperature. AFAIK, RawTherapee is right now the only raw processor we discuss here that can use DCPs this way.

I’m not a RawTherapee developer, so I can’t answer why the don’t have a comprehensive set of DCPs. But, if you want such for your camera, just download and install the Adobe DNG Converter and pull the files you want out of their collection. Me, I’ve done that for a couple of my NIkon cameras, but in most cases I prefer using the simple dcraw matrices - they do a good job with most images. The exception is with extreme hues; then I go to other ways of making camera profiles, a rabbit hole I won’t pull you into (yet!)…

I think this is your answer…Why reinvent the wheel… Take what you need from Adobe. RT provides the support to do so and off you go… There are also other tools to tweak these profiles in a custom way so all options are on the table.

Also in a way RT has some custom profiles and color management options but where DCP are concerned its likely best to stick IMO to the Adobe ones so as not to get constantly questioned why the image looks different in RT with ā€œtheirā€ DCP file… and if the Adobe look is not suitable the user can then proceed to tweak away…

I personally do not trust Adobe profiles - I’ve seen lots of evidence that they may ā€œcookā€ some camera-specific looks into their profiles. That said, RT ignores the embedded tone curve by default so that should mitigate MOST of that concern.

The official process for generating an RT color profile is outlined here:
How to create DCP color profiles - RawPedia - this process will be changing slightly in the next release, but that’s the appropriate process for 5.9

This requires shots taken in sunlight and also via an incandescent lamp (NOT an LED or CFL light!!!) of a ColorChecker target.

As to why RT may not include profiles for any given camera - all profiles are generated from user-submitted data. The RT team, being volunteers, can’t buy every camera on the market and characterize them.

Thank you all for your explanations. This allows me to see a little more clearly and understand why RawTherapee 5.9 develops my D780’s raws very well without needing its color profile.

@ paperdigits Mica Regular
@ ggbutcher Glenn Butcher

I installed Adobe DNG Converter, but I don’t see an associated folder in my ProgramData folder. All the DCP color profiles I found are located in a C:\ProgramData\Adobe\CameraRaw\CameraProfiles folder. And, indeed, there are more than a thousand of them!

@ marter Martin Werner

Yes, that’s what I would have thought to do, take the neutral profile at Adobe for the D780. I also did the test, I copied this file in the dcpprofiles folder of RawTherapee, and I looked at what happens. I didn’t see a noticeable difference, so I removed it, since RawTherapee develops my raws very well without this profile.

P.S. very good video, but I’m French speaking and I had to put the translation subtitles on Youtube, so it’s difficult to follow the changes in the settings and the changes in the image at the same time. There are many very well done videos on RawTherapee, but everything is in English and I regret that I am not an English speaker, because there is nothing equivalent in French. Or they are tutorials for beginners, which do not go into the depths of things.

@ Entropy512 Andy Dodd

I’m also wary of Adobe profiles. When we look at the neutral profile without any correction in Lightroom, it gives a flat image… Moreover, I developed the raws of my Nikon D700 for a long time with RawTherapee 4.2, which had the color profile of the D700. I found the color rendering in RawTherapee much more accurate than in Lightroom, which tends to give me a yellow cast when I warm up the image a bit. And the rendering of the skies was infinitely better than in Lightroom. In 2014 when I gave up Lightroom for RawTherapee, I said to myself: wow! what beautiful pictures!

I now also understand why not all color profiles of all cameras on the market can be delivered by the designers of RawTherapee.

As for generating a color profile for my Nikon D780, it’s really too delicate operation that I won’t get into.

1 Like

Just a disclaimer from my end… I wasn’t intending to imply that Adobe profiles were or should be the preferred choice , but rather are an available acceptable reference point. I think it is nice that RT supports these profiles so that you can easily evaluate how your images look in RT with or without any custom profiles or color adjustments using the Adobe one for context… in many cases showing you that you are doing a better job😁

It’s not too hard, but does require acquiring stuff, particularly a color target. Doing the target shot is a bit fiddly, need to get a glare-free capture.

If you want to try out the process without buying one, you can go get the D780 studio shot raw from DPReview and crop the ColorChecker target from it. I’ve made a few profiles doing that, and it works okay…

But really, for the majority of shots the built-in profile of most any raw processing software will do just fine, so enjoy it. I was in the same boat until I shot images at a local theater where they were using blue LED spots as wall accents, those extreme hues suffered under the simple matrix conversion used in most default profiles. When you get to that point, come back here and I’ll drag you down a profiling rabbit hole… :laughing:

Actually, the chromatic tools in darktable do a good job alternatively, from what I’ve seen others post here.

If you want to ping someone, type the at symbol followed by the first few letters of their use name, then you can select the user name from a pop up,

@preuthier and you can see the user handle is highlighted.

Hey guys. Is it possible to convert RawTherappe’s camera profile to Lightroom? Without taking pictures of chart etc only using data from camconst.json? Because Adobe’s profile for Nikon Z cameras is a rubbish and RawTherapee’s profile is excellent

Hey and welcome! Converting the camera primaries into something lighroom can use will likely not be compelling, you need more information than that to make a good ICC profile.

I won’t speak for everyone, but we like to use and promote free and open source software, so making things for Light room is generally not compelling.

Why not use this as an opportunity to learn RawTherapee?

Sometimes it is necessary to use another software. Different tasks, different tools. It is what it is. RT’s color profile and converting out-of-the-box into JPEG is even better than NX Studio. And it’ll be nice to have standalone color profile. But thanks anyway.

If Lightroom takes ICC files as camera input profiles, making one from the camconst dcraw_matrix can be done. I do it in my rawproc raw processor, but I don’t have code there to write it out to a file. Your inquiry prompted me to finish something I’ve been meaning to do, capture that code in a command-line program.

Just wrote and tested it, adobecoeff2icc.cpp. It uses the LittleCMS color management library, the psuedoinverse function from dcraw.c, and some helper functions from rawproc. Here’s the entire program:

#include <string>
#include <vector>
#include <algorithm> 
#include <lcms2.h>

static cmsCIEXYZ d65_media_whitepoint = {0.95045471, 1.0, 1.08905029};

std::vector<std::string> split(std::string s, std::string delim)
{
	std::vector<std::string> v;
	if (s.find(delim) == std::string::npos) {
		v.push_back(s);
		return v;
	}
	size_t pos=0;
	size_t start;
	while (pos < s.length()) {
		start = pos;
		pos = s.find(delim,pos);
		if (pos == std::string::npos) {
			v.push_back(s.substr(start,s.length()-start));
			return v;
		}
		v.push_back(s.substr(start, pos-start));
		pos += delim.length();
	}
	return v;
}

//psuedoinverse, from dcraw.c:
void pseudoinverse (double (*in)[3], double (*out)[3], int size)
{
 double work[3][6], num;
  int i, j, k;

  for (i=0; i < 3; i++) {
    for (j=0; j < 6; j++)
      work[i][j] = j == i+3;
    for (j=0; j < 3; j++)
      for (k=0; k < size; k++)
        work[i][j] += in[k][i] * in[k][j];
  }
  for (i=0; i < 3; i++) {
    num = work[i][i];
    for (j=0; j < 6; j++)
      work[i][j] /= num;
    for (k=0; k < 3; k++) {
      if (k==i) continue;
      num = work[k][i];
      for (j=0; j < 6; j++)
        work[k][j] -= work[i][j] * num;
    }
  }
  for (i=0; i < size; i++)
    for (j=0; j < 3; j++)
      for (out[i][j]=k=0; k < 3; k++)
        out[i][j] += work[j][k+3] * in[i][k];
}

const cmsCIExyY cmsCIEXYZ2cmsCIExyY(cmsCIEXYZ in)
{
	cmsCIExyY out;
	double s = in.X+in.Y+in.Z;
	out.x = in.X/s;
	out.y = in.Y/s;
	out.Y = in.Y;
	return out;
}


// make a linear D65 profile from a dcraw adobe_coeff entry, 
// e.g., D7000: 8198,-2239,-724,-4871,12389,2798,-1043,2050,7181
// also takes a json array, e.g., [ 8198, -2239, -724, -4871, 12389, 2798, -1043, 2050, 7181 ]
cmsHPROFILE makeLCMSAdobeCoeffProfile(std::string adobecoeff)
{
	double in_XYZ[3][3], inverse[3][3], out_XYZ[3][3];

	//if json, remove the extraneous characters;
	adobecoeff.erase(std::remove(adobecoeff.begin(), adobecoeff.end(), ' '), adobecoeff.end());
	adobecoeff.erase(std::remove(adobecoeff.begin(), adobecoeff.end(), '['), adobecoeff.end());
	adobecoeff.erase(std::remove(adobecoeff.begin(), adobecoeff.end(), ']'), adobecoeff.end());
	
	std::vector<std::string> mat = split(adobecoeff, ",");
	for (unsigned i=0; i<3; i++) {
		for (unsigned j=0; j<3; j++) {
			unsigned pos = i*3+j;
			float coeff = atof(mat[pos].c_str());
			if (pos < mat.size()) {
				if (abs(coeff) > 10.0)
					in_XYZ[i][j] = coeff/10000.0;
				else
					in_XYZ[i][j] = coeff;
			}
			else {
				in_XYZ[i][j] = 0.0;
			}
		}
	}

	pseudoinverse(in_XYZ, inverse, 3);
	//because pseudoinverse delivers it rotated:
	for (unsigned i=0; i<3; i++)
		for (unsigned j=0; j<3; j++)
			out_XYZ[i][j] = inverse[j][i];
	
	cmsHPROFILE profile;
	cmsCIExyYTRIPLE c;
	cmsCIExyY cw;
	cmsCIEXYZ p, w;
	cmsToneCurve *curve[3], *tonecurve;

	cw = cmsCIEXYZ2cmsCIExyY(d65_media_whitepoint);

	p.X = out_XYZ[0][0]; p.Y = out_XYZ[1][0]; p.Z = out_XYZ[2][0]; 
	c.Red = cmsCIEXYZ2cmsCIExyY(p);
	p.X = out_XYZ[0][1]; p.Y = out_XYZ[1][1]; p.Z = out_XYZ[2][1]; 
	c.Green = cmsCIEXYZ2cmsCIExyY(p);
	p.X = out_XYZ[0][2]; p.Y = out_XYZ[1][2]; p.Z = out_XYZ[2][2]; 
	c.Blue = cmsCIEXYZ2cmsCIExyY(p);

	tonecurve = cmsBuildGamma (NULL, 1.0);  //hardcoded linear, for now...
	curve[0] = curve[1] = curve[2] = tonecurve;

	profile = cmsCreateRGBProfile (&cw, &c, curve);
	
	if (profile) {
		std::string descr = "adobe_coeff linear profile";
		cmsMLU *description;
		description = cmsMLUalloc(NULL, 1);
		cmsMLUsetASCII(description, "en", "US", descr.c_str());
		cmsWriteTag(profile, cmsSigProfileDescriptionTag, description);
		cmsMLUfree(description);
	}

	return profile;
}

void makeICCProfile(cmsHPROFILE hProfile, char *& profile, cmsUInt32Number  &profilesize)
{
	if (hProfile != NULL) {
		cmsSaveProfileToMem(hProfile, NULL, &profilesize);
		profile = new char[profilesize];
		cmsSaveProfileToMem(hProfile, profile, &profilesize);
	}
}

void err(std::string msg)
{
	printf("Error: %s\n", msg.c_str());
	exit(0);
}

int main(int argc, char **argv)
{
	char * profile;
	cmsUInt32Number profilesize;
	
	if (argc < 2) err("No adobe coeff string (a,b,c,d,e,f,g,h,i)");
	if (argc < 3) err("No destination file name");
	
	cmsHPROFILE cmspprof = makeLCMSAdobeCoeffProfile(std::string(argv[1]));
	makeICCProfile(cmspprof, profile, profilesize);
	
	FILE* f = fopen(argv[2], "wb");
	if (f) {
                fwrite(profile, profilesize, 1, f);
	        fclose(f);
        }
	
}

I didn’t say it would be easy… :crazy_face:

I think but I am not sure that it only will use ICC for display and not as camera input profiles.

1 Like

LR accepting as a camera profile xmp, dcp, lcp and zip(?) formats.

DCP… converting an adobe coefficient set to, say, a DCP ColorMatrix tag wouldn’t be hard, but there’s a lot more to DCPs than that, they’re really a different workflow:

https://rawtherapee.com/mirror/dcamprof/camera-profiling.html#dng_profiles

xmp and lcp formats have specific uses. I have no idea what they’re looking for in a .zip…