Nikon D5600 RAW NEF and DCP Tone Curve Option

[nonophuran]

@ggbutcher, @rogerx,
thanks for your prompt feedback.

I’ll try to clarify. In my current context (i.e. digitising colour negatives with my Nikon D750),

A. I am not trying to reach what my camera jpg engine or capture NX-D would give me, as I have practically no idea what it corresponds to.

• Unless I am missing something obvious in Nikon Capture NX-D or Nikon Picture Control Utility, I have no way of knowing what would be the most “linear and homogeneous (across all channels)” profile (or “picture control” setting, in Nikon camera vocabulary).
• stated differently: I know that, e.g., the “landscape” setting seems to process greens and blues very differently from the “portrait” setting. But, I don’t know of a way to see the insides of these settings, on an R, G, and B channel basis. Same goes for the “flat” setting - I have no way of knowing how flat it is.

B. I just want to configure the DCP options properly so I have something that’s essentially what the “linear response” option in Capture One seems to achieve, cf. this page.

@ggbutcher

So, if you were to turn off the tone curve but keep the look table enabled, you’re not likely to like the outcome…

• Agree. But the other way around, it’s much more difficult to judge. And in my case, deactivating the “look table” option gives me, apparently, reds that are closer to what I believe film was able to record (both in terms of hue and saturation).

[ggbutcher (Glenn Butcher)]

I think you need to be careful in using Capture 1 to depict a true linear RGB. We’ve discussed that here in the context of raw files vice the renditions from Capture 1 and other software. It’s evident Capture 1 is doing something prior to presenting “linear” that other software is not.

If you just select the Neutral profile in RawTherapee, you should get true “linear”. That would be just the color transform from the DCP, with no “look” transforms enabled. I don’t regularly use RawTherapee, but when I do it’s usually to compare processing to what I get from my hack software, which I KNOW delivers neutral, and it and the RT neutral are virtually identical.

Edit: Note that in using a camera to copy negatives, you’re not getting the opportunity to re-measure the original light in the scene, so the colors you can expect are all about what was originally recorded on the film. And that is rather hard to judge with the orange cast…

[rogerx (Roger)]

Ah. Now I see what you’re doing.

I have a Canon CanoScan 9000F Mark 2 USB scanner for scanning
negatives and reflective photos. I also use VueScan (on Linux), and
have purchased the lifetime option. There’s also aftermarket assorted
sized negative holders for flatbed scanners.

As already stated, use the Neutral profile, both in camera and within
RawTherapee; enabling the Color DCP settings, Tone Curve, Base Table,
Look Table. You’ll also need to acquire your lens profile, from the
Adobe Camera Raw Windows package, else you’ll have some possible minor
lens deformation on side by side comparisons.

You’ll want to neutralize the negative masking to a see through clear.
Usaully, the masking is orange in color. Once the masking is see
through, you’ll have the colors usually intended with the exception of
film aging and other anomalies.

Sounds like a lot of work using a camera, along with the problem of
trying to keep the negative film flat while photographing, along with
using a caliberated backlight.

For this purpose, is why I have a Canon 9000f Mark 2 flatbed scanner
and VueScan. Using this combination, can scan negatives, neutralize
the masking (see VueScan instructions for neutralizing negative
masking) and save into a raw file for backup. Much easier and with
reproductive results versus using a non caliberated camera setup.

I have not tried any of the devoted negative scanners. From what I’ve
heard, the recent PlusTek negative scanners (similar to the famous
Nikon scanner) now have recent VueScan Linux drivers, but I have not
verified. Looks like Kodak, after ~20 years of scanning, is now trying
to jump into this market.

I scanned many types of old family photos/negatives several years ago
using the flatbed and VueScan. Scanned to raw, then produced
TIFF/JPEGS; saved to Bluray BD-R media. I also have a Canon Selphy
CP1200 for making prints on the new durable photo paper.

Would be nice to have a function within RawTherapee to neutralize the
negative masking to clear, for your purpose.

[Thanatomanic (Roel)]

@ggbutcher

If you just select the Neutral profile in RawTherapee, you should get true “linear”. That would be just the color transform from the DCP, with no “look” transforms enabled

That’s the gist of it. If you do not care about estimating the ooc jpegs, then start with Neutral, and only enable the “Base table” for your dcp. Only when your profile does not have a base table (which is rare in my experience), should you enable the “Look table”. If both of them are available, stick to the base one, otherwise you have some arbitrary color “look” applied.

@nonophuran: Despite what @rogerx claims, you don’t need the “Tone curve”, unless you want to have a quick fix to have your image match the tone of the ooc jpeg / manufacturer’s look. In most cases you actually want to use the “Auto-tone curve” option on the Exposure tab for that, which was specifically developed for this goal and works more generally.

@rogerx

You’ll also need to acquire your lens profile, from the
Adobe Camera Raw Windows package, else you’ll have some possible minor
lens deformation on side by side comparisons.

Or you make use of the lensfun database used in RT. In many cases you don’t need Adobe’s profiles.

@rogerx

Would be nice to have a function within RawTherapee to neutralize the
negative masking to clear, for your purpose.

Not a 100% sure what you mean here, but this sounds like something a flat-field correction could do? Any discussion related to negative processing is actually better suited here where it originally started: Any interest in a "film negative" feature in RT ? - #135 by nonophuran

[priort (Todd Prior)]

@ggbutcher Asking from a point of ignorance…there are tone curve options in the exposure controls of RT and you can also enable a tone curve in the dcp profile. Is it either or??? So if I intent to go neutral in exposure and enable the curve from the dcp profile is that a standard for processing the image. I am asking as I have a new pixel phone that will save DNG files and I am trying to understand a workflow wrt the tone curve settings. I don’t think RT supports those camera’s directly but if there is information embedded in the DNG then it will try to apply that from the camera profile?? I have done a quick search and not really found any info about the pixel DNG files…I do have the dcp files from Adobe for the phone so I could apply them in color management and enable the tone curve but then I am not sure how that now relates to settings made in exposure…in essence it would appear that you could be applying the same or similar tone curve twice?? Any clarification from those of you who understand this more fully would be greatly appreciated…Thanks in advance…

[ggbutcher (Glenn Butcher)]

Multiple tone curves can be applied, but you really need to tease apart and understandtheir individual effect, IMHO. If you don’t you might just be applying two curves that neutralize each other.

I’d treat any tone curve supplied in the DNG (well 'cept for a linearization curve, that one is supplied by the camera manufacturer to render the raw data as their notion of “linear”) as discretionary, same as any Picture Control parameters

Generally, I’m a fan of as little manipulation as needed. In my software I normally use a filmic curve, but I’ve been known to put an S-shaped control point curve after it for grins. I’ve also been known to use no tone curve in my tool chain. Note that this is all before the TRC in the display or output ICC profile.

[stuntflyer (Mike)]

Where do you enable DCP Look Table in the software?

[ggbutcher (Glenn Butcher)]

Under the Color tab, Color Management:

http://rawpedia.rawtherapee.com/Color_Management#DCP_Look_Table

[rogerx (Roger)]

Towards the bottom of the right side color tab menu, and should be
just underneath the DCP profile settings. Likely need to expand the
Color > Color Management submenu.

Should see Color Management submenu, then two items named Input
Profile and DCP, under the DCP item you’ll then see Tone Curve, Base
Table, Look Table, Baseline Exposure.

[stuntflyer (Mike)]

Thanks, Roger. With my Nikon D5100 shooting RAW I use the “neutral picture control”. What would be the recommended settings under the color management menu? I’m new to RT.

Mike

[rogerx (Roger)]

I’ve documented my experience using a Nikon D5600 and RawTherapee,
acquiring raw images with a neutral profile on my small website:

http://rogerx.sdf.org/
http://rogerx.sdf.org/programming/rawtherapee_and_nikon_cameras.html

I’ve removed a lot of my ramblings and narrowed the settings to just a
few required changes, to acquire images similar looking when compared
to Nikon Capture NX-D software. All the required information is there
on the website, that I can think of, including incorporating lens
profiles and having them applied automatically. (In essence, similar
to what Nikon Capture NX-D does.) About the only thing I have not
documented or cannot do automatically, decide between sharpening
and/or using noise reduction. Usually I default my profiles to
sharpening, then go back on low-light photos and disable sharpening
and enable noise reduction.

Just don’t ask me about the other Standard, Landscape, Portrait,
profiles as I think those are custom curves, and trying to apply the
Nikon (or Adobe) profiles within RawTherapee gives some really mixed
results, including trying to import the Adobe (Camera Raw) profiles.
Aside from that, nor have I every really ever used those profiles,
except only for the neutral profile, for the past years I’ve
purchased/owned the camera. Granted, I’m not saying it can’t be done;
if one really likes one of those custom curve profiles, it can be done
either with a custom curve and/or reproducing an identical curve by
reverse engineering the curve. Just takes a lot of time and desire
for the feature.

On the flip, I think the Flat profile can be easily acquired by just
disabling the RawTherapee Color Management > DCP > Tone Curve! On the
flip, I’d never use a flat profile, as I don’t spend much time in
front of the computer adjusting images except for the usual sharpening
or noise reduction. As others have stated here, I focus upon trying
to have all adjustments made within the camera before taking the
photo, and a lot less work afterwards.

[rogerx (Roger)]

As far as the Standard, Portrait, Landscape DCP profiles; it maybe
very possible they simply enable a Color > Vibrance option with a
Pastel tone of ~20 for Standard. Landscape/Portrait may
enable/disable the Protect Skin tone options, etc.

I’m exhausted and don’t have time to compare some photos of my XRite
ColorChecker and comparing to similar adjusted photos within
RawTherapee, but stumbled upon playing with those options just now,
figured I’d at least pipe-up here with the information. The images do
look comparatively similar to those DCP profiles. Likely these
additional DCP profiles simply enable some proprietary DCP extension
only recognizable within Capture NX-D or the camera firmware, merrily
only making similar modifications rather than applying a custom tone
curve.

[Thanatomanic (Roel)]

Hmm, I’m reading some confusing things in the last couple of posts. Let’s try to clear things up. I hope this makes things clearer @priort @rogerx and @stuntflyer. Also, please read this page carefully: Color Management - RawPedia

In your camera
In most camera’s your can select profiles such as “Neutral”, “Flat”, “Vivid”, “Landscape”, etc. These only apply to the way the OOC (out-of-camera) JPEGs are produced. It has zero influence over how your raw file looks like.

Replicating the OOC look
A lot of people like the look of their OOC JPEGs, but want the power of editing the raw file. To accommodate this, the people at Adobe have developed Digital Camera Profiles (DCP). These profiles try to emulate the look of the camera’s profiles. If you’ve used Lightroom you have probably seen ‘Adobe Standard’, ‘Adobe Neutral’, ‘Adobe Vivid’ and so on. If you select one of these, you apply Adobe’s idea of how to emulate the colors and tone to get something that comes close to your OOC JPEG. It will never be perfect, because the internals of the camera are not known, but these profiles are the ‘next best thing’.

RawTherapee also supports DCP profiles, so if you like a certain type of default processing from Adobe, you can get the same starting point in RT. Just load a DCP file that comes bundled with Lightroom. On my Windows machine they can be found here: C:\ProgramData\Adobe\CameraRaw\CameraProfiles.

What’s in the box?
A DCP profile can contain a few things, but it doesn’t necessarily have all of these:

1. In most cases there is a ‘Base table’ that is used to apply a calibrated color transformation to your image. This is an essential element to convert the raw data to correct colors. If you profile does not have a ‘Base table’, make sure you select the ‘Look table’ instead.
2. In many cases there is a ‘Tone curve’. This does exactly what I explained before: it applies a preconstructed look to the tones of your image. If you loaded a ‘Adobe Vivid’ DCP file, it emulates how a ‘Vivid’ OOC JPEG would look like. Importantly: if you don’t care about replicating the OOC JPEG look, leave this box unchecked!. You’re much better off using the other tone curve tools in RawTherapee.
3. There may also be a ‘Look table’. This also applies a preconstructed look, but changes colors instead of the tones. Again, if you’re not interested in the OOC JPEG look, leave this unchecked.

The problem with profiles
The main issue with enabling ‘Tone curve’ and/or ‘Look table’, is that you have no control over what kind of modifications are made to your raw file. It may give you something aesthetically pleasing with very little work, and that may be good enough for you. But to me, that kind-of goes against the nature of wanting to process raw files and have absolute control over what happens. What you like to do is up to you.

Alternatives
There are multiple ways to adjust the tones in an image with RawTherapee. They can all be used simultaneously if you want, without any problem. You just need to know which one does what.

• The DCP Tone curve is applied very early on in the processing pipeline. Like I said, it applies a preconstructed tone curve without further control over it.
• Tone curve 1 & 2 (first tab). These give your full control over the tones of your image. You can draw a curve to your own liking. The second tone curve is there for further fine-tuning, if you cannot manage all the nuances in the first tone curve. These curves are applied after the exposure controls, but before most other available tools.
• Auto-Matched Tone Curve. If your DCP file doesn’t have an embedded Tone curve, but you still want to get something similar to the OOC JPEG, you can click this button. RawTherapee will analyse the embedded JPEG file inside your raw file (which is usually present) and estimate a tone curve from that. It puts this curve into Tone curve 1. This is a pretty smart method that works very well most of the time.

So, I hope that cleared up most of your questions and made you a little wiser in using RawTherapee.

Some individual points:

@priort

Asking from a point of ignorance…there are tone curve options in the exposure controls of RT and you can also enable a tone curve in the dcp profile. Is it either or??? So if I intent to go neutral in exposure and enable the curve from the dcp profile is that a standard for processing the image.

I hope this is cleared up now: you can enable both options if you want, but they do different things. I don’t know what you mean by “to go neutral in exposure”.

@stuntflyer

With my Nikon D5100 shooting RAW I use the “neutral picture control”. What would be the recommended settings under the color management menu? I’m new to RT.

As I explain in my post, your camera setting has no meaning for how your raw file looks like. If you want to emulate your OOC JPEG look in RawTherapee, use the DCP Tone curve and Look table, or use the Auto-Matched Tone Curve option.

@rogerx

(…) and trying to apply the Nikon (or Adobe) profiles within RawTherapee gives some really mixed results, including trying to import the Adobe (Camera Raw) profiles.

@rogerx I have no idea what you mean by that. What do you consider a mixed result? The profile should work equally well in Lightroom as in RawTherapee.

@rogerx

Likely these additional DCP profiles simply enable some proprietary DCP extension only recognizable within Capture NX-D or the camera firmware, merrily only making similar modifications rather than applying a custom tone curve.

I think this is false. DCP files are very well understood: http://dcptool.sourceforge.net/.

[gaaned92]

@rogerx if you want some insight in a dcp file you can use dcptool or dcamprof to decompile.
exemple with dcptool

canoneos5Dmarkiiicameraportrait.xml.txt (2.4 MB)

edit : @Thanatomanic already referenced dcptool!

As a tone curve is a way to modify the brightness of an image, there can be a lot of ways to apply it depending of definition of brightness. There are 6 methods in RT. I don’t know how DCP tone curve is applied in RT

[stuntflyer (Mike)]

Maybe I’m being too simplistic, but all I’m looking for is a starting point within the RT Color Management module and for its startup profile. One that will yield something close to the way I saw the image through my eyes when the photo was taken. I can always adjust the tone curves, etc. later on. Btw; I do have the icm profiles for standard, neutral and monochrome which were taken from the NXD software.

[priort (Todd Prior)]

@Thanatomanic Roel thanks for the detailed reply…as to your query from my comments I was referring to the neutral processing profile of RT which basically just demosaics and WB the file essentially…I don’t think any color managing?? So I was using that to have the most unmodified starting point from which to try and determine if it was best to use either the tone options in the exposure controls ie Tone 1/Tone2 and/or auto-matched …or would it be better not to use those and to enable the one from the Adobe DCP file…I just didn’t want to push the colors around too much and create shifts…I think I have an understanding and backed up by your comments…I’ll play a round with a few combinations and see how they turn out…thanks so much for taking the time to reply in such detail…I had read the RT section on CM but was still not sure about the interaction between setting the tone curve in CM from the DCP vs the RT settings in exposure…the added issue I think for those with camera’s supported and mine is not is that RT has some custom profiles from what I gather that it selects if it recognizes your camera…WRT the automatch is it using info from an icc tied to the preview or what is it actually basing the curve on??

[priort (Todd Prior)]

Found more info here so now I think I have the complete picture of how it works in RT and likely ART to some degree…(new) features unexplained in Rawpedia?

[rogerx (Roger)]

[quote=“rogerx, post:27, topic:4836”]

Likely these additional DCP profiles simply enable some proprietary DCP
extension only recognizable within Capture NX-D or the camera firmware,
merrily only making similar modifications rather than applying a custom tone
curve.

[/quote]

I think this is false. DCP files are very well understood:
http://dcptool.sourceforge.net/.

Thanks for the clarification.

Most of the DCP source code seems to be related/used only on
Windows/MacOS, there are apparently, at one time, instructions for
compiling on Linux/GNU systems.

Also, thanks to the person attaching a sample de-compiled DCP XML profile.

Let me re-word so that it doesn’t look like I’m poking pun at
proprietary code within my last statement, “… don’t be surprised if
one can simply enable and decrease/increase the RawTherapee “Color >
Vibrance” for acquiring a similar curve to the In Camera (or Capture
NX-D) Standard/Landscape/Portrait DCP profiles.” As I previously
stated, upon my initial viewing and without any color measuring
instruments, surely looks strikingly similar or eye-pleasingly
similar.
Again, though, I would likely never use unless I were publishing my
photos, as I’m mainly focused on accurate reproduction.

Color Management - RawPedia includes the
previously iterated information.

I think further confusion can be incurred by RawTherapee’s more
purposeful/meaningful use of DCP camera D65/D50 profiles and the
cinematic use of DCP profiles. (eg. Digital Cinema Package -
Wikipedia) In other words, the intended use of DCP, whether for
accurate recreation or changing/modifying the authentic recreation
into something more eye pleasing.

I think the DCP Wikipedia clarified correctly, whether or not the
Wikipedia article is related or unrelated to DCP camera profiles, the
clarification within the initial part of the article “to compose”
something rather than contain any content seems worded quite well.

In my opinion, I never acquired desirable results using the Adobe DCP
profiles, nor the Nikon Capture NX-D extracted DCP profiles; only
providing a “starting point” of no return. Best to achieve/duplicate
a neutral profile using RawTherapee’s DCP default D50/D65 profile,
then further modify within RawTherapee to further achieve the
Standard/Portrait/Landscape profiles.

[ChasingShadows (Karlheinz Lehmann)]

Take the neutral DCP, at least I do this with my Nikon D7200

[priort (Todd Prior)]

From what I understand the base table is a set of hue sat values designed to work with the RGB matrix coefficients to produce some colors that match what would be expected for the embedded illuminant value …many DCP have 2 illuminant values and hence tables and the other WB values are interpolated…the tone curve is used to tweak the Look of the color but produces shifts so a custom lut is included to balance that…I believe it in not recommended to turn one off or on without the other…having said that the DNG converter…has general and camera styles/profile DCP files for a wide range of cameras…essentially equal to what you could get in Adobe Camera Raw and you can see the nuts and bolts of these using the free Adobe DCP profile editor including adding embedded curves of your own design…so the comment above about not being able to control the output is not entirely true on that front when using DCP files and a review of the discussion of profiles on the dcamprof website illustrates the workflow to create DCP files to achieve a certain look…so there is indeed some tweaking that can be done esp if even after you calibrate things maybe you say don’t have exactly the red tone you are looking for or you want to tweak your colors a bit for effect you can visually edit your DCP file and save it using the Adobe editor…not for everyone but some may want to play around with it…