[Questions] How to choose a DCP profile (if at all)

Is the camera supported … could that not be the issue if it is not then its not correctly handling the wb data??

I guess the release notes for 5.1 say added improved support for your camera so I would think it should grab the right values…

image

So when you look at the exif data for all those photos do they have different wb and RT is not picking that up??

As you saw, it was in the release notes. It has an entry in camconst.json, with a C ranking (better than nothing). This was actually one of my main criteria when choosing a new camera.
Strangely enough, it shares its entry with the R8:

camconst.json excerpt
{ // Quality C
    "make_model": ["Canon EOS R6m2", "Canon EOS R8"],
    "dcraw_matrix": [9539, -2795, -1224, -4175, 11998, 2458, -465, 1755, 6048],
    "raw_crop": [
        {"frame": [6188, 4120], "crop": [154, 96, 6024, 4024]},
        {"frame": [3936, 2612], "crop": [156, 96, 3780, 2516]}
    ],
    "masked_areas": [
        {"frame": [6188, 4120], "areas": [4, 4, 4116, 150, 4, 150, 92, 6184]},
        {"frame": [3936, 2612], "areas": [4, 4, 2608, 150, 4, 150, 92, 3932]}
    ]
}

No occurrence in dcraw.cc I think, however.

I’m still not sure which metadata field(s) is / are relevant, but at the very least the Measured RGGB field varies (perhaps not as much as one would expect) and yet I get the same temp and tint in RT (but not in ART). And even for pictures from other R6m2 owners, ART’s balance is notably different than RT’s.

I’ll try with other modes than “Flexible Value” just in case. Forgot to do that earlier with the penguin plushy…
Edit: Done. No luck. Shutter speed priority, Aperture-priority, Manual or even the full auto thingy, all yield the same old “5257-and-1.043” balance (in RT).

Huum in RT’s metadata viewer, I see something like:
Measured Color: 12 623 1024 1024 590 0 (in the “Canon” group)
instead of the
Measured RGGB: 623 1024 1024 590 (in the “MakerNotes” group)
from exiftools. So basically the same values but sandwiched between two extra ones (seemingly always 12 and 0). But perhaps that’s normal.

Edit: ART shows a greater number of metadata fields than RT in the “Canon” group, for a given CR3. I see for example the following additions (not exhaustive at all, mostly citing those that may sound related to color issues):

  • AmbienceInfo
  • LightningOpt

Same weird 12–0 Measured Color sandwich on both sides, though.

Can you run the canon software…what does it provide for wb …

I thought about that but it’s not cross-platform, etc. :cry: If anyone can, feel free to try on the raw from before: https://discuss.pixls.us/uploads/short-url/knqoX0Na100zNPuJVYjDVRNiWgt.CR3
I wouldn’t be surprised if it gave values different from RT’s and ART’s, at this point. :rofl:

This file has good data…as far as I can see… more like what I would expect

image

So wb co-eff of 1.623 1 2.023 CT 3794 from camera data

Darktable opens it with a warning of less than full support but it looks fine and the wb values are in the same ball park as the camera data

1.585 1 1.984 CT 3883

RT

is 1.879 1 1.507 CT 5257

So pretty close to the daylight settings from the camera data as opposed to as shot…

1 Like

Thanks for checking! :bowing_woman: And you see, this “5257”, it pops up for all my CR3s! :laughing: So my RT installation is not more messed up than yours, I guess.

Indeed, the issue becomes less noticeable on pictures shot in daylight.

I just think you will have to wait for the camera to be fully supported and then you will be able to get back to your original task of sorting out your dcp work flow. Or maybe get one of those small gray/white cards you could put on a key chain so it’s handy and then you can take your wb reference shot

I do have dozens of pics from the older camera waiting to be processed in the meantime, but if I can take shortcuts that’d be nice. It seems hard to predict how fast support for each camera will come or increase.

The “copy values from ART” method is not that bad, actually. But I’ll retain your idea. Haha for a moment I was like “the camera itself is black, that could serve as a neutral patch”, then realized that it would have a hard time shooting itself. Well, maybe the lens cap, though? xD

In an attempt to reassure myself (“Did I pay several k€ for a camera whose raws I won’t be able to process correctly, despite the precautions I thought I had taken?”) and remind myself that this is supposed to be a fun hobby, I tried going through the whole RT process on those damned flowers from the first post. I used the Adobe Neutral profile, without its curve and with its look table, and brutally copied white balance values from ART. This yielded something fairly close to the thumbnail (except for contrast and stuff), and from there I did my own stuff:

Still not the picture of the year (I kinda like it, though, to be honest, and it became a symbol of my struggle, haha), but at least I proved myself that I could end up with something that suits my tastes better than the embedded JPG, while still using RT.

I came across distortion and chromatic aberration oddities, but I’ll refrain myself from asking questions about that here. I’ll create a dedicated topic one day (and do my own research beforehand) if I’m motivated enough. :sweat_smile:

I’m late for bed yet again.
I think the info you (priort) rounded up and your tests (thanks again – now I know my CR3s are not that weird) are enough for a dedicated GitHub issue to be opened, but this will have to wait. :yawning_face: Plus, maybe some users from the other side of the globe will bring valuable insights while I’m asleep.

1 Like

RawTherapee is not able to read the camera white balance for some newer Canon cameras. There was a topic on this, but nothing came out of it due to a lack of suitable sample image. At least you have provided one now and I can finally go about adding the support for reading the white balance.

2 Likes

Oh, sorry, I missed that topic, probably because it was about the R7.
That’s cool! So you already have this in your radar and don’t need me to summarize that stuff in a GitHub issue? (But then I’m not sure how to know when this gets fixed.)

In case that helps, here are raws for three of the penguin tests from earlier (I exclude the “custom WB” one because I did not use that properly so I have no idea what it actually did):

  $ exiftool -WhiteBalance ./*.CR3
======== ./r6m2-wb_penguin_awb.CR3
White Balance                   : Auto
======== ./r6m2-wb_penguin_fluorescent-white.CR3
White Balance                   : Fluorescent
======== ./r6m2-wb_penguin_forced-4000k.CR3
White Balance                   : Manual Temperature (Kelvin)

r6m2-wb_penguin_awb.CR3 (11.0 MB)
r6m2-wb_penguin_fluorescent-white.CR3 (11.0 MB)
r6m2-wb_penguin_forced-4000k.CR3 (10.9 MB)

Tell me if you also want the uncompressed versions. Dunno if that changes anything on your side. I assume that a compressed CR3, once decompressed, has roughly the same metadata structure as the never-compressed ones.

Good luck. :bowing_woman:

I think I found the problem! RawTherapee is indeed reading the camera’s daylight white balance instead of the as-shot white balance. With the fix, the white balance multipliers match the levels reported in the metadata.

1 Like

That was fast! :smile: So priort’s hunch about daylight was right.
I’ll continue that discussion on the pull request, then. The white balance stuff has hijacked that topic more than enough. :sweat_smile:

I wanted to write a TL; DR in my initial post for potential passerby but it seems that posts can’t be edited once they get old-ish. :neutral_face:

At this point I think the main not-out-of-topic thing left that I’d really like to understand is:

I processed my first two pics by keeping the look table on and bending the RT tone curve in Exposure. The main reason was that keeping the table yielded colors much closer to what the embedded JPGs show (and which made more sense to me). Was that an heresy / terribly suboptimal?


G79A0118_edit.JPG.out.without-my-name.pp3 (14.8 KB)

I kept the curve in Film-like mode and did not get the impression that the colors were shifting dramatically or anything when I fiddled with the curve’s dots, but perhaps I’m just not looking properly.

This is something I’ve been wondering about. Do dual-illuminant profiles offer anything over single-illuminant ones, other than workflow convenience? That is, do they actually give better white balance or is it just easier to achieve?

Excellent Question!

Adobe put a lot of paid engineering effort into designing their dual-illuminant scheme, so there must be something compelling behind it, no? :crazy_face:

Now, I haven’t specifically dug around for such, but I know of no research that specifically addresses this. To your white-balance assertion, such a thing should be easy to test, with the objective of minimizing the magnitude of the needed whitebalance multpliers… ??

There’s more to color than just white balance - and a profile that is accurate for the illuminant is important to this.

The ideal profile is one that is exactly matched to the illuminant. But since most illuminants are somewhere on or close to the black body locus, interpolating between StdA and D65 gets you pretty close. Shooting in tungsten light with a D65 profile is better than nothing, but inferior to a dual-illuminant profile.

Apple has worked with Adobe to go even beyond dual illuminant profiles - DNG 1.6 adds support for a third calibration illuminant+profile combination, which is the “as shot” illuminant.

I think the question is, “What specifically is the egregious impact?”

When we open a RAW file in RT, the first thing that happens is the image is demosaic’d. Since the color accuracy of the demosaic’d image isn’t precise, “Color Management” is applied. We can see this if we turn off “Color Management” and look at just the base RAW demosaic’d. It’s pretty ugly. So it’s clear something needs to be done.

In principal, “Color Management” fine-tunes the colors to match a known standard. Each sensor can have a slightly different color cast that needs a different set of corrections to get to a known state/starting point. But .dcp files can provide more than accurate color matching, including giving us a starting point on curves (Tone Curve) as well as some kind of color grade (Look Table).

Because .dcp can include so much, in practice, there are subtle/not so subtle differences between various .dcp. Here’s what I understand.

RT “Camera Standard” is a generalized correction that may “work” OK if you’re not trying to, say, match colors from different sensors (ie: retaining a common “look” across various sensors). It’s a pretty good starting point, particularly when we apply things like “Film Simulations” or strong “Curves”.

Camera specific .dcp can match to a color standard, and, as you know from all the previous discussion here, there are details/differences between various .dcp. For instance, if we look at now Adobe lays out their directory structure for the .dcp files we see it’s broken into two. One directory contains “Adobe Standard” .dcp file, one file per camera model. The other is “Camera” where specific camera models are organized as their own sub-directories.

“Adobe Standard” .dcp specifies a tone curve, base table, two point luminance (tungsten and daylight), and “look” that the RentWare has themselves developed. If you have different cameras/sensors, this might be one way to get a common look/feel between systems. For example, Canon DSLR output and Sony mirrorless can be easily made to match using these .dcp. Only you can decide if that’s important to you or not.

On the “Camera” specific sub-directory side, we see several .dcp files in each sub-directory. These are the RentWare’s attempt to match various “styles” or “looks” offered by the camera manufacturer, such as Vivid, Clear, Neutral, etc, and they can use the in-camera jpg as further reference. The jpg reference is “Baseline Exposure”, but since it is not integrated in RT, I never select it as it only makes my images too dark. The selection seems to work only in RentWare.

I got to thinking about all this after someone asked about how to achieve Hasselblad’s “Natural” colors outside Hasselblad’s system of software. Foccus uses .xml files to define colors, including their “Natural” color grade. I couldn’t find an easy way to convert .xml to .dcp to try and emulate the color grade in RT. BUT, I find that the “Adobe Standard” with all the boxes ticked except “Baseline Exposure” comes really really close to matching Hasselblad’s “Natural” colors.

Coming back to RT and .dcp, one of the really great things about RT is that once I’ve sorted out the base settings for demosaic algorithm, color management (Tone Curve, Base Table, Look Table selected), lens, Capture Sharpen (or not), vignetting, “Curves” (I always use “Luminance” to avoid color shift/distortion), the selections are saved as an RT Profile, meaningfully named, of course, so I know which to select in the future. In this way I only have to sort stuff out once and I’m “good to go.” I’ve gone as far as to write a set of RT Profiles, one file each for every .dcp available to me, so if I really want, say, Sony “Vivid” on an A6000, I can quickly grab the Profile and start processing from there.

I hope this helps. Yes, I know I haven’t said anything the other’s haven’t already said. But I hope the organization of what I wrote gets you to the kind of clarity (harumph!) you’re looking for.

3 Likes

Thanks for the summary. :smile:

Oh, OK, I failed to get that part.

I think I never experimented with that. :no_mouth: When I tick that box, the effects of the curve tend to get so muffled down that I’m suddenly underwhelmed by the preview. But It’s probably in large part because I never read about the mathematical specifics behind that feature.

Regarding profiles, as I mentioned, I’m currently using the one that emulates the “Neutral” Canon style, which gets me close to the embedded JPG. I don’t use its own tone curve, though. I dunno, I’ll try again perhaps, but then every little change in the Exposure tab’s own curve seems to suddenly have a huge impact and it makes me feel uneasy while tweaking it.

And yeah I have a bunch of profiles, but mostly tailored for my former camera, for which I did not touch “color management”. I’ll have to make a new one. I also use profiles to change some defaults to save time. For example, I always set the RGB curves on “Parametric”, and doing it manually for the three channels each time quickly became annoying. :laughing:

I carefully read the following to better understand the different “Curves” selections - Exposure - RawPedia

Then I took a standard RAW image (such as from DPReview - Studio shot comparison: Digital Photography Review ) and worked on it using the different curve selections to confirm how the colors did or did not shift.

It’s possible that many of us have gotten so used to the RGB color distortions that come with the standard curve that we don’t think about it much and, in fact, expect higher color contrast.

Yes, the other curve types can at first look “dull” by comparison. But I’ve found that using a good .dcp with a decent “Look Table” can give a cinematographic color graded effect that I don’t want to ruin by applying a standard or film like curve. Further, I found I had to train my eye not to expect high color contrast to appreciate this. It’s definitely not to everyone’s taste, that’s for certain.

Again, I hope some of this can help.