I thought about that but it’s not cross-platform, etc. 
If anyone can, feel free to try on the raw from before: https://discuss.pixls.us/uploads/short-url/knqoX0Na100zNPuJVYjDVRNiWgt.CR3
I wouldn’t be surprised if it gave values different from RT’s and ART’s, at this point. 
This file has good data…as far as I can see… more like what I would expect
So wb co-eff of 1.623 1 2.023 CT 3794 from camera data
Darktable opens it with a warning of less than full support but it looks fine and the wb values are in the same ball park as the camera data
1.585 1 1.984 CT 3883
RT
is 1.879 1 1.507 CT 5257
So pretty close to the daylight settings from the camera data as opposed to as shot…
1 Like
Thanks for checking! 
And you see, this “5257”, it pops up for all my CR3s! 
So my RT installation is not more messed up than yours, I guess.
Indeed, the issue becomes less noticeable on pictures shot in daylight.
I just think you will have to wait for the camera to be fully supported and then you will be able to get back to your original task of sorting out your dcp work flow. Or maybe get one of those small gray/white cards you could put on a key chain so it’s handy and then you can take your wb reference shot
I do have dozens of pics from the older camera waiting to be processed in the meantime, but if I can take shortcuts that’d be nice. It seems hard to predict how fast support for each camera will come or increase.
The “copy values from ART” method is not that bad, actually. But I’ll retain your idea. Haha for a moment I was like “the camera itself is black, that could serve as a neutral patch”, then realized that it would have a hard time shooting itself. Well, maybe the lens cap, though? xD
In an attempt to reassure myself (“Did I pay several k€ for a camera whose raws I won’t be able to process correctly, despite the precautions I thought I had taken?”) and remind myself that this is supposed to be a fun hobby, I tried going through the whole RT process on those damned flowers from the first post. I used the Adobe Neutral profile, without its curve and with its look table, and brutally copied white balance values from ART. This yielded something fairly close to the thumbnail (except for contrast and stuff), and from there I did my own stuff:
Still not the picture of the year (I kinda like it, though, to be honest, and it became a symbol of my struggle, haha), but at least I proved myself that I could end up with something that suits my tastes better than the embedded JPG, while still using RT.
I came across distortion and chromatic aberration oddities, but I’ll refrain myself from asking questions about that here. I’ll create a dedicated topic one day (and do my own research beforehand) if I’m motivated enough. 
I’m late for bed yet again.
I think the info you (priort) rounded up and your tests (thanks again – now I know my CR3s are not that weird) are enough for a dedicated GitHub issue to be opened, but this will have to wait. 
Plus, maybe some users from the other side of the globe will bring valuable insights while I’m asleep.
1 Like
RawTherapee is not able to read the camera white balance for some newer Canon cameras. There was a topic on this, but nothing came out of it due to a lack of suitable sample image. At least you have provided one now and I can finally go about adding the support for reading the white balance.
2 Likes
Oh, sorry, I missed that topic, probably because it was about the R7.
That’s cool! So you already have this in your radar and don’t need me to summarize that stuff in a GitHub issue? (But then I’m not sure how to know when this gets fixed.)
In case that helps, here are raws for three of the penguin tests from earlier (I exclude the “custom WB” one because I did not use that properly so I have no idea what it actually did):
$ exiftool -WhiteBalance ./*.CR3
======== ./r6m2-wb_penguin_awb.CR3
White Balance : Auto
======== ./r6m2-wb_penguin_fluorescent-white.CR3
White Balance : Fluorescent
======== ./r6m2-wb_penguin_forced-4000k.CR3
White Balance : Manual Temperature (Kelvin)
r6m2-wb_penguin_awb.CR3 (11.0 MB)
r6m2-wb_penguin_fluorescent-white.CR3 (11.0 MB)
r6m2-wb_penguin_forced-4000k.CR3 (10.9 MB)
Tell me if you also want the uncompressed versions. Dunno if that changes anything on your side. I assume that a compressed CR3, once decompressed, has roughly the same metadata structure as the never-compressed ones.
Good luck.
I think I found the problem! RawTherapee is indeed reading the camera’s daylight white balance instead of the as-shot white balance. With the fix, the white balance multipliers match the levels reported in the metadata.
1 Like
That was fast! 
So priort’s hunch about daylight was right.
I’ll continue that discussion on the pull request, then. The white balance stuff has hijacked that topic more than enough.
I wanted to write a TL; DR in my initial post for potential passerby but it seems that posts can’t be edited once they get old-ish. 
At this point I think the main not-out-of-topic thing left that I’d really like to understand is:
I processed my first two pics by keeping the look table on and bending the RT tone curve in Exposure. The main reason was that keeping the table yielded colors much closer to what the embedded JPGs show (and which made more sense to me). Was that an heresy / terribly suboptimal?
G79A0118_edit.JPG.out.without-my-name.pp3 (14.8 KB)
I kept the curve in Film-like mode and did not get the impression that the colors were shifting dramatically or anything when I fiddled with the curve’s dots, but perhaps I’m just not looking properly.
This is something I’ve been wondering about. Do dual-illuminant profiles offer anything over single-illuminant ones, other than workflow convenience? That is, do they actually give better white balance or is it just easier to achieve?
Excellent Question!
Adobe put a lot of paid engineering effort into designing their dual-illuminant scheme, so there must be something compelling behind it, no? 
Now, I haven’t specifically dug around for such, but I know of no research that specifically addresses this. To your white-balance assertion, such a thing should be easy to test, with the objective of minimizing the magnitude of the needed whitebalance multpliers… ??
There’s more to color than just white balance - and a profile that is accurate for the illuminant is important to this.
The ideal profile is one that is exactly matched to the illuminant. But since most illuminants are somewhere on or close to the black body locus, interpolating between StdA and D65 gets you pretty close. Shooting in tungsten light with a D65 profile is better than nothing, but inferior to a dual-illuminant profile.
Apple has worked with Adobe to go even beyond dual illuminant profiles - DNG 1.6 adds support for a third calibration illuminant+profile combination, which is the “as shot” illuminant.
I think the question is, “What specifically is the egregious impact?”
When we open a RAW file in RT, the first thing that happens is the image is demosaic’d. Since the color accuracy of the demosaic’d image isn’t precise, “Color Management” is applied. We can see this if we turn off “Color Management” and look at just the base RAW demosaic’d. It’s pretty ugly. So it’s clear something needs to be done.
In principal, “Color Management” fine-tunes the colors to match a known standard. Each sensor can have a slightly different color cast that needs a different set of corrections to get to a known state/starting point. But .dcp files can provide more than accurate color matching, including giving us a starting point on curves (Tone Curve) as well as some kind of color grade (Look Table).
Because .dcp can include so much, in practice, there are subtle/not so subtle differences between various .dcp. Here’s what I understand.
RT “Camera Standard” is a generalized correction that may “work” OK if you’re not trying to, say, match colors from different sensors (ie: retaining a common “look” across various sensors). It’s a pretty good starting point, particularly when we apply things like “Film Simulations” or strong “Curves”.
Camera specific .dcp can match to a color standard, and, as you know from all the previous discussion here, there are details/differences between various .dcp. For instance, if we look at now Adobe lays out their directory structure for the .dcp files we see it’s broken into two. One directory contains “Adobe Standard” .dcp file, one file per camera model. The other is “Camera” where specific camera models are organized as their own sub-directories.
“Adobe Standard” .dcp specifies a tone curve, base table, two point luminance (tungsten and daylight), and “look” that the RentWare has themselves developed. If you have different cameras/sensors, this might be one way to get a common look/feel between systems. For example, Canon DSLR output and Sony mirrorless can be easily made to match using these .dcp. Only you can decide if that’s important to you or not.
On the “Camera” specific sub-directory side, we see several .dcp files in each sub-directory. These are the RentWare’s attempt to match various “styles” or “looks” offered by the camera manufacturer, such as Vivid, Clear, Neutral, etc, and they can use the in-camera jpg as further reference. The jpg reference is “Baseline Exposure”, but since it is not integrated in RT, I never select it as it only makes my images too dark. The selection seems to work only in RentWare.
I got to thinking about all this after someone asked about how to achieve Hasselblad’s “Natural” colors outside Hasselblad’s system of software. Foccus uses .xml files to define colors, including their “Natural” color grade. I couldn’t find an easy way to convert .xml to .dcp to try and emulate the color grade in RT. BUT, I find that the “Adobe Standard” with all the boxes ticked except “Baseline Exposure” comes really really close to matching Hasselblad’s “Natural” colors.
Coming back to RT and .dcp, one of the really great things about RT is that once I’ve sorted out the base settings for demosaic algorithm, color management (Tone Curve, Base Table, Look Table selected), lens, Capture Sharpen (or not), vignetting, “Curves” (I always use “Luminance” to avoid color shift/distortion), the selections are saved as an RT Profile, meaningfully named, of course, so I know which to select in the future. In this way I only have to sort stuff out once and I’m “good to go.” I’ve gone as far as to write a set of RT Profiles, one file each for every .dcp available to me, so if I really want, say, Sony “Vivid” on an A6000, I can quickly grab the Profile and start processing from there.
I hope this helps. Yes, I know I haven’t said anything the other’s haven’t already said. But I hope the organization of what I wrote gets you to the kind of clarity (harumph!) you’re looking for.
3 Likes
Thanks for the summary.
Oh, OK, I failed to get that part.
I think I never experimented with that. 
When I tick that box, the effects of the curve tend to get so muffled down that I’m suddenly underwhelmed by the preview. But It’s probably in large part because I never read about the mathematical specifics behind that feature.
Regarding profiles, as I mentioned, I’m currently using the one that emulates the “Neutral” Canon style, which gets me close to the embedded JPG. I don’t use its own tone curve, though. I dunno, I’ll try again perhaps, but then every little change in the Exposure tab’s own curve seems to suddenly have a huge impact and it makes me feel uneasy while tweaking it.
And yeah I have a bunch of profiles, but mostly tailored for my former camera, for which I did not touch “color management”. I’ll have to make a new one. I also use profiles to change some defaults to save time. For example, I always set the RGB curves on “Parametric”, and doing it manually for the three channels each time quickly became annoying.
I carefully read the following to better understand the different “Curves” selections - Exposure - RawPedia
Then I took a standard RAW image (such as from DPReview - Studio shot comparison: Digital Photography Review ) and worked on it using the different curve selections to confirm how the colors did or did not shift.
It’s possible that many of us have gotten so used to the RGB color distortions that come with the standard curve that we don’t think about it much and, in fact, expect higher color contrast.
Yes, the other curve types can at first look “dull” by comparison. But I’ve found that using a good .dcp with a decent “Look Table” can give a cinematographic color graded effect that I don’t want to ruin by applying a standard or film like curve. Further, I found I had to train my eye not to expect high color contrast to appreciate this. It’s definitely not to everyone’s taste, that’s for certain.
Again, I hope some of this can help.
Oh I was mixing stuff in my head. The “Luminance only” checkbox is for other tools. Indeed here it’s a curve mode of its own. Well, I generally use Film-like if I want to keep things realistic, and Standard when I don’t mind fancy colors too much. Sometimes I change my mind halfway through if the results seem to go in a direction that doesn’t suit me.
Edit: Tried the Luminance mode right now, on a slightly overexposed picture, and it ended up satisfying me more than Standard and Film-like. I guess I’ll keep that mode in my sleeve, now. Comparing the three isn’t much longer than two modes.
It sure helps, thanks, but the countless possibilities are mind-boggling. And I’m still not sure how dangerous it is to fiddle with the Exposure curve while having the DCP curve off. The three first pics I post-processed that way did not strike me as particularly weird.
This can be really complex stuff, can’t it? Consider for a moment that RentWare won’t let you select any of the .dcp fields that RT does. Perhaps RentWare knows people could be overwhelmed?
To illustrate how RT can work with .dcp, select a “Processing Profile” as soon as one of your unprocessed RAW images loads. Try, for instance, the “Bundled Profile” “Auto-Matched Curve low ISO” and stroll over to the “Color Management” panel to have a look at how the .dcp is handled.
If your system is anything like mine and “Camera Standard” is used (because it couldn’t find a camera specific .dcp) you may see that there is no selection for “Tone Curve.”
So you have a choice. You can use one of the “Bundled Profile”, which will set the “Curves” as it sees fit. Or, you can ignore any of the pre-canned RT profiles and let “Color Management” .dcp do the “Curves” initial heavy lifting for you as long as you have a camera specific file.
Flipped the other way around, if you don’t enable "Tone Curve"s in “Color Management” and you’ve not used one of the "Bundled Profile"s, then you’ll need to lean on the curves yourself under the “Exposure” panel.
So, no, it really doesn’t seem to matter which process approach you use (unless you are very exacting in your color matching, color profiling, color grading, which some people justifiably are), just choose one that fits the need and stick with it to avoid further confusion/complication.
Just to make things delightfully more complex, if you ever find yourself working in B&W, there are “before” and “after” processing curves available under the B&W panel as well.
… and to underscore a point that I passed perhaps too lightly over: You can use the .dcp “Tone Curve” and the curves found on the “Exposure” panel. It’s all up to you and there are no demerits for doing anything you find “works” for you.
Said another way, any of the approaches I’ve outlined here are simply setting the starting point for further processing. What you do after an image is loaded to a known state that your are comfortable with is, again, entirely up to you.
I generally do that, yeah. And then either use the “auto-match” button to make the curve thumbnail-like or start from scratch in “Control cage” mode.
I guess that’s why someone all the way up here said “the problem is that the moment you alter that curve, the LookTable is no longer an appropriate match”. That got me confused.
I used B&W 2–3 times and left its own curves alone. Just picked the mode that looked best to me, and then did the rest of the processing more or less as usual.
I suppose so, but that’s when doing this that I find it hard to predict and gauge the effects of moving dots around on the Exposure curve. That’s what I was referring to earlier. But I’ll try again someday.