[Questions] How to choose a DCP profile (if at all)

[priort (Todd Prior)]

Not sure but loading a R6 II test image into both DT and RT there are very different values used for as shot wb…

Canon EOS R6 Mark II in-depth review

DT

RT

And the exif is a bit weird looking to me THere is a temp for I assume WB but the coefficients look funny??

[alicem]

@ggbutcher

For white balance, you want to use the Pick tool on a white or neutral patch of the image.

I understand the logic there, but:

• Sometimes I’m having a hard time finding such a patch. If I go for a walk, I’m not sure I want to take a shot of a paper sheet every time I come across something I want to shoot, to adapt to the amount of clouds and stuff at that specific moment.
• I almost never had to use that with the Panasonic raws: I only made temperature adjustments of 100–200 at most in RT and almost never touched the tint.

@priort

loading a R6 II test image into both DT and RT there are very different values used for as shot wb

So there seems to be something wrong, you think? And what about the output? Did the picture look similar in DT and RT? (But if those are test shots I guess they were shot in a fairly good lightning so if there’s an issue it should be less visible there I guess.)

@priort

the exif is a bit weird looking to me There is a temp for I assume WB

Oh, see? That temperature and the blue / red zeroes above are what I was referring to. I have those exact same values everywhere. So perhaps these are not the fields we should be looking at?

@priort

but the coefficients look funny

Which ones? The “Measured RGGB” thing? Or are you just referring to the red / blue zeroes?

I don’t know if RT’s file browser uses the embedded JPGs or something else. Look: pictures look OK in the browser and get the 5257-and-1.043 balance as soon as I open them in the editor:

(This is, by the way, an example of a picture where I’m not sure where to find a neutral patch. The twigs are a bit brown and lead to a balance skewed toward coldness. Edit: OK, the black part in the background is kinda OK as long as the picked area is not clipped as hell.)

I’ll attach the raw if you want to fiddle with it, either for neutral-patch-finding advice or to inspect the metadata and balance:
G79A0118.CR3 (10.5 MB)

Tell me if this requires a brand new topic: we’re straying a bit from what the title and tags imply. Or even a GitHub issue if @Lawrence37 or others think that may be a bug.

[alicem]

@priort

a R6 II test image […]
Canon EOS R6 Mark II in-depth review: Digital Photography Review

Their https://www.dpreview.com/reviews/image-comparison/download-image?s3Key=9dc4402898fe4882950f3b1ec600daa0.cr3 displays fine in my RT, with a balance that is not the one I keep getting for my pictures.
Edit: Actually, I get something too cold with that image, I think. If I pick something obviously neutral from the picture (and it’s very easy to find, there), the balance goes waaaay up (something like 5340 / 1.049 → 6865 / 1.070). My eyes were just deceiving me because their test shot is kinda beautiful. xD

A quick diff between the exifs of that sample and of my bear-and-sound-card picture from earlier (reporting only what may be interesting here):

Hid the table because maybe unrelated since I’m not 100% fond of the auto balance of the dpreview sample either. Still, check it if you want.

Field	Me	Them	Comments
Exposure Mode	Auto	Manual	Probably unrelated
Quality	CRAW	RAW	I may have to try, but meh, they’re big.
Record Mode	CRAW	CRAW+JPEG	Quality RAW but recorded as CRAW? Not sure I get it.
White Balance	Auto	Custom	Sure I could customize it for every shot but that’s not exactly what I’m aiming for.
Camera Temperature	19 C	30 C	That’s the literal temperature of the camera body, right? Not related to WB, is it?
Picture Style	Neutral	Auto	Probably useless but I note it just in case.
WB Shift AB	0	1	Sounds somewhat interesting but not sure what that is. Probably related to that two-axis WB diagram viewable in the camera itself, where I’m always at 0;0.
WB Shift GM	0	0	Just to show that the field above has a twin. And I confirmed that so far all my raws have 0 for both.
Measured RGGB	[…]	[…]	Obviously very different.
Auto Lighting Optimizer	Standard	Off	?
Shooting Mode	Flexible-priority AE	Manual	I like that mode but perhaps it has side-effects?
Light Value	6.3	10.3	?

Yet another edit: Could that be related to the fact that, while the R6 Mark I has "ranges" : { "white" : 16382 } in camconst, the Mark II has no such property? I guess its support is still rated C for a reason, but those things are obscure as hell to me.

[Entropy512 (Andy Dodd)]

@ggbutcher

It’s not too hard, most challenging part is shooting the target without glare. Just a matter of the right angle…

Especially since if it’s a camera that doesn’t have an included profile in RT, that’s a valid issue to file on github.

@alicem

… but that confuses me a little, as the profile’s tone curve tends to be disabled by default. I suppose leaving that off does not count as an “advanced user” choice?

Rawpedia definitely needs an update for this in regards to the LookTable - especially the “Standard Film Curve” profiles. These implement a tone curve that replaces any embedded tone curve in the DCP profile (note, Adobe seems to rely heavily on the DCP profile’s embedded tone curve, none of RawTherapee’s default processing profiles do - they’re either neutral or use RT’s built-in tone curve function). The LookTable appears to be intended to be used with the default “Standard Film Curve” to provide the equivalent of the slower “Perceptual” curve mode when the tool is set to “Film-Like” - Exposure - RawPedia - the problem is that the moment you alter that curve, the LookTable is no longer an appropriate match.

Setting the Curve Mode to “Perceptual” and disabling the LookTable and Tone Curve should provide identical results for RT’s included profiles to “Film Like” + LookTable enabled - but will be slower. I’m going to look into finding ways to optimized the “Perceptual” mode so it can be used more readily one of these days…

[alicem]

@Entropy512

that’s a valid issue to file on github

You mean, just opening an issue to say “hey there’s no profile for XXX it’d be nice to have one”?

@Entropy512

The LookTable appears to be intended to be used with the default “Standard Film Curve” to provide the equivalent of the slower “Perceptual” curve mode when the tool is set to “Film-Like” - Exposure - RawPedia - the problem is that the moment you alter that curve, the LookTable is no longer an appropriate match.

Huuum… Do you mean that using the Look table does not make sense if I move even a single dot in “Exposure→Tone curve”? (It’s a bit hard to follow for me.)

Back to white balance

ART 1.12.1 succeeds in giving different balance values across my pictures.
The bear gets 3296 temp 1.028 tint, and the planks covered with moss get 3909 temp 0.929 tint, for example.
The bear still looks weird but less (it was shot in terrible conditions, so I guess manual fixes are mandatory) and the planks look so “normal” that I can’t really tell the difference with the thumbnail.
Is that sufficient to say that RT is the culprit and reads stuff incorrectly? @Lawrence37 Can I help in any newbie-compatible way?

I was also wondering if I had weird configuration files from older RT versions interfering with that, but I don’t see anything suspicious in ~/.config/. Plus, I don’t see why such an issue would interfere only with my CR3s. For the heck of it, here are Color Management entries that appear in my RT options but not in ART’s option file:

Options

Autocielab=false
RGBcurvesLumamode_Gamut=true
AdobeRGB=RTv2_Medium
ProPhoto=RTv2_Large
WideGamut=RTv2_Wide
DCIP3=RTv2_DCIP3
sRGB=RTv4_sRGB
Beta=RTv2_Beta
Best=RTv2_Best
Rec2020=RTv2_Rec2020
Bruce=RTv2_Bruce
ACES-AP0=RTv2_ACES-AP0
ACES-AP1=RTv2_ACES-AP1
GamutICC=true
GamutLch=true
ProtectRed=60
Amountchroma=40
JzAmountchroma=40
ProtectRedH=0.29999999999999999
CRI=0
DenoiseLabgamma=2
CBDLlevel0=0
CBDLlevel123=30
Itcwb_enable=true
Itcwb_deltaspec=0.074999999999999997
Itcwb_powponder=0.14999999999999999
Previewselection=5
Cbdlsensi=1

Edit: Tried renaming my config to get the default one instead. Same issue. Assuming it only looks in ~/.config/RawTherapee-AppImage/ and not in ~/.config/RawTherapee/.

I’d be curious to hear about what people get with RT 5.10 (I’m using the Linux AppImage from the stable release) and other apps with the plank-and-moss raw from above.

Fun-weird fact: for a same DCP (camera standard), ART disables the Look table by default and even prevents the activation of the Baseline exposure, while RT does the exact opposite (Baseline forced ON). I’d rather not pay attention to that, though, now.

A workaround for now would be to open the file in both ART and RT, and to copy the temperature and tint values. (Or migrate to ART but I fear I’d sorely miss some tools.) It yields very similar results as long as I don’t load any DCP (if I do, they seem to be applied a bit differently). Of course this does not prevent me from using the color picker if a good occasion presents itself.

By the way I did a few tests with different WB settings and with RAW quality (but it still produces .CR3 files?! Uncompressed and big, but same extension it seems).

• RT still uses the same default temperature and tint for all those pictures, regardless of camera settings. As a result, even the most dissimilar of those pictures (according to embedded JPGs) look near identical when opened in RT.
• It appears the “Color Temperature” field is the value selected for the “Manual Temperature (Kelvin)” WB mode, in which a temperature is forced. I guess it gets embedded in the metadata even if that mode is not the currently-selected one. I changed the value in the camera and it seems to confirm this.

Sorry for the repeated posts, edits, and mentions; I want to make sure I won’t needlessly do brutal stuff like factory-resetting my camera and so on. Plus, this looks important-ish if the issue does not come from me after all.

[priort (Todd Prior)]

@alicem

Which ones? The “Measured RGGB” thing?

Ya normally the r and b numbers are larger and dividing by 1024 give the multipliers used for wb. Maybe expressed differently here??

[ggbutcher (Glenn Butcher)]

@alicem

Sometimes I’m having a hard time finding such a patch. If I go for a walk, I’m not sure I want to take a shot of a paper sheet every time I come across something I want to shoot, to adapt to the amount of clouds and stuff at that specific moment.

For mid-day, you just really need one shot, somewhere in the period. It’s a little tougher at dawn/dusk, as the light is changing more rapidly. And you’re not worried about the specific scene so much as you are about the prevalent light, so one would do it for a series of captures.

Really, the very best source of white balance information is a neutral/white patch in the scene. Me, I’ve become confident in Nikon’s auto daylight modes so i don’t worry it out side, but in my workspace its a “zoo” of various light sources so i do it there for various light combinations.

And also, keep in mind that most scenes are a mishmash of temperatures, even in daylight-lit, as sometimes significant parts of the scene are predominantly illuminated by reflected light, which has it’s own non-whitebalance tints. Mid-day shadows are especially vexing, as they are usually illuminated by the blue sky rather than directly by the sun. And, don’t get me going about city-scapes at night…

And, you wonder why movie crews drag along sometimes literally tons of lighting equpment…

[alicem]

@priort

normally the r and b numbers are larger and dividing by 1024 give the multipliers used for wb.

Really? If you want, here are mine:

Measured RGGB data + test overview

$ exiftool -'Measured*RGGB*' ./*.CR3 | awk -F ': *' '{ print $2 }' | column -t

# “Normal” picture, not taken just for tests:

1850  1024  1024  185
641   1024  1024  586
744   1024  1024  1087
499   1024  1024  612
629   1024  1024  514
333   1024  1024  871
620   1024  1024  612
559   1024  1024  556
710   1024  1024  351
609   1024  1024  362
556   1024  1024  486
605   1024  1024  564

# Test series:

623  1024  1024  590 # CR3, AWB
618  1024  1024  592 # CR3, Fluorescent white
622  1024  1024  592 # CR3, Custom (assuming I did the calibration correctly…)
620  1024  1024  592 # CR3, Forced 4000K
648  1024  1024  580 # RAW, AWB
626  1024  1024  589 # RAW, Fluorescent white
630  1024  1024  588 # RAW, Custom (assuming I did the calibration correctly…)
637  1024  1024  587 # RAW, Forced 4000K

Here’s an overview of the embedded JPGs of the test series:

Perhaps RT relies only on that and there’s another field somewhere that needs to be used to tweak that? These values are indeed always often lower than 1024. At least with my camera, but also with two raws I got before my purchase to try stuff. One from raw.pixls.us and the other from that other topic on this site.

[ggbutcher (Glenn Butcher)]

@Entropy512

Especially since if it’s a camera that doesn’t have an included profile in RT, that’s a valid issue to file on github.

Indeed. Your target profile could easily become a contribution to RT.

[alicem]

@ggbutcher

Your target profile could easily become a contribution to RT.

I’ll consider it, but probably not while I’m stressed up over the auto-balance issue. I tend to have a single-track mind when something worries me.
Also, I fail to understand how much added value that would have over the Adobe ones. Is it basically better because at least we know how we made it and what’s in it?

@ggbutcher

you just really need one shot, somewhere in the period

I’ll make sure to give a read to the “Custom” WB mode of the camera manual. Can’t hurt. … Yeah I just checked, I did not do it correctly earlier. I was supposed to take a picture for real, not re-press the “Delete” button once the sheet of paper was in sight.

[ggbutcher (Glenn Butcher)]

@alicem

s it basically better because at least we know how we made it and what’s in it?

Yes. That said, however, a lot of the camconst color profiles are extracts from Adobe DCPs. But even that is someone else’s measurement; there’s nothing like your own direct measurement of the mechanism to soothe the soul…

@alicem

I’ll make sure to give a read to the “Custom” WB mode of the camera manual. Can’t hurt. …

And really, keep in mind the “one WB to rule them all” is hooey in most anything but a studio…

[alicem]

@ggbutcher

the “one WB to rule them all” is hooey in most anything but a studio

Sure, but the “Auto” mode is supposed to help somewhat, I suppose.

OK I’m even more lost, now that I’ve noticed something else:

• ART providing varying temperature and tint values across my pictures points towards an RT issue, BUT…
• … RT is able to give other temperature and tint values for the raws that are not from me, which I mention at the end of [Questions] How to choose a DCP profile (if at all) - #28 by alicem (ART still handles that vegetable plate better though), and this points in the other direction, towards a misconfiguration of my camera.

I’ll force myself to take a break; I can’t make head or tails of the things I see. I hope someone will be able to explain that.

[Entropy512 (Andy Dodd)]

@alicem

You mean, just opening an issue to say “hey there’s no profile for XXX it’d be nice to have one”?

No, unfortunately since we don’t own the camera in question there’s not much we can do.

However if you have a ColorChecker and submit shots of the target (one in noon sunlight, one with a tungsten bulb), plenty of the developers (including myself if I’m paying attention) can handle the rest and/or let you know if there’s an issue with the shots.

If you have no way to get a tungsten bulb, just a noon sunlight shot is better than none. Tungsten bulbs are getting hard to come by, but can still be found if you look. I bought a cheap “work light” style lamp holder and a single tungsten bulb at Home Depot a while back. I only use the bulb for ColorChecker shots so it should last a LONG time.

Will read the rest of your post later, gotta get ready for lunch.

[ggbutcher (Glenn Butcher)]

@alicem

I’ll force myself to take a break; I can’t make head or tails of the things I see. I hope someone will be able to explain that.

My apologies, I thought I was making it easier but I guess I’m not.

[alicem]

@ggbutcher

My apologies, I thought I was making it easier but I guess I’m not.

Oh, don’t apologize: I was mostly referring to the fact that I still don’t know why my R6m2 pictures, specifically, yield nonsensical and constant default white balance values in RT.

@Entropy512

if you have a ColorChecker

I don’t have one currently. As for bulbs, I have no idea. Do they have any distinctive trait that’d make them recognizable? Is there any chance to find one already installed in a room somewhere? Not sure I’d want to buy stuff like this just for a single use.

[priort (Todd Prior)]

Is the camera supported … could that not be the issue if it is not then its not correctly handling the wb data??

I guess the release notes for 5.1 say added improved support for your camera so I would think it should grab the right values…

So when you look at the exif data for all those photos do they have different wb and RT is not picking that up??

[alicem]

@priort

Is the camera supported … could that not be the issue if it is not then its not correctly handling the wb data??

As you saw, it was in the release notes. It has an entry in camconst.json, with a C ranking (better than nothing). This was actually one of my main criteria when choosing a new camera.
Strangely enough, it shares its entry with the R8:

camconst.json excerpt

{ // Quality C
    "make_model": ["Canon EOS R6m2", "Canon EOS R8"],
    "dcraw_matrix": [9539, -2795, -1224, -4175, 11998, 2458, -465, 1755, 6048],
    "raw_crop": [
        {"frame": [6188, 4120], "crop": [154, 96, 6024, 4024]},
        {"frame": [3936, 2612], "crop": [156, 96, 3780, 2516]}
    ],
    "masked_areas": [
        {"frame": [6188, 4120], "areas": [4, 4, 4116, 150, 4, 150, 92, 6184]},
        {"frame": [3936, 2612], "areas": [4, 4, 2608, 150, 4, 150, 92, 3932]}
    ]
}

No occurrence in dcraw.cc I think, however.

@priort

So when you look at the exif data for all those photos do they have different wb and RT is not picking that up??

I’m still not sure which metadata field(s) is / are relevant, but at the very least the Measured RGGB field varies (perhaps not as much as one would expect) and yet I get the same temp and tint in RT (but not in ART). And even for pictures from other R6m2 owners, ART’s balance is notably different than RT’s.

I’ll try with other modes than “Flexible Value” just in case. Forgot to do that earlier with the penguin plushy…
Edit: Done. No luck. Shutter speed priority, Aperture-priority, Manual or even the full auto thingy, all yield the same old “5257-and-1.043” balance (in RT).

[alicem]

Huum in RT’s metadata viewer, I see something like:
Measured Color: 12 623 1024 1024 590 0 (in the “Canon” group)
instead of the
Measured RGGB: 623 1024 1024 590 (in the “MakerNotes” group)
from exiftools. So basically the same values but sandwiched between two extra ones (seemingly always 12 and 0). But perhaps that’s normal.

Edit: ART shows a greater number of metadata fields than RT in the “Canon” group, for a given CR3. I see for example the following additions (not exhaustive at all, mostly citing those that may sound related to color issues):

• AmbienceInfo
• LightningOpt

Same weird 12–0 Measured Color sandwich on both sides, though.

[priort (Todd Prior)]

Can you run the canon software…what does it provide for wb …

[alicem]

I thought about that but it’s not cross-platform, etc. If anyone can, feel free to try on the raw from before: https://discuss.pixls.us/uploads/short-url/knqoX0Na100zNPuJVYjDVRNiWgt.CR3
I wouldn’t be surprised if it gave values different from RT’s and ART’s, at this point.