Issue with desaturated DNGs produced by a phone camera

[Entropy512 (Andy Dodd)]

@Keksoj

the cloud link is okay to me though, I created it myself on a cellar add-on at Clever Cloud

I’m currently in a location where ALL cloud storage providers are blocked (especially the lesser known ones, but even the “big boys” like Google Drive are), but forum attachments usually work.

Interesting, so it’s not the usual problem of them putting an sRGB XYZ->CAM matrix into the CM1 tag. It is a bit strange to see two different ColorMatrix1 values, but only one ForwardMatrix value for both 1 and 2…

White and black points seem correct - the smallest value in the raw data is 64, largest is 1023.

[ggbutcher (Glenn Butcher)]

ColorMatrix1 are D65 values, so I extracted them from the metadata and used them dcraw-style. The black level from libraw didn’t seem to be right, so I dialed it up. That, and a rather sporty loggamma-lift/s-curve-shape, oh, and highlight recovery for the blown pavement, got me this from rawproc:

Edit: I’d daresay that poppin’ color in the OOP JPEG is an application of ludicrous color saturation, added that to my processing chain:

Edit #2: @Entropy512 is right, the black point corresponds with the minimum value of all three channels. I just hate milky shadows…

[garrett (Garrett LeSage)]

Apologies that this is specific to Pixels (possibly just the Pixel 6 Pro with the default camera app’s raw). But the same idea applies to every camera, including every phone.

I shot my Color Checker Passport with my Pixel 6 Pro and made a daylight profile that gets auto-applied. (It even helps when it’s not purely daylight, but a specific profile per non-daylight scene would always be better.)

The colors before versus after were… wow. After using the color calibration’s color checker part, the colors are bold and saturated like the JPEGs without manually tweaking things.

It would be better, of course, to get improved colors by default in darktable. Meanwhile, I wonder how applicable my profile would be to other Pixels.

(I still have to calibrate the noise and lenses sometime, somehow.)

Anyway, here’s my makeshift profile for the color calibration module for the built-in camera app’s raw on the Pixel 6 Pro’s main camera. I should make some for the other 3 cameras too, and probably for open camera as well. (I assume it might all be similar… but it might not.)

Firstly: You need to use “modern” color “chromatic adaption defaults” (in darktable’s processing settings) to use this profile.

Pixel 6 Pro - Standard.dtpreset (1.1 KB)

You can import this preset: go to darktable’s settings, select the “presets” page, then click on “import preset…” and choose this file (probably in your Downloads directory — after you download the file, of course).

Then, to apply it, choose a DNG photo from your Pixel and click on the three-line menu icon on the “color calibration” module and select “Pixel 6 Pro - Standard”.

[Keksoj]

Thank you all so much for such an enthusiast enquiry in the consistence of these DNGs
I understand nothing yet about these color matrix. Is it something I should worry/know about, or is it a minor issue in my case?

I mean. Is the Open Camera App doing utter nonsense?

[ggbutcher (Glenn Butcher)]

For all supported cameras, the raw processor will have a humungous list of all the color matrices. Phones are a bit out there, so for DNGs I’m thinking the raw processor should look for the appropriate ColorMatrix in the metadata and use that as a last resort (thinking that the oh-so-carefully measured internal matrix should be used first if available). My hack raw processor doesn’t have that logic right now, but it will in a few days…

[Keksoj]

@garrett that workflow seems amazing. I’ll look into your presets but I’ll be very curious about repeting your method with a Color Checker Passport. Would you give me details?

I imported your preset and it does make nice and vivid colors on my DNGs (even though my phone is NOT a google pixel, it’s a fairphone), however I get a white balance module error. I realize I didn’t set chromatic adaptation defaults to modern before. I did now, but I’m afraid all my DNGs have been preprocessed already. What can I do? Delete all xmp files?

[Entropy512 (Andy Dodd)]

@Keksoj

I understand nothing yet about these color matrix. Is it something I should worry/know about, or is it a minor issue in my case?

It’s effectively a characterization of how your sensor responds to color.

For DNG files, it SHOULD be embedded in the file, but in many cases, DNG implementations make mistakes such that this metadata is wrong. Sometimes it’s just copypastaed from another phone by an app vendor, sometimes it’s from a “reference device” with the same CPU but not necessarily the same camera sensor in the phone’s HALs, and sometimes it’s flat-out wrong for any Bayer-on-silicon sensor with a real color filter array. (No CFA in existence has a response that exactly matches sRGB, yet I’ve seen MANY cameras that have the XYZ-to-CAM matrix which matches the sRGB inverse matrix in Welcome to Bruce Lindbloom's Web Site - as the sRGB gamut is quite narrow, any sensor data misinterpreted as being sRGB will be severely desaturated and have some weird color shifts.

Ideally, if you don’t trust your camera’s metada, get a ColorChecker and shoot it in daylight and under tungsten-illuminated light to generate a dual-illuminant profile. (Not sure if dt supports dual-illuminant profiles. RawTherapee supports dual-illuminant DCPs, generated by the process described in https://rawpedia.rawtherapee.com/How_to_create_DCP_color_profiles )

[priort (Todd Prior)]

I think at one time or another I did something similar for my 3aXL… I will see what your preset does on that one…

EDIT: Shot of Hamilton Harbour sunset last night on my bike ride…

OOC JPEG
OOC JPG

Basic filmic and modern wb NO Other Modules

@garrett preset same conditions…

PXL_20220814_235045698.dng.xmp (6.8 KB)
PXL_20220814_235045698.dng (11.4 MB)

[priort (Todd Prior)]

You can also try the adobe DCP files as starters in RT and ART…they have profiles for the pixel…although in ART selecting embedded I think grabs all the same information it would seem when toggling back and forth and you can choose the look and tone curves from the embedded data… I use DT more often but when I need to check something weird on color…not that it has to be Adobe stuff but it can serve as a common and accepted reference to compare any shenanigans that appear in DT or elsewhere…

[Entropy512 (Andy Dodd)]

I personally would advise against using any of the tone curves or the LookTable (which is intended to be used after a tone curve) in a DCP profile - https://github.com/Beep6581/RawTherapee/issues/6467

One thing I noticed is that the ForwardMatrix entries in the DNG provided by @Keksoj have no negative coefficients, but there are no ProfileHueSatMapData LUTs. That combination is extremely suspicious to me, even moreso than the fact that both ForwardMatrix entries are identical.

When you don’t trust the camera manufacturer’s profile, doing a profiling run with a ColorChecker is the way to go.

This particular case could be challenging since you are going to have significant lens shading (vignetting + radial color distortions) in the raw data - the GainMap corrects for that in theory, but if the color profile is not correct, can you be sure that the GainMap is correct?

Does the stock camera app on the device support raw capture? If not, that’s a big red flag that Open Camera may be using built-in defaults and/or the camera HAL has some generic metadata sitting inside it. If so, does the embedded profile data look different?

[priort (Todd Prior)]

Thanks for your insightful feedback and analysis… For sure its some sort of hack. Using it on the my pixel phone produces those crazy washed out DNG files…yet the jpg that comes from shooting raw and jpg on open camera has a ton of saturation. I have taken a few of the open camera files and even messing with WB and cranking saturation etc to the roof and its still not possible…

Perhaps someone that knows the ins and outs of that particular recipe for saving a DNG could provide more insight that this experiential account…

[kofa (István Kovács)]

@Keksoj

The sidecar xmp file was already well named, I double checked.

The name of the file is irrelevant, but you have to import it. Read load sidecar file here:

https://darktable-org.github.io/dtdocs/en/module-reference/utility-modules/lighttable/history-stack/

[Keksoj]

My fairphone’s native app does NOT support raw capture. Even the Open Camera app wouldn’t save to DNG until I found the Camera-API setting that offered two options:

• Original Camera-API (default, no raw)
• Camera2-API (raws available)

The same kind of setting appeared in Open Camera app when installed on a Galaxy S21 ultra (those things make insane good jpegs though).

I understand I could correct a fault in the Color Matrix and the Forward Matrix by performing a color check with a standard test picture under standard light, but what you are telling me, @Entropy512 , is that if I go for this correction, I’d have to account for other corrections as well, like lens shading. Is that right? Wow that seems a lot.

[kmilos]

FWIW, darktable only ever uses ColorMatrix1/ColorMatrix2 (whichever one is done for D65 or closest to it), and no other parts of the Adobe DCP, AFAIK.

[hannoschwalm (Jens-Hanno Schwalm)]

Matrix3 should also work. With dng files all raw developers & viewers make use of those exif tags as specified in dng specs.

Unfortunately many vendors just don’t care. There were many issues and reports like this before.

[Entropy512 (Andy Dodd)]

That’s unfortunately a red flag that the manufacturer did not fully test their Camera2 implementation and reports default/filler metadata to Open Camera.

For a while even Pixels had this flaw - if you queried Camera2 for metadata like the color matrix, you got garbage. So at least for a while after release, the Pixel 4 XL had its color data overridden by the camera app!

A DCP from another phone isn’t guaranteed to work - one that shares the same sensor MIGHT work, but not all sensors share a CFA design, and sometimes the lens has significant color effects.

The big question is whether or not you can trust the GainMap - I would take a picture of a very flat evenly lit white surface (or even better, shoot a white wall while holding a translucent object right against the lens like a slice from a plastic milk jug) and verify that the raw, when processed with software that supports GainMap (not all do), it looks evenly lit. If you can, then the remaining task is to shoot a ColorChecker in daylight and tungsten to generate an appropriate profile.

@hannoschwalm ColorMatrix3 is relatively new, part of the DNG 1.6 spec which is almost entirely new tags Adobe co-developed with Apple, I’m not sure if anyone other than Apple have implemented it camera-side yet.

[Keksoj]

That’s definitely the very precise answer that I needed, thank you so much!

This is overall pretty disappointing from Fairphone, and too much work for me as a beginner photographer.

You know what. I think I’ll drop the phone and settle for a proper camera, some of my relatives have an idle Canon G7x that I could make use of

[hannoschwalm (Jens-Hanno Schwalm)]

You could open a bug report for the camera app

[Keksoj]

I’m not sure about the Open Camera app being responsible, according to @Entropy512 it’s the phone manufacturer who messed up the phone’s Camera2 API implementation, on which Open Camera connects.

We can ask Open Camera app to take the Fairphone’s lacking Camera2 API in consideration and provide sensible Color Matrix values instead, but I’m not someone to ask that of an open source contributor.

[Entropy512 (Andy Dodd)]

I’d have to look at the source for Open Camera, but I BELIEVE they try to use metadata provided by the camera HAL - which if the phone doesn’t offer raw capture itself may be incorrect. DngCreator  |  Android Developers

There’s not really any way for a camera app to fix this, except to detect phones with known-bad metadata and replace it. But that’s asking a huge amount of the camera app developer, because correcting the data requires spending the time to characterize the camera, which requires owning that particular phone and a ColorChecker.

Reporting the problem to the hardware manufacturer is unlikely to go anywhere. The Android world is a shitshow in that regard. I’ve seen critical flaws remain unfixed for more than a year after not only was the problem reported to a manufacturer’s developer relations, but a fix was provided as a patch. (For example, the Galaxy S2 kernel had a bug in the power management, where it would enter a state that would keep the phone from entering deep sleep if the fuel gauge was critically low. For one, this seems ridiculous, if you’re almost out of battery why use more? They may have been worried about current consumption during resume… But clearing the condition required the state of charge to be EXACTLY the threshold value. They used == instead of >= to clear the condition if the SoC rose above that threshold when on charger. I submitted a patch to all of my devrel contacts, and more than a year later, people still constantly complained about fuel_alerted wakelocks.)

After all, this is fairphone we’re talking about. A manufacturer who got so caught-up in anti-Qualcomm whining that they didn’t realize that for all of their flaws, Qualcomm was the least of all evils, and went with Mediatek who were blatant GPL violators instead for their first generation or two.