The name of the file is irrelevant, but you have to import it. Read load sidecar file here:
https://darktable-org.github.io/dtdocs/en/module-reference/utility-modules/lighttable/history-stack/
The name of the file is irrelevant, but you have to import it. Read load sidecar file here:
https://darktable-org.github.io/dtdocs/en/module-reference/utility-modules/lighttable/history-stack/
My fairphoneβs native app does NOT support raw capture. Even the Open Camera app wouldnβt save to DNG until I found the Camera-API
setting that offered two options:
The same kind of setting appeared in Open Camera app when installed on a Galaxy S21 ultra (those things make insane good jpegs though).
I understand I could correct a fault in the Color Matrix
and the Forward Matrix
by performing a color check with a standard test picture under standard light, but what you are telling me, @Entropy512 , is that if I go for this correction, Iβd have to account for other corrections as well, like lens shading. Is that right? Wow that seems a lot.
FWIW, darktable only ever uses ColorMatrix1/ColorMatrix2 (whichever one is done for D65 or closest to it), and no other parts of the Adobe DCP, AFAIK.
Matrix3 should also work. With dng files all raw developers & viewers make use of those exif tags as specified in dng specs.
Unfortunately many vendors just donβt care. There were many issues and reports like this before.
Thatβs unfortunately a red flag that the manufacturer did not fully test their Camera2 implementation and reports default/filler metadata to Open Camera.
For a while even Pixels had this flaw - if you queried Camera2 for metadata like the color matrix, you got garbage. So at least for a while after release, the Pixel 4 XL had its color data overridden by the camera app!
A DCP from another phone isnβt guaranteed to work - one that shares the same sensor MIGHT work, but not all sensors share a CFA design, and sometimes the lens has significant color effects.
The big question is whether or not you can trust the GainMap - I would take a picture of a very flat evenly lit white surface (or even better, shoot a white wall while holding a translucent object right against the lens like a slice from a plastic milk jug) and verify that the raw, when processed with software that supports GainMap (not all do), it looks evenly lit. If you can, then the remaining task is to shoot a ColorChecker in daylight and tungsten to generate an appropriate profile.
@hannoschwalm ColorMatrix3 is relatively new, part of the DNG 1.6 spec which is almost entirely new tags Adobe co-developed with Apple, Iβm not sure if anyone other than Apple have implemented it camera-side yet.
Thatβs definitely the very precise answer that I needed, thank you so much!
This is overall pretty disappointing from Fairphone, and too much work for me as a beginner photographer.
You know what. I think Iβll drop the phone and settle for a proper camera, some of my relatives have an idle Canon G7x that I could make use of
You could open a bug report for the camera app
Iβm not sure about the Open Camera app being responsible, according to @Entropy512 itβs the phone manufacturer who messed up the phoneβs Camera2 API
implementation, on which Open Camera connects.
We can ask Open Camera app to take the Fairphoneβs lacking Camera2 API in consideration and provide sensible Color Matrix values instead, but Iβm not someone to ask that of an open source contributor.
Iβd have to look at the source for Open Camera, but I BELIEVE they try to use metadata provided by the camera HAL - which if the phone doesnβt offer raw capture itself may be incorrect. DngCreator Β |Β Android Developers
Thereβs not really any way for a camera app to fix this, except to detect phones with known-bad metadata and replace it. But thatβs asking a huge amount of the camera app developer, because correcting the data requires spending the time to characterize the camera, which requires owning that particular phone and a ColorChecker.
Reporting the problem to the hardware manufacturer is unlikely to go anywhere. The Android world is a shitshow in that regard. Iβve seen critical flaws remain unfixed for more than a year after not only was the problem reported to a manufacturerβs developer relations, but a fix was provided as a patch. (For example, the Galaxy S2 kernel had a bug in the power management, where it would enter a state that would keep the phone from entering deep sleep if the fuel gauge was critically low. For one, this seems ridiculous, if youβre almost out of battery why use more? They may have been worried about current consumption during resumeβ¦ But clearing the condition required the state of charge to be EXACTLY the threshold value. They used == instead of >= to clear the condition if the SoC rose above that threshold when on charger. I submitted a patch to all of my devrel contacts, and more than a year later, people still constantly complained about fuel_alerted wakelocks.)
After all, this is fairphone weβre talking about. A manufacturer who got so caught-up in anti-Qualcomm whining that they didnβt realize that for all of their flaws, Qualcomm was the least of all evils, and went with Mediatek who were blatant GPL violators instead for their first generation or two.
Well I would say open camera generates a problematic or at least unconventional DNG on my Pixel and its pretty mainstream so I just think its the way it takes the data and packages its version of DNG and since this differs likely from what is expected by standard protocol when opened in raw processing software you get a distorted imageβ¦ I might have to go back and see β¦there was another app called hedgehog2 or something like that. I think its really a different face on opencamera but maybe it offers a different DNG outputβ¦ I will dig it up and checkβ¦
EDITβ¦it was called Hedgecamβ¦ha ha β¦ you could give it a whirlβ¦
Sure! I tried Hedgecam2 and here are the dngs and the color matrix, next to each other.
opencamera.dng (23.3 MB)
hedgehog2.dng (23.3 MB)
ββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β File: hedgehog2_color_matrix
ββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
1 β Color Matrix 1 : 1.3828125 -0.65625 -0.2109375 -0.96875 1.875 0.0390625 0.0390625 -0.1484375 0.78125
2 β Color Matrix 2 : 2.2421875 -1.546875 -0.390625 -1.0703125 2.1796875 -0.0078125 0.0625 -0.140625 1.2578125
3 β Forward Matrix 1 : 0.4375 0.3828125 0.140625 0.21875 0.71875 0.0625 0.015625 0.09375 0.7109375
4 β Forward Matrix 2 : 0.4375 0.3828125 0.140625 0.21875 0.71875 0.0625 0.015625 0.09375 0.7109375
ββββββββ΄βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β File: open_camera_color_matrix
ββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
1 β Color Matrix 1 : 1.3828125 -0.65625 -0.2109375 -0.96875 1.875 0.0390625 0.0390625 -0.1484375 0.78125
2 β Color Matrix 2 : 2.2421875 -1.546875 -0.390625 -1.0703125 2.1796875 -0.0078125 0.0625 -0.140625 1.2578125
3 β Forward Matrix 1 : 0.4375 0.3828125 0.140625 0.21875 0.71875 0.0625 0.015625 0.09375 0.7109375
4 β Forward Matrix 2 : 0.4375 0.3828125 0.140625 0.21875 0.71875 0.0625 0.015625 0.09375 0.7109375
ββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Iβm afraid itβs all the same.
Iβll have to check β¦very interestingβ¦I seem to get much better DNG with the Pixel using itβ¦also it exposes quite a few options in bug fixes and other areasβ¦one is to use their DNG or android DNGβ¦also some other settings are there that could alter thingsβ¦ I am experimenting nowβ¦one thing I noted is the illuminants are way off comparing native camera app to Open cameraβ¦will have to check HGH2β¦ this is Googleβ¦
And this is open camera with D55 illuminant vs d65 and they are reversed??
Well for a phone image, in the end I donβt think its too bad. I wasnβt there to see those flowers but if you want a pretty picture you can get oneβ¦not sure if its better or worse than your experience but I took your image and did a quick editβ¦
Then added the legacy CB preset β¦more punchy
Then cheating because this has some interesting profile issuesβ¦ I tried substituting rec2020 for the input profileβ¦
I actually like the color this givesβ¦
And then on steroids adding the CB preset to this one.
Funny thing, at least when the Pixel 4 XL shipped, there was evidence that the HAL was reporting bad color metadata, because the issues I had with DNGs on day -1 (my phone arrived one day before the βofficialβ release date) were fixed by a day zero camera app update, not an OS/HAL update.
Interestingly - different ColorMatrix1/2 tags than the Fairphone, but the EXACT same ForwardMatrix tags. Thatβs deeply concerning.
Itβs clear that Google is overriding whatever the HAL provides and embedding a full blown DCP including HueSatMap 2.5D LUTs in every image.
What is CB ? And what is this preset you talk about ? Iβd like to reproduce it, will try with the sidecar xmp you provided.
rgb Color balanceβ¦there are 4 presetsβ¦by defaultβ¦
Its interesting when I make a DNG with DNG converter it sets Illum-1 to Std-A and illum-2 to D65. My pixel does the same for the native camera app but using opencamera it sets illum-1 to D55 and Std-a for two. The fairlight phone sets it to D65 for 1 and Std-A for 2. So correctly D65 but reversed and then has the same matrix issue⦠So I guess it buyer beware on how open camera might interpret and save the image data as a DNG⦠and how that might be interpreted as a raw file
As long as the CM matrices are correctly calibrated for the given illuminant, the order doesnβt matter as far as DNG spec (and darktable implementation) are concerned. If there is no D65 matrix, dt will look for the closest one and do chromatic adaptation on it. But if the matrices are garbage in, youβll get garbage outβ¦
@priort I managed to import your sidecar file, thanks!
Now what are those color matrices anyway ? I tried to look it up but thereβs a lot of math involved apparently, and Iβm just a humble developer. Anyone have a good explanation, or a link to follow?
This is a bit of math at least worth appreciating. Whether in-camera to a JPEG or raw file through a raw processor, the camera data has to be transformed in color down to a gamut of colors the rendition media can handle. There are a number of ways this can be done, but the simplest and most predominant method is a bit of linear algebra using two sets of 9 numbers, the first set describing the input data, and the second set describing the output destination.
The ColorMatrix sets weβve been discussing here are the first set; they describe the color response of the particular camera. These numbers are sometimes called the βcolor primariesβ, and they consist of three numbers for each of the red, green and blue encoding of color. The destination numbers describe the gamut of a particular display medium; when we export a JPEG, we usually use sRGB primaries because we donβt know what all is out there on which folks might display our images, and sRGB is a sort of least-common-denominator.
HTHβ¦