Hi @snibgo I’ve made a big update of my initial post. To summarize:
I take the median of images as a reference point for comparison. The differences with the mean were not shockingly big.
I calculate the RMS in Lab space to determine similarity. This has changed the order quite a bit, but now I’m much more confident about the correctness of the results.
I scale the images down before determining similarities, purely for performance reasons.
Thanks to a tip from @ggbutcher rawproc is now included, but I think I still have done something wrong, because from visual inspection it shouldn’t be last in the rankings. Something for a future update.
I’ve included Scene 8 that contains a color chart. This one is very interesting to examine further. There are quite big differences, and all non-commercial programs end up last in the list.
As an example, look at the differences between darktable (first), DxO PhotoLab (second) and RawTherapee (third).
It would be interesting to see a mosaic of small thumbnails. Columns or rows of same scene across software. Just to know if the difference is perceptible at that scale.
Flipping throught those colour chart images at high speed was very interesting. Very different processing. Darktable blue patch really stand out.
RT and DXO colour chart looks quite similar but the red flowers differ a lot. Very interesting stuff!
Oh, don’t be so quick to indict your process; there HAS to be something wrong with rawproc…
If you don’t mind, can you send me the TIFF? With that, I can inspect both the image itself and the rawproc toolchain, which is in the metadata… I wouldn’t want you to spend quality time chasing a problem in your chain if there’s something in mine…
Peculiar that the darktable one seems to only have the best pastel representation (looking at the warmer/colder section, top part of the colour checker). The other 2 seem to have trouble with that with the default settings used.
Overall DxO and RT seem to have the more pleasing colour and detail representation. RT has a bit more detail compared to DxO and I think I like the colours just a bit better in the DxO version if I do not look at the colour checker. If I do look at the checker I prefer the RT colours.
Assuming that this would be a starting point for users that use these defaults I don’t think there’s much of an influential difference between 2 and 3.
They are probably pushed out by the contrast in the DXO RT sampes?
It’s interesting how different the flowers are. I guess that’s a sign of different handling of extreme colours. (a frequent topic on these boards)
I think good* handling of extreme colours and contrast is one area where defaults really matter. Not knowing what the flowers look like make it difficult to decide though.
I agree. It is the main reason why I don’t like using all these defaults. As mentioned before I like my starting point to be as bare-bones as possible.
Not knowing what the flowers/subject actually looked like does make it harder.
looking at the chart/flowers again it does seem like perhaps the difference in red yellow and orange patches might translate to the flowers. What do you think?
Yes, the handling of very chromatic colors is a bit of an unknown. For the patches of the CC24 target (a bit less chromaticity(I think all of those patches are well within sRGB, please correct me if I am wrong EDIT: I was somewhat wrong, see @snibgo 's post)) though I think it’s fair to say that the defaults used here are not particularly reproducing what they should be.
For people not having a CC24 target, the default maybe only for this camera is not colorimetrically ‘good’. I must repeat that for stylized looks this is to be expected. So I am not sure yet what to make of all this, yet.
I’m trying to understand why I like the colour of the flowers better in the DxO version but prefer the colours of the ColourChecker in the RT version.
I do think that @PhotoPhysicsGuy’s remark about rendering/handling very chromatic colours lies at the base of that. If that is the case then I would definitely choose the way RT applies its defaults.
But the slight contrast difference also plays a roll, at least it does for me.
FWIF, RawTherapee provides a dual illuminant custom-made DCP file shot under good conditions for the calibration. So that the CC looks good is a nice confirmation.
Of those three images, which would I prefer as an initial rendering?
I prefer the tones of the darktable version, but I’m not keen on the colour patches, especially the saturated red and blue in the third row.
The other versions have increased mid-tone contrast at the expense of crushed highlights. This makes the foliage pop, because that doesn’t contain highlights. Of course, the software may have noticed the image has very little highlight areas, so may be deliberately lowering highlight contrast.
Makes for very interesting comparisons of the tricky files such as the led bridge and the flowers. It also makes it obvious that simple scenes are simpler and less vulnerable to dodgy settings.
Looks to me like the commercial products are a lot more consistent with one another, and when the FOSS comes closer to the top (like the foggy forest or the landscape with the river), it’s because the FOSS ones are all biased one way (usually darker).
One thing to keep in mind is that the contrast of RT/ART will mirror your sooc settings. So there’s no default rendering. If you set your camera to Bright your out of the box RT/ART image will be bright and contrasty. If you set it to Flat it will be flat. High key will shift it, etc, etc.
I shoot with a customized flat setting with my camera so I get a out of the box image that suit’s me. The colour profile will be the dual illuminant auto matched unless I prefer the embedded out of camera dng matrix. So I can get more accurate colours but the contrast curve i set in camera. (the dng matrix is a bit desaturated which I actually quite like)
The stupid thing is that I didn’t fully realize the utility and genius of this despite using it. Two posts in the other thread made me understand it fully.
What it means is that you control important aspects of the default rendering in camera. What you see from those software above is actually what the photographer has chosen. I only considered it a sooc jpeg setting. Not my sooc jpeg setting.
If you had them as layers you could turn them off and on in difference or divide blend modes to visualize in another way how each part of the image deviated from the jpg…
I’m assuming the thumbnails are ordered by similarity?
It seems to me that the most different thumbnails are always different because of brightness differences.
I don’t quite know what to make of that. Perhaps one could do another comparison with an L* correction? Add/subtract the mean L* difference before comparison, to mitigate the worst of the brightness difference.
Perhaps that is misguided, though. If we’re comparing defaults, then we shouldn’t mess with the data.
At any rate, this comparison somewhat matches my own results from a few months ago: the default rendering of any of these tools is fine as a starting point. All of them are probably capable of editing most pictures if used well. The differences are in the details, and in UI design, and by and large probably more of a creative choice than a technical one.
