In light of the heavy chatter about presets and default looks and which program does it “best”, I decided to do a small experiment. I took 7 images from my collection of gathered images from the past years frequenting this forum (am I sorry, I haven’t spent time tracking the exact sources, but some images may seem familiar). These images are very different to each other, either because of their file format, sensor format, expose, dynamic range, noise level, etc. but I’ll not claim these images to be fully representative for my purpose.
Question: if I take the default processing of various raw conversions using different programs, average them and then compare each version with the average, who is closest to the average, the “hypothetical best default”?
Methodology: open the RAW file in the editor of choice, do as little processing as possible (in most cases, zero processing*), export as 16 bit TIFF, load these files in a custom script, align and crop away edges so that we compare identical pixels, resize to the size of the embedded JPEG (this is the closest I can get to the OOC JPEG - not every vendor embeds a full size JPEG) or 3000 pixels at most (update: added size restriction to improve comparison speed), convert all versions including the OOC JPEG to L*a*b* and average** across channels, then recombine to form the final image, then calculate the
simple Euclidean root-mean square distance between the average and each original.
* I disabled lens distortion correction if it was enabled, or left out the image if I couldn’t (looking at you Photoshop ). In two cases the embedded JPEG had distortion correction applied, this introduces some error I cannot mitigate.
I now take the arithmetic mean, but as people already point out, this may not be a great choice. Update: I take the median value, which is less biased to outliers. The differences between the mean and median images were relatively small.
Software: the chosen editors are, in alphabetical order: Affinity 1.9.4, ART (latest dev), Capture One 20, darktable (latest master), Digikam 7.2.0, DxO Photolab 4.3.0, Filmulator (latest release), Lightroom Classic 10.2, LightZone (latest release), Luminar 4, ON1 Photo RAW 2021, PhotoFlow (latest release), Adobe Photoshop 2020, RawTherapee (latest dev), rawproc.
I have omitted Photivo, because it could not process most of my samples properly.
I have also omitted rawproc, but am in contact with Glenn, because I saw some weird behavior. Update: rawproc is now included.
Whenever an editor could not open a file (e.g. because it doesn’t support X-Trans sensors), those are obviously not included in the analysis.
This is going to be a long list. For each list, the smaller numbers mean a better similarity. The image shows the average. I have yet to think of a good way to share the other images if you want to compare them yourself…
Scene 1 - Wide dynamic range, sun clipping
0.00578805 Affinity 1.9.4 0.00876031 Luminar 4 0.0106033 ART 0.0113892 Capture One 20 0.0166794 RawTherapee 0.0206756 OOC JPEG 0.0232419 ON1 Photo RAW 2021 0.0235942 LightZone 0.0253301 Lightroom Classic 10.2 0.0253313 Photoshop 2020 0.026776 Filmulator 0.0476889 darktable 0.0695579 PhotoFlow 0.095769 Digikam 0.198581 rawproc
Scene 2 - Wide dynamic range, detailed, some clipping
0.0269101 Photoshop 2020 0.0269101 Lightroom Classic 10.2 0.0295188 LightZone 0.0316127 OOC JPEG 0.0358251 Capture One 20 0.037328 DxO PhotoLab 4.3.0 0.0427293 ON1 Photo RAW 2021 0.0450921 ART 0.0533722 RawTherapee 0.0615355 PhotoFlow 0.0617971 darktable 0.0668121 Affinity 1.9.4 0.072927 Filmulator 0.0773843 Digikam 0.0978238 Luminar 4 0.178724 rawproc
Scene 3 - Noisy scene
0.0342636 Luminar 4 0.0356646 Affinity 1.9.4 0.0357915 DxO PhotoLab 4.3.0 0.0416407 ART 0.0453766 Photoshop 2020 0.0453766 Lightroom Classic 10.2 0.0487241 Capture One 20 0.0690045 ON1 Photo RAW 2021 0.0732908 darktable 0.0748285 RawTherapee 0.0753369 OOC JPEG 0.102341 Filmulator 0.103192 PhotoFlow 0.107457 LightZone 0.137185 Digikam 0.240059 rawproc
Scene 4 - Typical bright landscape with sky
N.B. OOC JPEG had distortion correction applied, so some haze visible because of that.
0.0285808 Affinity 1.9.4 0.0297209 ART 0.0311513 Filmulator 0.0327813 Luminar 4 0.0358095 RawTherapee 0.0374848 Capture One 20 0.0390227 ON1 Photo RAW 2021 0.0478497 DxO PhotoLab 4.3.0 0.0494095 darktable 0.0635937 Digikam 0.0834656 OOC JPEG 0.0844319 PhotoFlow 0.103437 LightZone 0.216024 rawproc
Scene 5 - Bright blues, sharp small contrasts
0.0255882 darktable 0.0282571 Luminar 4 0.0336957 PhotoFlow 0.0392326 Digikam 0.0482835 ART 0.0485683 Affinity 1.9.4 0.0587456 RawTherapee 0.0631657 Capture One 20 0.0632366 LightZone 0.116961 rawproc 0.123669 OOC JPEG
Scene 6 - Blown highlights, strong CA
N.B. Some misalignment present.
0.044375 Luminar 4 0.0545543 Filmulator 0.0590963 ART 0.0604006 PhotoFlow 0.0615064 Affinity 1.9.4 0.0650114 DxO PhotoLab 4.3.0 0.0670756 RawTherapee 0.0722563 darktable 0.0725098 OOC JPEG 0.0778986 Capture One 20 0.0783475 Photoshop 2020 0.0783475 Lightroom Classic 10.2 0.0793131 ON1 Photo RAW 2021 0.0804826 LightZone 0.11173 Digikam 0.19541 rawproc
Scene 7 - Overexposed clouds, bright scene
0.0122128 Photoshop 2020 0.0122128 Lightroom Classic 10.2 0.0165731 Luminar 4 0.0175009 OOC JPEG 0.0182036 Affinity 1.9.4 0.0188194 ART 0.0211127 RawTherapee 0.0241559 ON1 Photo RAW 2021 0.0319731 Filmulator 0.0324239 DxO PhotoLab 4.3.0 0.0324248 LightZone 0.035527 Capture One 20 0.0542818 darktable 0.0932279 Digikam 0.0975635 PhotoFlow 0.184258 rawproc
Scene 8 - New: bright red hue and color target
0.03284 OOC JPEG 0.0385984 ON1 Photo RAW 2021 0.041847 Capture One 20 0.0430696 DxO PhotoLab 4.3.0 0.0446965 Affinity 1.9.4 0.0475985 Photoshop 2020 0.0475985 Lightroom Classic 10.2 0.0481924 Luminar 4 0.050561 Filmulator 0.0526564 RawTherapee 0.0596389 ART 0.0714936 darktable 0.0756244 Digikam 0.0824655 LightZone 0.0863091 PhotoFlow 0.179325 rawproc
Summary of ranks
Abbreviations should be self-explanatory:
1 Aff PS Lm4 Aff dt Lm4 PS OOC 2 Lm4 LR Aff ART Lm4 Film LR ON1 3 ART LZ DxO Film PF ART Lm4 C1 4 C1 OOC ART Lm4 DK PF OOC DxO 5 RT C1 PS RT ART Aff Aff Aff 6 OOC DxO LR C1 Aff DxO ART PS 7 ON1 ON1 C1 ON1 RT RT RT LR 8 LZ ART ON1 DxO C1 dt ON1 Lm4 9 LR RT dt dt LZ OOC Film Film 10 PS PF RT DK rp C1 DxO RT 11 Film dt OOC OOC OOC PS LZ ART 12 dt Aff Film PF LR C1 dt 13 PF Film PF LZ ON1 dt DK 14 DK DK LZ rp LZ DK LZ 15 rp Lm4 DK DK PF PF 16 rp rp rp rp rp
Some conclusions: there are probably many things you can say about the results. I have not studied them extensively yet, but a few things to mention:
- No program is a clear winner (it was not a competition anyway)
Photoshop and Lightroom are not always equal
- Being low on the list means you’re different, not necessarily worse
- Whether this average could be considered a better starting point for further editing than any of the individual options is debatable.
- This says nothing about how easy it is to improve the result (sometimes a single slider or setting can make a world of difference).
Please feel free to ask for clarifications. Looking forward to hearing some opinions or ideas for further comparison.