I also have a 3a and am astounded at the quality of the jpeg’s that come out.
The toning is really amazing. I actually found an interview with the guy who was managing imaging at google, and subsequently left to work on imaging elsewhere. He was speaking about it much like a photographer developing a raw, saying they were trying to reach the right balance, contrast, brightness, color…etc. Interesting read, cant remember where I found it. he said he left partially because he felt the work was pretty much done.
I have to agree with him. Looking at the raw images, they are unremarkable in nearly every way. Yet the toned jpegs are spectacular. Its really something. I guess all that expertise and effort going into getting the best out of one particular hardware configuration may be part of the explanation. But i cant help but feel there is some ‘magic sauce’ that would be difficult to replicate with a raw developer, whether RT, adobe or what have you.
RAWs saved by Pixel phones are saved out after the tiled align-and-merge but before tonemapping AFAICT. It appears that when saving out a RAW, Google does not use their multiframe superresolution algorithm (which replaced the tile based align-and-merge in recent Pixels, and I believe may have been retrofitted to older ones with an update), saving out a cropped image that is still Bayer-mosaiced. This limitation is especially apparent at higher digital zoom settings, at which point the JPEG is vastly superior in sharpness due to the multiframe superresolution algorithm. An attempt was made to create an opensource implementation of the MFSR algorithm, but it has stalled - GitHub - JVision/Handheld-Multi-Frame-Super-Resolution: An implementation of the paper Handheld Multi-Frame Super-Resolution
Note that the tonemapping algorithm described in the whitepaper is a variation on the Enfuse exposure fusion algorithm, where two synthetic images are generated from the original raw image at different EV compensation, and then exposure fused.
You will never be able to reproduce Pixel JPEG tonemapping in darktable because the official position of the darktable development team is that Google’s algorithm is “pixel garbage”.
At least with current opensource tools, the closest you’re likely to get is RT’s Dynamic Range Compression tool - Fattal '02 doesn’t always provide as good results as the Google approach, but it’s quite good in nearly all scenarios, enough that I have basically not bothered reimplementing the Google algorithm in RT as I’ve got higher priority projects. One case where Fattal '02 definitely fails compared to Google’s approach is handling highly saturated monochromatic (e.g. colored LED) lighting. I have yet to find a way to come close to matching Google’s approach here without major hassle.
Alternatively you might be able to get some interesting results using LuminanceHDR - as I said, RT’s implementation of Fattal '02 is good enough for the majority of my own use cases so I haven’t played with LuminanceHDR yet.
But you would expect Jpegs to look better than raw images straight out of camera. The beauty of RAW is not what the unprocessed file looks like, it’s what the processed file looks like. Do you really think the OOC Jpeg in this thread is superior to all the attempts people have made on the raw file? I don’t. Google may apply some special sauce, and that is likely good enough for people who want to share phone photos quickly - but that is not raw shooters.
The HDR+presentation is very illustrative and perhaps easier to digest for most people.
dt includes Local Laplacian Pyramids, which is supposedly what Google HDR+ also uses… OTOH, Google might be doing other “clever” stuff as well… So the “pixel garbage” you might be referring to is for the super-resolution part only?
Laplacian pyramids are a general image processing concept. There are many use cases for them, and dt’s LL implementation and the Mertens exposure fusion algorithm are very different. Among other things, the dt workflow of running LL after a tone curve is attempting to reconstruct local contrast after it has been lost, as opposed to preserving that information in the first place.
dt did implement the Mertens algorithm - compressing dynamic range with exposure fusion | darktable - but that implementation is very broken due to not catching some subtleties in the original Mertens paper which are only clarified deep within the bowels of the enfuse source code (which is kinda hard to read/follow…). Attempts were made a year or two ago (as a result of COVID my sense of time is severely impaired…) by myself and another developer, and it was made clear that rather than fixing the algorithm it was to wither and die in its nonfunctional state. It is explicitly the Mertens algorithm that was declared as “pixel garbage” despite the numerous examples of it being put into practice with great results.
As to Google also doing other “clever stuff” - yeah, while exposure fusion is a solid algorithm that has almost always delivered great results for me, Google’s HDR+ whitepaper indicates that they were solely using luminance as their metric and there is NO way that it is the sole metric in the tonemapping flow for Night Sight, because using luminance only leads to bad results with highly saturated near-clipping reds and blues (such as red, blue, or purple LED lighting). Night Sight also changes the AWB algorithm significantly, USUALLY for the better, but NS always fails white balance in Lynah Rink at Cornell University.
Good link on hdrnet, and apparently there is now a published academic paper on Night Sight on Sam Hasinoff’s page at Sam Hasinoff - The Revolutionary Home Page - side note as far as history of papers on the subject, Hasinoff was one of the coauthors of the 2011 LL paper, but by 2016, he was a coauthor of the HDR+ academic paper where the older Mertens fusion algorithm was chosen over the 2011 LL approach. Also, the description of the 2011 LL algorithm states “no haloing”, but Figure 13 of https://people.csail.mit.edu/hasinoff/pubs/ParisEtAl11-lapfilters.pdf clearly shows haloing. No algorithm is perfect, they all have their individual strengths and weaknesses, but certain developers insist that the 2011 approach is the One True Way due to the claim of “no haloing” despite the clear evidence in figure 13 that it suffers from the exact same limitations (visual artifacts when used improperly) that cause them to declare anything else which can ever fail if used improperly to be “pixel garbage”.
Edit: Yup, Night Sight still uses exposure fusion but with a significantly enhanced weighting metric beyond simple luminance. https://people.csail.mit.edu/hasinoff/pubs/LibaEtAl19-nightsight.pdf is a very good read on AWB algorithms, exposure merging (I think some of those techniques need to be brought into the Tim Brooks implementation because they are PERFECT for MILC/SLR bursts), and tonemapping.
