Computational DOF and Bokeh

Ok, I never thought of the fact that most cameras already have pretty accurate depth sensors for AF. Dual pixel improves the image coverage by a mile, but still that data is not available for the user.

The k-lens approach is essentially light field processing, which can do refocusing/artificial bokeh without actually computing a depth map. Of course 3D (ok, 2.5D) imaging is also possible with such a lens.

@Tobias I’m not sure if pixel shift would work, the base line is very small, but that’s just gut feeling.

In theory it also should be possible to calculate a depth map by depth-from-defocus, which requires blur estimation. The current focus detection mode is based on edge detection if I’m not mistaken, but its gradient classification might be a base for blur estimation. I think our local hero @aurelienpierre also has fought with deconvolution sharpening at some point, that might also be a way in.

Does anyone know if it is possible to get the AF information with magic lantern? (I have no Canon.)

If you want computational bokeh you need to do it in the camera. Just as the phones use multiple exposures or even the multiple lenses to help calculating the depth map. I think doing it post mortem from a single photo might not get you as good results as the phones.

Apparently it’s already contained in the raw file:
"During Dual Pixel RAW shooting, a single RAW file saves two images into the file. One image consists of the A+B combined image data and the other only the A image data. This means the Dual Pixel RAW files contains both the normal image and also any parallax information, which can be measured and subject distance information extrapolated. As Dual Pixel RAW images contain two images they are therefore double the file size of normal RAW images. "

source:

Dual Pixel RAW

At least in past it was a problem that one raw frame is Image A while the next raw frame is Image A + B, which are encoded using the same bit depth, means that A + B is often clipped and B can not be recovered completely…

As anything else, it’s just a matter of getting the maths right and respect lights transport models.