Is there an interest to support Canon EOS 5D Mark IV dual pixel raw files in RT?

(Ingo Weyrich) #1

As a fallout from work on Pentax Pixel Shift, RT dev version supports raw files with more than one frame.
Means one can also extract both frames of Canon EOS 5D Mark IV dual pixel raw files in RT.

The files contain one frame which is a+b and another one which is only b. That means it would be at least possible to recover clipped highlights which are clipped in a+b but not in b. Doing that on raw level would be better than in post processing. So, if someone here uses a Canon EOS 5D Mark IV and RT, please let me know :slight_smile:


(Ingo Weyrich) #2

Here’s an example. Left is frame a+b (the first frame), right is frame b (the second frame). Combining them in RT would allow to use the highlights from b while keeping the lower noise in darker regions from a+b:


@heckflosse is there your first step to include hdrmerge in RT?

(Ingo Weyrich) #4

No, that’s a completely different thing

(Ilias Giarimis) #5

Ingo, nice to see you have time and motivation to use the dual pixel properties for enhancing 5D4’s output :slight_smile:

Although there is a problem with these data because the 2 frames are not aligned … these is paralax offset depending on the used aperture and on how much OOF the objects are

Well … maybe even with no correction the result will be interesting …(increased blur at OOF areas )

I hope I will have time to participate 1-2 weeks later …

(Ingo Weyrich) #6

IMHO the frames are aligned in xy but not in z. The difference in z is the trick which makes dual pixel possible. But that’s just a guess. I have only three files to test…

(Ilias Giarimis) #7

Hmm … what do you mean by “z” ?

(Ingo Weyrich) #8

Ilias, afaik Canon dual pixel has this layout (ignore rgb)

ab ab ab ab
ab ab ab ab
ab ab ab ab
ab ab ab ab

where a combination of a and b is a ‘full’ pixel and ‘a’ and ‘b’ are half pixels. So they are perfectly aligned in xy (of course with half pixel distance in x). My guess is that b pixels are closer (or more far away) from lens a tiny bit to get the effect (z is the distance from lens).

(Ilias Giarimis) #9

Ingo, the xy arrangement is as you describe but there is no depth difference … the ab pair is under a single microlens and the result is each of the a/b sees only a part of the apperture (pupil). So for the OOF areas each one sees (and records) a different picture.
Try to cover the backside half of your lens, then the other half without changing anything else …
Bobn2 had pictured this at Dpreview forums but I cannot find the posts :frowning:

The result is shown in the gif I linked as displacement

If the pixels were normal (just not square, like your old Nikon ;)) we could take double horizontal resolution with 5D4 …

(Ilias Giarimis) #10

Found it …

Here’s a little example of sampling the light field from different directions. I took two photos from the same point of the same scene with the same lens (a 180/2.8). The only difference between them is in one I blocked off the left half of the rear element with a piece of black plastic foam, in the other I blocked off the right half.

(Ingo Weyrich) #11

Here’s a first example which uses frame 2 to get back some details in clipped regions of frame 1. Left is original frame 1, right is also frame 1, but uses values from frame 2 when pixels in frame 1 are clipped:


Very interesting!

Any news about this feature, and will it be in RT stable version?

And would it be possible to exract the 2nd frame raw data as a separate cr2 file?

(Ingo Weyrich) #13

Currently no news about this feature because nobody seemed to be interested. Maybe that changes now :slight_smile:

No, rt does not write raw files. But you can process each of the two frames separately and output as tiff, jpeg or png

(Sander Knopper) #14

I own a 5D Mark IV and am willing to help :slight_smile:

So what exactly do you need? Since there already is a raw file with dual pixel at

(Ingo Weyrich) #15

@saknopper I already have two raw dual pixel raw files. But that’s not enough for development. While developing Pentax Pixelshift for rt we used about 100 different raw files… :wink:

(Sander Knopper) #16

Makes sense :wink: What is the preferred method of providing you the images? Should I just upload the files to ?

Also, are you interested in specific kinds of images?

(Mica) #17

For testing, please do not put them on, we are looking for very specific things there.

You can share them on, dropbox, google drive, or whatever makes you comfortable! :slight_smile:

(Sander Knopper) #18

Alright! I will take some photos the next few days :slight_smile:


A general question: Is the parallax between the half pictures enough to compute a rough depth profile of the picture?

(Ingo Weyrich) #20

Don’t know currently. More sample files are needed.