Handling Fujifilm X-Trans Auto Focus Pixels

[setnes (Kristopher Setnes)]

The current logic is working only with cameras that use the X-Trans III sensor. The X-T3 uses the X-Trans 4 sensor.

The X-Trans 4 sensor has an auto focus area that covers the majority of the sensor. There might be an advantage to applying this logic to that sensor. It should be possible to see the pattern in a stacked dark frame. Since the pattern covers the entire sensor, however, the normal astrophotography practice of dithering the mount between each exposure might be just as effective at eliminating the pattern noise. It wasn’t a priority to handle these sensors when making the logic work.

The X-Trans III sensor has an autofocus area that covers only part of the sensor. This is why it was so important to have a fix for that sensor. It presents a problem that cannot be fixed by dithering alone.

The logic for this proved to be a bit more complicated than what’s in the original code I shared. The X-Trans III sensors can have different starting points for the RGB pattern. This might be related to how the sensors are manufactured when the color filter array is added to the wafer of sensors before they are cut. We’re unsure. In addition to the shifting RGB pattern, the sensors can also have different auto focus pixel locations. What this means is a given RGB pattern could have different auto focus pixel locations compared to another sensor with the exact same RGB pattern. Locating the pixels to use for a correction becomes a problem.

To solve this, the logic actually looks at the RGB pattern and computes the correction based on four possible locations of the auto focus pixels within the RGB pattern. The correction, and pixel locations, that has the largest difference is what gets used as a correction.

When writing the logic that “detects” the location of the auto focus pixels, we also made it only look at green pixels when computing the average of non-auto focus pixels. Normally, you would only need to correct the master dark and master bias frames to correct this artifact. For those corrections, the color of the pixel used in the computation does not matter. I tested an OIII filter with my Fujifilm X-T20, however, and determined that images with very heavy color casting could cause the pattern to be visible in light frames. If the logic is applied to light frames, the color of the pixels used by the logic suddenly matters. It probably isn’t normal to use a narrowband filter with a color camera, so this falls outside of the expected use and hasn’t been heavily tested, but the underlying logic is setup to be used on light frames if necessary.

Sorry for the long post. I never explained all of this on this thread. It was fun working with @lock042 on this code.

[lock042]

@setnes

Sorry for the long post. I never explained all of this on this thread. It was fun working with @lock042 on this code.

And you did a really, really good job. Thanks for the nice explanation.

[erratasama]

I’m on 0.99.6 currently, and I’m strangely getting this artifact sometimes, but not always, and am curious what I’m doing wrong. My master dark and master bias both have this artifact, but typing in “fix_xtrans” doesn’t fix it, and also claims that the X-T1 is an unknown camera type. Do you guys need some files? I was going to try putting in the preferences on my own, but I can’t really decide what is and isn’t an AF pixel from what I’ve got, and would like to be accurate.

Update: I ran “preprocess dark -fix_xtrans” and “preprocess bias -fix_xtrans” and then stacked those with no errors, but also to no avail–the square is still present in the dark and bias frames.

[lock042]

HEllo, for now, camera handled:

		{ "Fujifilm X-T2",   { 1510, 504, 3009, 3019 }, { 1992, 990, 2048, 2048 } },
		{ "Fujifilm X-T20",  { 1510, 504, 3009, 3019 }, { 1992, 990, 2048, 2048 } },
		{ "Fujifilm X-Pro2", { 1510, 504, 3009, 3019 }, { 1992, 990, 2048, 2048 } },
		{ "Fujifilm X-E3",   { 1510, 504, 3009, 3019 }, { 1992, 990, 2048, 2048 } },
		{ "Fujifilm X-H1",   { 1510, 504, 3009, 3019 }, { 1992, 990, 2048, 2048 } }

I have X-T1 files with no visible pattern. Of course, If you have a bunch of dark files (60s exposure) that could help us.

[erratasama]

Thanks for the quick response! Do you have a preference for ISO? How many darks do you want? Uncompressed RAF, or just compressed FITS? I’m on a terrible connection, but can have them taken and slowly uploading soon.

[lock042]

10 darks in RAF format should be fine.

[erratasama]

Finally got it up! Here it is: https://ttm.sh/hlG.7z

[lock042]

OK; so I don’t know what is your artifact because it is not the one we are talking about in this thread.
I think you have another issue, not related to Xtrans.

[erratasama]

Oh, strange! I didn’t check these in siril before I uploaded them. You’re right that the artifact in those tests is weird, and I have never seen that in a stacked dark before. Neato!
For comparison, this is a stack of 150 darks at .67 seconds each. I’m really surprised the 60s ones went crazy like that.
(Note: I just checked the image again, and you can definitely see the AF pixels in the center also being out of sorts; it’s just not as pronounced)
dark_stacked.fit.txt (11.6 MB)

[lock042]

ok thx.
Very difficult to get square coordinates.

[erratasama]

After my battery’s charged, I’ll do another run of 10x60s; sorry about the delay on all sides.

[lock042]

@erratasama
Choosing this value

Maybe @setnes will be better than me to retrieve the right coordinates but it looks not so bad.

[erratasama]

Using some scrying and a bunch of other stacks (and then semi-verifying with setnes’s technique), I think I have the right AF pixel arrangement. 1480x,997y (Just like yours), but spanning 1972w, 1313h. I’m including everything that seems part of the grid pattern.
What I’m unsure of is what the significance of the Sample Pixels aspect is. Does that need to encompass a specific grouping of the rest of the image?
Thanks for your work on siril, by the way; I’m getting great images, even with the artifact.

(Edit: Looks like using your sample pixel settings with my AF pixel settings is working really well; I’ll do a real test while I head to bed! Thanks!)

[lock042]

@erratasama

but spanning 1972w, 1313h. I’m including everything that seems part of the grid pattern.

I think you’re right.
Next version will have the X-T1 DSLR. Thanks!!

[erratasama]

Thank you to both @lock042 and @setnes. I’ll hopefully have something worth posting eventually, but for now, this is already going to improve my terrible untracked shots.

[setnes (Kristopher Setnes)]

I’m just now looking at your file. I am looking at the fits file you created from 150 short exposures. There’s a lot of vertical banding, but I can still easily see the auto focus pixels.

This is the first data from an XTrans II sensor I’ve looked at. When we set the area for the XTrans III cameras we were able to get a box that worked for all four possible locations of auto focus pixels. I assume we might need to tweak the edges the same way for XTrans II, but we will probably need to see data from other cameras. The numbers you provided certainly work well on this data.

The sample size is really just to prevent looking at all of the pixels when computing the correction to apply. In the case of the XTrans II cameras, I think it makes sense to just use the entire auto focus area as the sample area since it’s a small piece of the sensor. I was initially unsure how amp glow could skew the correction, so avoiding the edges of the sensor seemed like a good idea. That’s not an issue here either. The box is nowhere near the edges.

Thanks for sharing this data!

[lock042]

@setnes

Thanks for sharing this data!

And big thanks to you for your wonderful contribution to the FujiFilm hack code :).