Stacking Lights with Flats is Creating a Halo Effect

Hi Everyone,

I am working with some images I took with my Sony ZV-E10 using a Viltrox AF 13/1.4 E lens.

What I am noticing is that when I stack my lights with darks and flats (preprocessed with bias or with offset constant) I get this weird “halo” effect in my lights stack. The halo appears to correspond to the flats but is inverted. In the flats the center is light and the vignetting is darker. In the light stack the center is dark (or clear, maybe?) and the vignetting shows up as light.

I have reshot my flats a number of times now in a number of different ways and reprocessed everything. The exact extent of the “lightness halo” changes depending on the method chosen to reshoot the flats but the effect remains:

This screenshot is with a Histogram stretch to make it more apparent what I’m talking about but it is visible in autostretch as well. Additionally, as I stretch the image this halo becomes more and more visible.

If I exclude the flats and just stack the lights with darks the halo is not there.

I carefully went through the Synthetic Biases tutorial and determined an offset to use instead of biases (bias=512) but the halo is still there so it seems to be caused by the flats (this synthetic bias version is what the screenshot is of).

The only thing I can see is that the ADU values are much lower than I have read on some blogs and forums that asserted the flats should be in some mythical sweet spot around 50% to 75% of the range … these are far below this recommendation.

Is this an indication that I’m doing something wrong or maybe just an artifact of using this lens? Should I be shooting brighter flats somehow or maybe just somehow artificially boosting the level in the flats images so they are magically in that aforementioned sweet spot but maintain the same relative falloff and vignetting shape?

My current workflow is derived directly from the OSC scripts although I have adjusted a variety of things as I have tried to work through this issue. I have tried the Linear Fit Clipping and the Winsorized Sigma Clipping so far but haven’t tried any of the others. I have also altered the sigma levels trying to keep the rejections within the .1-.5% amount recommended in one of the tutorials.

For all I know this is something that can be ignored but since it shows up in the stretching I’m thinking I shouldn’t be ignoring it.

Thank you, in advance, for any help that may be given.

edit: Of course the more I stare at this the more this halo seems to resemble the images seen in the synthetic biases tutorial; specifically the L/F and the L/(F-O) examples so maybe there is somehow something wrong with my darks processing? Will have to read the tutorial more carefully to see if this is an error in my processing.

Hello, it’s quite tricky to do a correct calibration, as many parameters can vary. The procedure you describe seems correct, yet, your images seem to suffer flat overcorrection.

Flats are supposed to be shot with a short exposure, with a lot of light, they don’t need to keep the same ISO/gain or exposure as lights and darks, but their offset value needs to be understood and subtracted. If you’re not sure about the offset level, you might just take bias shots.

Darks must have same parameters as lights: ISO or gain, exposure, binning, temperature (or close enough), offset.

Hi vinvin,

As mentioned this effect shows up with biases, too, and first showed up with me just running the OSC_Preprocessing.ssf script and looking at the final results. My original resulting light stack had this weird “clouds” effect in that confused me because there were very few clouds visible when I was shooting.

I tracked those clouds back to my flats which I had shot through a white tissue paper pointed at my bright iPad screen. So I grabbed some other flats I already had for that lens and used those. The “clouds” went away but were replaced by this halo.

The latest version of the flats were shot in manual mode at 1/1600s, f/6.3, and iso4000 with the lens a few cm in front of an iPad with an all white screen set at maximum brightness. In this case the f/stop and the iso matched the lights and darks. I did this per this article: https://practicalastrophotography.com/a-brief-guide-to-calibration-frames/.

I only manually did this offset version after verifying the halo appeared even after reshooting the lights. The offset I used for the screenshot in the original post was derived from following the Siril Synthetic Biases tutorial from which my original biases (shot at20s, f/6.3, iso4000) yielded these ADU values:

image mean (adu) median (adu) sigma (adu) min (adu) max (adu) noise (adu)
1 510 511 21 0 979 20
2 509 507 21 40 1047 20
3 510 511 21 0 999 20
4 510 507 21 40 992 20
5 510 507 21 27 1084 20
6 510 507 21 8 984 20
7 510 511 21 31 987 20
8 511 511 21 19 1044 20
9 510 507 21 36 984 20
10 511 511 21 63 1096 20
11 510 511 21 8 1039 20
12 510 511 21 40 1016 20
13 511 511 21 40 1096 20
14 510 511 21 16 996 20
15 510 511 21 72 999 20
16 510 511 21 0 976 20
17 510 511 21 51 1039 20
18 510 511 21 59 1071 20
19 510 507 21 8 1036 20
20 510 507 21 40 1111 20
21 510 507 21 12 1031 20
22 510 511 21 67 1116 20
23 510 511 21 55 1011 20
24 510 511 21 23 1004 20
25 510 507 21 80 976 20
26 510 511 21 0 972 20
27 510 511 21 16 1039 20
28 511 511 21 4 976 20
29 510 507 21 55 1047 20
30 510 511 21 48 1056 20
31 510 511 21 40 1024 20
32 510 511 21 44 1007 20
33 509 507 21 36 1027 20
34 510 511 21 40 1004 20
35 509 507 21 27 992 20
36 509 507 21 31 1079 20
37 510 507 21 55 979 20
38 510 511 21 48 1011 20
39 510 507 21 59 1019 20
40 510 511 21 31 1016 20
41 510 511 21 51 996 20
42 510 507 21 63 1011 20
43 510 507 21 59 1007 20
44 510 511 21 16 1051 20
45 510 511 21 76 1024 20
46 510 511 21 51 999 20

I took the nearest power of two of the “median (adu)” column to get the bias=“512” value I used for the offset.

The reshot flats and manual step by step processing made the halo “thicker” but otherwise resulted in the same halo as the original automated processing with biases. Whether I processed the reshot flats with the script or by hand made no difference.

I have found so many articles describing so many different ways to shoot flats that I don’t know if I am shooting them in a way that is messing with Siril’s expectations. The only difference between the flats and the all the other images (biases, darks, lights) is that I shot the flats inside the house, later, so the sensor was at room temperature instead of the -7C the other images were taken at. The biases, darks, and lights were all shot in the same session, outside, in the same temperature and all within an hour or so of each other.

Using only darks with no biases or flats results in no halo.

I haven’t gone back through that Synthetic Biases tutorial yet but I am still wanting to figure out why my images seem very similar to the “L/F” and “L/(F-O)” images at the bottom of the article. If these are representative of what is happening to my processing it strongly suggests that my processing is somehow excluding my darks. I don’t know how this would be happening but until I can prove otherwise my current hypothesis is that I’m doing something silly that is skipping, or negating, my darks or that possibly there is something wrong with my darks (like maybe I chose my biases as the images for my darks or something else weird like that).

Any further suggestions or help is appreciated. If I figure something out I’ll update my findings, here.

Hello again @vinvin,

I’ve continued struggling with this and after experiencing this again from shots taken this weekend, and a fair number of other issues that crop up when I use flats on images taken with this 13mm lens and my Sony zv-e10, I tried digging deeper (source code reviewing, lots of scenario testing, etc.).

I spent some more time trying to figure out what was going on since this issue doesn’t appear to show up with my Nikon D7000 images. To rule out a Siril issue I did some stacking of these images in DSS and found the issue was there as well.

Yesterday I tried a large number of different ways of processing these images (Linear rejection instead of Windsor rejection, adjusting the rejected pixel counts via sigma adjustments, not debayering, debayering everything, etc.) but bizarre issues just kept showing up and leaving my lights stack with a giant dark eyeball and a bright halo.

However, yesterday I found a very obvious 12-ish sided polygon shape hidden in the picture data. My lens has nine aperture blades so I knew it wasn’t the lens and I was able to see this huge artifact on my biases so I figured it had to be the camera itself doing something with the RAW files.

For anybody else having these issue I’d like to explain some things I discovered this morning and it appears to be specifically related to Sony cameras. There are numerous posts about it but I would never have found any of them if I hadn’t seen those giant 12 sided polygons hidden in the data. If interested here is a like to one such discussion from over five years ago: Re: A7R corner color cast: Sony Alpha Full Frame E-mount Talk Forum: Digital Photography Review

My camera and lens are both newer than the cameras and lenses mentioned in these (and related) posts but the zv-e10 is built from older Sony tech so it may, or may not, have these physical cross talk issues but almost certainly does have the “vignetting fix” firmware issue that is also causing, and/or worsening, the effect.

I am currently trying to figure out how to get rid of the issue by attempting to make sure that all “Lens Comp” settings are off. This probably will not work because apparently the firmware item that is causing, or worsening, the issue is not user accessible and many folks have reported the effect even when the compensation settings were turned off.

If that doesn’t work I’m looking into other ways to adjust the workflow that might work around the issue. This might involve using something like “cornerfix” to undo the issue, processing things differently using dcraw and/or imagemagick (flat field processing), or maybe coming up with a pixel math solution that could account for these things all within Siril.

The cornerfix website says: “CornerFix corrects for vignetting effects in digital images. It’s most often used to correct for the color vignetting that occurs when wide angle lenses are used with digital sensors that include IR filters, resulting in so-called “Cyan Corners””. It may, or may not, help correct the problem but I’ll have to figure out where and how to use it in the workflow if it is going to work, at all.

While my 13mm lens can certainly be described as a “wide angle lens” and therefore could be expected to display this issue badly if it is indeed the cause of problems I found plenty of folks’ posts that were experiencing the problem on 50mm+ focal lenses and I don’t normally associate those lengths as “wide angle.”

Mostly to help future readers I’ll come back and post what I figure out but if anybody else already knows about this and how to work around it in Siril workflows I’d love to hear about what you know.