Dark flats Rejection Percentages

Software Siril
1

Hi,
I have recently transitioned to a cooled Astro camera from my trusty DSLR. FYI the ZWO 533 MC Pro.

I am using Sirils manual preprocessing tutorial, as I always have done. One difference is that I am using Darkflats rather than Bias files.

I read on another post to simply substitute the Darkflats for the Bias processing. Which I have done, at the next step however I have pre-processed the flats using my stacked dark flat ( just as I would have done with the master bias).

When it then comes to generating the master flat through stacking of these preprocessed flats, I get large rejection percentages:-

14:07:12: Starting stacking…
14:07:29: Pixel rejection in channel #0: 14.286% - 0.159%
14:07:30: Rejection stacking complete. 30 images have been stacked.
14:07:30: Integration of 30 images:
14:07:30: Pixel combination … average
14:07:30: Normalization … multiplicative
14:07:30: Pixel rejection … Winsorized Sigma Clipping
14:07:30: Rejection parameters … low=3.000 high=3.000
14:07:30: Background noise value (channel: #0): 11041.700 (1.685e-01)
14:07:30: Saving FITS: file pp_flat_stacked.fit, 1 layer(s), 3008x3008 pixels

Even setting the Sigma Low to 10 still gives a 2.3% rejection rate, My DSLR using the master bias files never gave this sort of figure so it has caught me by surprise.

If I continue to process (using Sigma Low of 10.) then I get decent images but it is making me wonder I I am doing something wrong here.

Any suggestions welcome?

2

I would bet you did make dark optimization.
And probably you have clipped a lot of pixels in the dark frames by subtracing biases.
I then recommend to not use dark optimization.

Oh. It is on flats. So I bet on dark flats level too high. Whish is the same.

3

This is rather strange, could you share one flat and one dark flat images?

4

Or maybe a few of the flat frames are too dark in some areas compared to others, make sure they all look good

5

Compare level of master bias and of your master flat dark

6

How do you create the flats? Which lightsource? Which exposure time? If you use a lightsource where the brightness is controlled by PWM or coupled to the frequency of the electric mains there can be very different flats when the exposure time is short (in the range of the modulation of the flat panel) - or you even can get “bands” of different brightness. These effects should be visible in the flats at least when you look at them with auto- or histogram-stretch.

Ciao, Udo

7

Thanks all for the speedy response. I’ll try and answer all below:-

@Udo_Baumgart I use the NINA program and use the Flat Wizard. I allow it to choose the exposure time. I use an iPad set to brightest screen and display a pure white. In effect I use the t-shirt method, I have some doubled white cotton material stretched across one of this circular Cross Stitching frames. I will attached an example flat and a dark flat. In this case it is a 7.6 second exposure, I tell the flat wizard that I want a 50% histogram (around 32000) with a 5% threshold.
A typical flats Histogram, shown in ZWO fitsViewer App:-

image
image808×506 127 KB

All the flats look very similar (I take 30).

A typical Dark Flat (obvs same exposure):-

image
image826×557 141 KB

In terms of Bands in the flats - when auto stretched there is a definite blotchiness…
image

@vinvin I have had a look through the Flats and Dark flats, they are pretty consistent, no major changed in brightness between the frames.

@lock042 you are correct - NO Dark Optimisation as this is flat processing, and as I have moved over to this astro camera I no longer use Dark optimisation as part of the later pre-processing of lights - I did do this on the back on one of the FAQ’s when I have my DSLR.
Dark Flats level - with my limited experience this looks ok - see the histogram for Dark Flats above.

Appreciate you guys taking a look at this. Some raw files and the master dark flats are shown below.
FLAT_2022-12-30_11-13-44_7.60s__0005.fits (17.3 MB)
DARKFLAT_2022-12-30_11-19-27_7.60s__0004.fits (17.3 MB)

Master DarkFlat:-
darkflats_stacked.fit.zip (10.3 MB)

8

Thanks for sharing the images.
Green is higher than the others, but it shouldn’t be a problem. I don’t understand why your tool shows the three histograms at the same level and where the high rejection comes from.

Red layer: Mean: 24682.8, Median: 24688.0, Sigma: 366.8, AvgDev: 291.9, Min: 22436.0, Max: 26508.0
Green layer: Mean: 40344.9, Median: 40344.0, Sigma: 454.7, AvgDev: 363.1, Min: 37432.0, Max: 42888.0
Blue layer: Mean: 25404.1, Median: 25404.0, Sigma: 338.3, AvgDev: 269.8, Min: 23120.0, Max: 27028.0
9

@vinvin, thanks for your followup, I have been comparing histogram displayed for my files in Siril vs the ones in ASIFitsView, I think there is a problem with the ZWO ASI FitsView application (I run the Mac OS version), the stats it gives are simply wrong and in some cases the histogram makes no sense - especially when viewing a file that has been processed by Siril. This gave me a clue, which I have confirmed. The ASI View Application cannot work with 32 bit floating point Fits files. If I save the file as a 16 bit unsigned integer format file from Siril then ASI opens it and the stats and histogram make more sense (although the 3 channels do seem to be swapped around compared to Siril). So this I think this solves the mystery of the strange histogram and reported Statistics that you noticed - a 32 bit float Fits format not correctly supported or interpreted by ZWO’s app.

During my analysis I have been going through the master flat production in some detail and I now have something that does not have the high rejection levels - although for the life of me I am not sure what has changed, other than me being very careful!

I will continue processing the set of three nights worth of data (with 3 set of flats) tomorrow, my hope that this is my error (albeit something I have repeatedly been doing for that last few weeks with this new data).

10

We may never know :wink: Good luck!