Hi
I’ve just taken some flat fields to correct some night images taken on my Canon EOS-R. I was using RawTherapee 5.8 and wasn’t getting the expected results, so I’ve upgraded to 5.9 and I get even stranger results.
If I apply the flat to itself or to another flat (I took 4, in case of a problem), then I get an excellent flat image in both versions.
However, when I attempt to apply it to an image I took earlier in the year, I don’t get anything like I expected. In v5.8, nothing happened at all - the image didn’t change, the histogram didn’t change. Nothing. In version 5.9, sometimes the same thing happens, but sometimes I get something that looks like the vignetting has been removed, but the image is turned to monochrome magenta. Not a magenta tint - looks like it’s been converted to greyscale and then white replaced with magenta.
Ooh, full magenta images sounds too familiar to a current bug in the GCC compiler for which we have had to disable a compiler optimization. But since we have this workaround in place, this should not happen…
Please note I’ve not tried flat frames in RT before (despite buying, but not yet using, a now $500+ Star Adventurer 2i Pro Pack a year ago… ) but it looks straightforward enough.
I just tried a few times in 5.9 and I get nothing. No magenta, no change:
Thanks for the suggestion with ART - I hadn’t heard of it before, but I will give that a try.
It may be of note that I also tried to apply the flat in an astrophotography software called deep sky stacker. During the reading of the image I noticed something flash past referring to 16 bit gray Canon Eos-R image. The result of that process was that the vignetting was removed, but I ended up with a greyscale image. Could this be a problem with reading in the canon cr3 format? When googling I noticed a number of reports of problems with this format.
I also forgot to say that I’m a Windows user, if that makes any difference.
If it’s a genuine bug, rather than me just being stupid, then I presume I should report it on the proper bug tracker?
It looks to have applied the flat, but it’s been a while since I even opened DSS and I never used it really. If you wanted one other test you could try it in SiriL. I actually have SiriL installed but I’ve not really tried to use it.
Like I mentioned before, I bought a Sky-Watcher Star Adventurer 2i during COVID but by the time it arrived months later my initial enthusiasm for learning wide field astrophotography had been tempered by Louisiana mosquitoes and I kinda moved on… So I’m having to dig into my mental Wayback Machine to remember anything at all. I may not have done it correctly.
There’s definitely something fishy going on with the application of the flat-frame in RT. I have not yet seen the magenta, but at least application of the frame should do something.
I’ve determined that the problem comes from the interpreted black levels. In the code below the values of black[c4] are higher than the values in cfablur, so that refcolor = 0 and everything correction is eventually multiplied and nullified by that.
My guess is that because you shot your sky at ISO 1600 and your flatfield at ISO 100 the camera actually applies different black levels and you cannot combine the flatfield for correction. Try shooting a flatfield at the same ISO and I am pretty sure things will work.
Edit: I know that RawPedia does not talk about this and actually advises to shoot ISO 100 for your flatfield. Perhaps we need to update the documentation, or there is still an actual bug here.
Great work finding the issue so fast!!! I was never that good on the open source projects I worked on.
I see the issue
My feel is that RT should deal with this scenario, especially if ART is dealing with it. Usually with flats, you would build a library for a particular lens, at particular focal lengths and perhaps apertures. But if we have to do it for different ISOs too, that’s multiplying our work by a factor of 10 or so.
Also, the whole point of a flat is that it should be, well, flat. If it is noisy, due to using a high ISO, then there will presumably be some degradation of quality. I presume this is why RawPedia states we should use ISO100. That said, given that the flat is blurred before applying it, the noise will probably be less that the original image, so the impact may be small.
I’ll try generating a flat tonight using the same ISO and see what happens.
That’s the same result I got for DSS. Flat applied, but image converted to greyscale. I guess that’s a bug in DSS, but I was flagging it here in particular in case the bug has a similar source to that in RT - maybe they use the same raw reader library for example.
The DSS image isn’t greyscale, it’s in color. I’m no astrophotographer but I stretched it in Affinity Photo and got this, edge artifacts (of my doing) and all:
Thanks, I posted on the DSS mailing list and they told me the same. So it is just RawTherapee struggling with this. Deep Sky Scanner and ART both apply the flat as expected.
oh and I’m not entirely sure which part of the sky it is to be honest
The shot was taken pointing vaguey north across Tornetrask lake in northern Sweden this march. I suspect you are right with M31 Andromeda Galaxy. A bit of checking seems to indicate that if you go up and slightly right of andromeda, 2 of those bright stars are part of the W of Cassiopea and the four bright stars on the right are zeta cephei, Alderamin, Herschels Garnet Star and nu cephei. The first two being corners of the box of Cepheus.
I think at this focal length (18mm) everything is just a little too zoomed in to pick out the usual constellations. I actually did a full sky panorama that night, because the aurora was not as good as some other nights. If I actually manage to get it stitched together I’ll post it.
And this is exactly why I wanted to sort the flat. The vignetting is bad enough that when I stretch it, all I do is get a brighter centre and darker corners, there’s no way I’d have been able to pick out Andromeda Galaxy without the flat.
) but it looks straightforward enough.
So I’m having to dig into my mental Wayback Machine to remember anything at all. I may not have done it correctly.