Siril very slow in TIFF to FITS conversion, what other software could do this?

I am doing astrophotography with a Canon R5.

Normally I use Raw Therapee, but it does not support R5 .CR3 files. So I used the Adobe DNG converter to convert CR3 --> DNG. Then I used Raw Therapee to make a linear TIFF file.

In order to use Siril for image stacking, I am currently trying to use Siril to convert the TIFF files to FITS files. It is very slow - it has taken about 138 seconds for each TIFF file. I have about 1000 of the files to process, and in the last 18 hours, it has done 471.

This is on a very fast machine reading to and from SSD drives over USB 3. So it should be much faster.

Raw Therapee does the full raw conversion - with debayering, lens profile correction and more - in much less time. Yet that is doing a substantial calculation. Going from TIFF to FITS is just pushing bits around.

There must be a software tool out there that would convert from TIFF to FITS faster than this. Do you have any suggestions?

Alternatively, I wish Raw Therapee supported FITS output.

Siril has its own way of processing DSLR RAW files - but Raw Therapee has a lot more features in this regard - particularly the lens profile corrections. I use Siril when processing astrophotos from dedicated astronomy cameras but Raw Therapee is better when going from a conventional camera.

Something sounds extremely strange, because my assumption is that converting TIFF to FITS should not be any slower than converting raw images from a camera (such as Sony ARW) to FITS, which I know from experience siril does MUCH faster than 2 minutes per image even on my old Acer laptop (i5-7200U , mechanical hard drive)

Here are a few examples from the output log - you can see it taking 2 minutes per file.

They are admittedly big files, but again, this is massively slower than Raw Therapee took to do a lot more actual computation.

10:28:21: Reading TIFF: 16-bit file 3T8A0486.tif, 3 layer(s), 8216x5482 pixels
10:30:38: Saving FITS: file mw40_00483.fit, 3 layer(s), 8216x5482 pixels
10:30:39: Reading TIFF: 16-bit file 3T8A0487.tif, 3 layer(s), 8216x5482 pixels
10:32:57: Saving FITS: file mw40_00484.fit, 3 layer(s), 8216x5482 pixels
10:32:58: Reading TIFF: 16-bit file 3T8A0488.tif, 3 layer(s), 8216x5482 pixels
10:35:16: Saving FITS: file mw40_00485.fit, 3 layer(s), 8216x5482 pixels
10:35:17: Reading TIFF: 16-bit file 3T8A0489.tif, 3 layer(s), 8216x5482 pixels
10:37:34: Saving FITS: file mw40_00486.fit, 3 layer(s), 8216x5482 pixels
10:37:35: Reading TIFF: 16-bit file 3T8A0490.tif, 3 layer(s), 8216x5482 pixels
10:39:54: Saving FITS: file mw40_00487.fit, 3 layer(s), 8216x5482 pixels
10:39:55: Reading TIFF: 16-bit file 3T8A0491.tif, 3 layer(s), 8216x5482 pixels

Hmm… Ultrahigh resolution combined with being demosaiced makes those files a lot larger (probably 300MB each) - my own stacking/averaging workflow had mosaiced images going into and coming out of siril. (e.g. all color conversion and demosaicing was done after stacking)

What version of siril do you use. That should not be so long.
Last version of siril reads CR3 files.

I am using Siril 0.9.12

Even if Siril does read CR3 files for the Canon R5 , I like the features that Raw Therapee has

OK. So use the 0.99.6 version

Which RT features do you refer to?

OK but doing this in one pass would be faster.

Main feature that RT has is Lens Profiles. The vignetting profile obviates the need for flat frames.

The lens distortion correction makes it easier to stack frames from a wide angle lens that are not perfectly aligned.

One pass would be faster in an idealized case, but in this case the Siril conversion from TIFF to FITS is slower than the sum of RT and Adobe DNG converter, by a huge factor (I would estimate about 20X slower).

Not using the last version.

I never use TIFF, maybe siril’s TIFF reader is very slow? That’s a bit strange. If you demosaic with siril, it’s faster in the recent versions, some other things are faster or parallelised too.

Conversion was not multi-threaded in 0.9.12.

0.99.6 is MUCH faster, thanks. It gave me a message saying this

3:21:14: Conversion: processing 36 files…
13:21:14: You should enable the Developer Mode in order to create symbolic links instead of simply copying files.

I don’t know how to enable Developer Mode, and I am not sure quite what this means.

Only needed for FITS to FITS conversion. Fixed in dev version.

While I am at it, here is why I want lens distortion correction.

I will sometimes shoot a bunch of sub-exposures with a stationary camera. If these are stacked normally, with registration on the stars, it means that over time the frame drifts. Instead of being stacking with approximately the same framing, it is stacking where the edge of one frame may be overlapped with the center of the first frame.

I did this, for example, for the Orionids earlier this week, which is where the images I am working come from.

The problem is that in a wide angle lens there is enough distortion between the center and edge of a the frame that they are not going to stack well. Indeed I found that you can’t even plate solve the result from even an excellent wide angle lens (Sigma 40mm f/1,4 ART lens on full frame). That is barely “wide angle” but it is enough that programs like ASTAP won’t plate solve.

In a full panorama stitching program like Hugin or PTGui, the program solves for lens distortion at the same time as doing registration. But those programs don’t do the outlier removal and normalization that Siril does.

APP does the full distortion calculation, but rather strangely it uses an N^2 complexity algorithm - where N is the number of files the stack or mosaic: it checks every pairwise overlap of the frames. That is of course unnecessary because you can plate solve to get the rough overlap.

So, by doing lens distortion correction in the RAW conversion stage, it becomes possible to use Siril to do stacking of partially overlapping frames.

OK, but converting to debayered TIFF files makes not possible the preprocessing step.

I don’t understand.

Normally, Siril does what most other astro software does - it uses Master dark, light, bias and other files to make a calibrated file.

Raw conversion software such as Raw Therapee does much of that work. The lens vignette profile, for example is essentially what the Master flat is for. So a lot of people who shoot astro with a DSLR (or now mirrorless like R5) find that it is not necessary to do the standard processing.

10:28:21: Reading TIFF: 16-bit file 3T8A0486.tif, 3 layer(s), 8216x5482 pixels
10:30:38: Saving FITS: file mw40_00483.fit, 3 layer(s), 8216x5482 pixels
10:30:39: Reading TIFF: 16-bit file 3T8A0487.tif, 3 layer(s), 8216x5482 pixels
10:32:57: Saving FITS: file mw40_00484.fit, 3 layer(s), 8216x5482 pixels

Those timings suggest that reading each TIFF is taking the time (over two minutes) and writing the FITS is fast (1 second).

If it helps, ImageMagick can convert from TIFF to FITS. For example:

magick x.tiff x.fits

For a 8216x5482 pixel TIFF, this takes 7 seconds on my laptop.

1 Like

Of course siril can. But we never subtract darks and/or biases from interpolated data. This is non sens.