Grouping and using data from different sessions

I’ve been using the Siril scripts for DSLR (or one shot colour as I think the new name is), for which I put files in lights, darks, flats, and biases folders. That works. I’m wondering how I can make use of data from different sessions without having to reprocess everything that’s been done before. e.g. if I could say, here’s my previously “processed” data, and here are lights, darks, flats, and biases from my new session. Please process them. The goal would be to add on more data from newer sessions so I have 10-20 hours of data for a target instead of 2. Is this possible in Siril, and if so, what would the steps be?

If the field of view and hardware remains the same, you can make a new sequence that contain all preprocessed images, either by renaming them to have non-conflicting numbers, or by using the conversion with all preprocessed images.

You may take a look at sirilic that automates all this, using several sessions, several filters and so on, generates a corresponding script and runs siril.

Do you mean I should take all the fits files for lights, dark, etc. from all the sessions and put them in the same respective folders?

I’ll have a look at sirilic, thanks.

No I mean take the images that are the result of the preprocessing part (offset dark and flat processing). In general this sequence is called pp_lights.

Thanks… fairly new to this, so wondering if I understand correctly:

  1. I’m seeing that for the darks, flats, and biases, in respective folders, there’s a a dark_stacked.fit, pp_flat_stacked.fit, bias_stacked.fit. Does that mean I can delete the other files, and just keep the stacked.fit files for darks, flats, and biases?
  2. Lights don’t have a stacked file. However, for each raw file, I see a light_number.fit file, a pp_light_number.fit file, and an r_pp_light_number.fit file. Which of these should I keep for the next session? Or should I just use the raw files again? And what’s the difference between pp_light and r_pp_light?
  3. Is there a place to learn these nuances? The siril manpage takes us through some processing steps, but as a beginner, I’m finding it difficult to fill in the blanks.

By default, a sequence has a name bias, dark, flat and light I think. And by default you are using FITS images for the sequence, so you have bias_001.fit, bias_002.fit and so one, and the same with dark flat and light.
By default, the preprocessed sequence is prefixed with pp_. By default the registered and rotated sequence is prefixes with r_. So the final result is the stack of the r_pp_light sequence.

What I’m saying is you can group all pp_light_* images from all your nights into a single sequence, then run registration and stacking (with add-scale normalization) on it to get the overall result.

We are working on the new documentation. We have a clear version that explains this kind of things in French that’s being translated, and the content of the current documentation will be remade too. It’s quite old and not complete enough.

@vinvin thanks… that really helps.

So in that case, one last question (famous last word!)… if I take the pp_light_* from all nights into a folder and run registration and stacking, I imagine I don’t need to involve darks, flats, and biases anymore as that information is already in the pp_light_* files. Is this correct?

Also (lol), if I wanted to build up a darks dictionary at various temperatures, can I take the dark_stacked.fit bias files from various sessions and make a master bias?

Extra docs would be awesome. I’ll try running google translate on the French docs in the meantime.

This is correct.
Yes dark_stacked is the master bias for these conditions (temperature and exposure).

Note that the variation of signal in darks with regards to temperature seems to be linear, so the dark optimisation function that automatically applies a scale factor to the master dark on a not temperature-regulated sequence is quite helpful and leads to good results with DSLR cameras for example.

So are the following correct?

  1. I can take darks at different temperatures (but the same exposure), and put them together to make a master dark file.
  2. I can take darks at close temperatures on different days a the same exposure, and take the master dark for each session, and process that to make a single master dark for that temperature and exposure.
  3. I can take darks at close temperatures at the same exposure from different sessions, put them in a folder, and process those raw files to get a single master dark file.

1 is correct, 2 and 3 are incorrect.
Making a master require consistent images in the sequence. If you take the darks at different temperatures, they will not have the same levels and stacking them will give an average or median which is not representative of anything.
What you can do is make a master dark with darks all taken at the same temperature, process light frames taken at different temperatures with it as long as dark optimisation is enabled (which scales the master dark to the level of the lights).
I suggest you make a master dark per night if temperature is not stable, preferably at the middle of the exposure, a master flat per night, preprocess the lights of each night with their master dark and flat and then combine the pp_lights together for final processing.

Thanks, @vinvin. That’s very helpful.