More Live Stacking Observation + Suggestions

Hi, with more simulation (ie manually dump files into folder for live stack) done I come to a number of observations. To start with I spend most of my time on EAA and only do AP occasionally.

Looking into the EAA folder, I observe that every RAW file dumped into the folder will generate 3 extra files. Hence the folder size grow exponentially. This might be the way for normal AP but for EAA that means a heavy burden. Light weight setup with mini PC etc could be strained.

Also please clarify if the “latest” stacked image is the one labelled “r_live_stack_000xxx.fit” where 000xxx is the biggest number in the sequence?

For EAA it is quite normal to display the stacked image via a webserver so many people can share the updated image real time. This can be a nice feature to be considered.

I understand that Siril started off as a astrophotography tool. LIve stacking is something new. For EAA we simply need a “light weight” stacking tool with good stacking capability and not so much post processing tools. If Siril can spin off a light version specific for Live stacking it will be great.

I believe the latest image is numbered 00001, overwritten with every new input.

We could indeed add a simple file removal at each step to not make the directory grow more than needed. An I agree this is a suboptimal approach, it’s just that we had the tools to do that easily in Siril…

Hi, I would like to raise some new points after some more trials. It is not necessarily a Siril problem but I think it might help to make the program more flexible. Still I am using a simulation by dumping image files into the work folder.

Currently I am doing EAA with very primitive setup. A Raspberry Pi camera connected to a Raspberry Pi. The Pi controls the camera to take images and also executes live stacking using ASTAP. This is a rather slow setup as the Pi is doing everything with its limited capability. With Siril, I try to install it in a very old netbook running Linux. The Pi will now only control the camera to take images and the files are dump to the netbook via WiFi using ssh file transfer. The file transfer take a couple of seconds for 1 file (~6M in size) and Siril on the netbook fails to stack it. The reason being Siril starts to convert the DNG image file to FIT before the file transfer is finished and it runs into file reading error.

Is there a time limit for the file to become a “completed” file so Siril can work on? For a camera to dump a file over a network to a computer a certain time delay and duration is expected. If this is too short then the image capturing and Siril live stacking has to be happening in the same computer. That can be a huge issue in the field when wireless connection among computer devices is rather common. Not sure if there is anything that can be done about this. Thanks

The way Siril considers the file to be completely written is looking at the file size and if it doesn’t change for 1.2 seconds, it should be done. If there are such pauses in your transfer, with the current implementation you have two ways of dealing with this:

  1. you should copy the file as a hidden file then rename it when it’s completely there
  2. if you don’t use Siril to display the images, you can use siril-cli to do the live stacking operations, and in that mode images are processed when a command informing they have arrived is received.

Otherwise if you tweak the timeouts in the code you could make it work… See WAIT_FILE_WRITTEN_US and WAIT_FILE_WRITTEN_ITERS in src/livestacking/livestacking.c.

Note: there are better functions to deal with this at a filesystem level, but they don’t work on windows so we don’t use them.

Thanks for the detailed info.

I am using AppImage so to tweak the timeout I need to compile from source. This can be a tough one for me. I will take this as last resort.

By the way, what stacking method is used for live stacking, a simple sumation or average with rejection?

it is a weighted mean without rejection.

I would like to report the progress etc. I add a watchdog python script in the netbook to monitor incoming image files. The image is transferred via Wifi from the Raspberry Pi to a Parking folder in the netbook. When the watchdog script concludes that the image file is completely transferred, it will move the file from the parking folder to the work folder.

Siril still does not stack. It ignores these files.

In parallel I also install ASTAP in this netbook. Before employing the watchdog ASTAP also did not stack, likely due to the time delay in file transfer over WiFi. With the watchdog ASTAP stacks successfully. It is the same setup now and ASTAP recognizes the files while Siril ignores.

I cannot find anything in the Console to say anything wrong. It just says waiting for files.

I try to manually open these DNG files with Siril. It works. Hence I believe the file as much is not an issue. I also believe the file transfered from1 folder to another in the same hard drive should take less than 1.2s (~6M in size). I have to stay with ASTAP for live stacking now. For your ref I enclose 1 sample DNG file here.
Img_20230303_155143_no_6_g45_60s.dng (5.9 MB)

Hello, I have tried your image with live stacking and I cannot see a problem with it. It’s a rather small image, it would be surprising that the transfer or copy time is too high. What settings are you using for live stacking?

I downloaded the most recent Beta version (1.2.0 Beta 2) on my Mac and tried the live stacking feature. Unfortunately, I can get beyond image 2 when I then get en error that stacking has failed. I tried several objects, all with the same result.

Hello and welcome!
I don’t think it was every tried on mac, maybe it just doesn’t work. Or maybe there’s something special with your images, what format are they in, and what options did you use?

Hi, I am using FITS and the default Live Stack options.

We need more details. For example logs to understand what is happening.

I only see something called live_stack_.seq files. Do you mean those?

no, starting live stacking will hide the right hand side of the main window, you should be able to put it back by clicking on the thin vertical button at the extreme right, and there’s a console tab in there that will show messages