More Live Stacking Observation + Suggestions

Hi, with more simulation (ie manually dump files into folder for live stack) done I come to a number of observations. To start with I spend most of my time on EAA and only do AP occasionally.

Looking into the EAA folder, I observe that every RAW file dumped into the folder will generate 3 extra files. Hence the folder size grow exponentially. This might be the way for normal AP but for EAA that means a heavy burden. Light weight setup with mini PC etc could be strained.

Also please clarify if the “latest” stacked image is the one labelled “” where 000xxx is the biggest number in the sequence?

For EAA it is quite normal to display the stacked image via a webserver so many people can share the updated image real time. This can be a nice feature to be considered.

I understand that Siril started off as a astrophotography tool. LIve stacking is something new. For EAA we simply need a “light weight” stacking tool with good stacking capability and not so much post processing tools. If Siril can spin off a light version specific for Live stacking it will be great.

I believe the latest image is numbered 00001, overwritten with every new input.

We could indeed add a simple file removal at each step to not make the directory grow more than needed. An I agree this is a suboptimal approach, it’s just that we had the tools to do that easily in Siril…

Hi, I would like to raise some new points after some more trials. It is not necessarily a Siril problem but I think it might help to make the program more flexible. Still I am using a simulation by dumping image files into the work folder.

Currently I am doing EAA with very primitive setup. A Raspberry Pi camera connected to a Raspberry Pi. The Pi controls the camera to take images and also executes live stacking using ASTAP. This is a rather slow setup as the Pi is doing everything with its limited capability. With Siril, I try to install it in a very old netbook running Linux. The Pi will now only control the camera to take images and the files are dump to the netbook via WiFi using ssh file transfer. The file transfer take a couple of seconds for 1 file (~6M in size) and Siril on the netbook fails to stack it. The reason being Siril starts to convert the DNG image file to FIT before the file transfer is finished and it runs into file reading error.

Is there a time limit for the file to become a “completed” file so Siril can work on? For a camera to dump a file over a network to a computer a certain time delay and duration is expected. If this is too short then the image capturing and Siril live stacking has to be happening in the same computer. That can be a huge issue in the field when wireless connection among computer devices is rather common. Not sure if there is anything that can be done about this. Thanks

The way Siril considers the file to be completely written is looking at the file size and if it doesn’t change for 1.2 seconds, it should be done. If there are such pauses in your transfer, with the current implementation you have two ways of dealing with this:

  1. you should copy the file as a hidden file then rename it when it’s completely there
  2. if you don’t use Siril to display the images, you can use siril-cli to do the live stacking operations, and in that mode images are processed when a command informing they have arrived is received.

Otherwise if you tweak the timeouts in the code you could make it work… See WAIT_FILE_WRITTEN_US and WAIT_FILE_WRITTEN_ITERS in src/livestacking/livestacking.c.

Note: there are better functions to deal with this at a filesystem level, but they don’t work on windows so we don’t use them.

Thanks for the detailed info.

I am using AppImage so to tweak the timeout I need to compile from source. This can be a tough one for me. I will take this as last resort.

By the way, what stacking method is used for live stacking, a simple sumation or average with rejection?

it is a weighted mean without rejection.

I would like to report the progress etc. I add a watchdog python script in the netbook to monitor incoming image files. The image is transferred via Wifi from the Raspberry Pi to a Parking folder in the netbook. When the watchdog script concludes that the image file is completely transferred, it will move the file from the parking folder to the work folder.

Siril still does not stack. It ignores these files.

In parallel I also install ASTAP in this netbook. Before employing the watchdog ASTAP also did not stack, likely due to the time delay in file transfer over WiFi. With the watchdog ASTAP stacks successfully. It is the same setup now and ASTAP recognizes the files while Siril ignores.

I cannot find anything in the Console to say anything wrong. It just says waiting for files.

I try to manually open these DNG files with Siril. It works. Hence I believe the file as much is not an issue. I also believe the file transfered from1 folder to another in the same hard drive should take less than 1.2s (~6M in size). I have to stay with ASTAP for live stacking now. For your ref I enclose 1 sample DNG file here.
Img_20230303_155143_no_6_g45_60s.dng (5.9 MB)

1 Like

Hello, I have tried your image with live stacking and I cannot see a problem with it. It’s a rather small image, it would be surprising that the transfer or copy time is too high. What settings are you using for live stacking?

I downloaded the most recent Beta version (1.2.0 Beta 2) on my Mac and tried the live stacking feature. Unfortunately, I can get beyond image 2 when I then get en error that stacking has failed. I tried several objects, all with the same result.

Hello and welcome!
I don’t think it was every tried on mac, maybe it just doesn’t work. Or maybe there’s something special with your images, what format are they in, and what options did you use?

Hi, I am using FITS and the default Live Stack options.

We need more details. For example logs to understand what is happening.

I only see something called live_stack_.seq files. Do you mean those?

no, starting live stacking will hide the right hand side of the main window, you should be able to put it back by clicking on the thin vertical button at the extreme right, and there’s a console tab in there that will show messages

Out of curiosity I tried lifestacking to see what it does. I made a command script that copies a bunch of raw files to the Livestack folder I made. Before every copy there is a wait , I use 30 seconds delay now. If the files come in to quick cpu and disk overload and stucks all. So now it works like a charm. Every new RAW file in the folder gets converted to fit file and processed further. Takes 30 to 40 secs per incoming file. Only thing is that the only image that stays on the screen is the first .FIT file. I expected it to change and enhance the view , the more images are stacked. Should I configure anything for that?

It is supposed to show the last stacked image, updating every time one is created. If you don’t see change, make sure your script is not copying always the same input image. Does the noise level decreases in the live stacking window? I haven’t tried it with beta2, but it worked not long ago… Maybe it’s broken now.

It seems to display only the first file that is converted to fits format. I add some screenshots.
From the begiinnig to the end it displays the same image, which name shows on the screen.
I use the latest beta2. And there seems no decrease in noise level.
And which image is it supposed to show? is that the file “”
I will upload that one as well. That image looks green, nothing like the gui image displays during run.
So another question is if there might be a possibility to put some commands to remove green noise per image run and background removal?

If we take a look to your screenshots, noise is decreasing as expected between 17 and 19 images stacked.

My mistake, pictures where not from the same run sorry. Looks great now, I can see the image is changing on the screen, after feeding image 30 a plane comes flying in :-). And the noise started 11.951 and at feed image 90 it was down to 1.996. Keep up the good work. Thanks.

Not work on MacOS, files .fits from ASIAIR PLUS or from NINA are not comptaible.