I am with you here, I had the same issues until a comment from @lock042 opened my eyes in another thread.
I do not limit it (at least not being aware of doing this). What I do, step by step, is:
- open the stacked fit image in siril
- open Stellarium and activate the ‘image sensor frame’ option. I have it already configured in the settings for the sensor and lenses I have
- move the sky around in Stellarium until I get a framing similar with what I have in siril
- click on a star that is close to the middle of this frame. Stellarium will tell you some info about it and its name in various catalogues. I usually pick the one that is like HD
- open the ‘photometric color calibration’ in siril
- click on get meta data from image or insert the numbers there
- put the name of the star in the find field and search for it
- click OK to let it do its magic.
It can happen that sometimes it fails and I will just pick another star in Stellarium and search again. I don’t normally need more than 3 tries, but maybe I am just lucky