I am not quite sure how the current algorithm is working, but one idea could be using e.g. a command like
find <image_folder_path> -name “*.xmp” -newer
for each of the folders from where images were imported into darktable. So you will get a list of the files which need to be re-imported into the darktable database.
Another idea could be to asynchronously run the check for new / updated images in the background and “force checking” the corresponding folder if the user starts to edit an image.
Ok… well, then I think the better option would be to be able to place your library at the top level of your photo folders and have it only contain relative paths and be able to point any Darktable installation, be it on Windows, Linux or Mac to that library file.
But then I guess lots of people will suddenly chime in and say they have their photo collections spanning multiple drives… 
My library does not span multiple drives but is not in “my photos folder” (whatever that means) nor will it ever be in that location.
I have to think you misunderstood me.
What I meant was if the library database file could be placed at the top level of whatever folder structure your photos are placed in, then it would be enough to use relative paths to reference all your photos, and thus it could be kept OS agnostic regardless of how you mounted that drive and then be accessed through any number of Darktable installs.
That’d make things much slower for me, my database is on an nvme while the raw files are on a spinning rust array.
I don’t know, how does that work under windows if the relative paths span multiple drives? (which is no problem under linux, where drives can be mounted wherever you want in the folder structure…)