bulk import workflow advice

An update of my workflow, years later.

I have built up quite a backlog. I managed to start doing semi-regular imports again, but my practice of "rate all images from reject to :star2::star2::star2::star2::star2: (1-5 stars) has completely fallen aside in the last … 4 years.

So I’ve built up a backlog of about 17,000 images (out of about 55,000) that are not rated. I want to resume work on producing print material, and rating is a key part of that work. So I need to deal with this mountain!

Here’s what it looks like per year:

And per rating:

Kind of wondering what that “7” is in there.

I noticed some images were “half-rated”, that is: only the RAW or JPEG file were rated in a group. I was hoping I could easily knock down thousands of rating like this so I went deep in a Lua scripting rabbit hole and discussed that in batch copying ratings in group. It didn’t work that well: only 700 images were processed with the script.

Last night, I spent a hour or two rating images here and there, and processed about 500 more images.

At this rate (about 500 images a day), I’ll be done in (checks math) 34 days. Ugh.

For now I’m still using rapid-photo-downloader but i’m not labeling rolls anymore: I have adopted @paperdigits’s “flat” folder structure although I do have top-level “year” folders, so I have files named YYYY/YYYY-MM-DD/IMG_...JPG, e.g. ~/Photos/2025/2025-03-15/PXL_20250315_214759183.jpg. I add all images to git-annex manually after RPD does its thing, which allows me to confidently delete (which i bound to shift delete) images directly from Darktable when they are really bad or confidential stuff that should be moved elsewhere or destroyed.

To process the images, I import them in Darktable (either by starting it with darktable 2025 or by using the “import” button) and examine images in the light table view. My goal is at least keep up with the incoming flow, but also deal with 100-500 images in the backlog per day.

To deal with the backlog, I have a collection preset that has a “capture date” and “rating” field. It allows me to quickly see which year my unrated images are in and jump to a specific date. I can filter for all unrated images, then I iterate over each image in full screen mode (W) because i find the light table too small. I’m starting to experiment with the culling mode, but i find it a little confusing, especially since it doesn’t seem like the rating shortcuts work reliably.

Sometimes I remove the “no rating” filter when i suspect the unrated images are part of a shoot that has rated images, to avoid promoting images I didn’t mean to promote above already rated images (because yes, that’s a thing).

I’m making good use of the change_group_leader script though, really useful! I also have enabled the jpg_group_leader script to keep future imports bound to JPG more than RAW, because I rarely edit RAW anymore (lack of time, mostly, although I have high hopes DT 5.0’s new styles can help get me started with matching Fuji’s JPG output better).

One script I can’t quite figure out is the image_path_in_ui script: it looks like it only shows the folder where the image is, not the full image path itself, kind of less useful? I might fork that one to give me what I want. In the meantime, I can copy the full path in the image description, but that’s kind of buried, and works only one image at a time.

I’m also hoping I can improve this by rating images directly on camera and use the fujifilm_ratings script to import those ratings in DT, but I suspect that will not help much. I have rated a couple images like this in the X-T2 and I don’t remember it as being particularly easy, and the display is tiny compared to a 4k monitor, so sometimes you don’t even notice blurriness or other flaws, especially in the field, busy.

I also have other scripts enabled, but that’s the gist of the workflow right now.