I have a problem. I have been failing at keeping up with rating my images for years now. I want to fix that problem and slowly start to rate all my images. According to Darktable, i have about 17k images that are unrated, among 55k images. But that’s kind of a lie: I found that I have a lot of images that are rated, but only one of the images of the group. Ugh.
So what I would need is a way to fix this for the huge backlog. I estimate I must have thousands of such images. Here’s what Darktable thinks of my ratings:
What I am thinking could work is to process each unrated image, and look at the other image in the group. If that image is rated, copy the rating over.
I suspect this will require writing a Lua script, which I have never done (although I have written Lua before, so I’m not a completely lost cause). I’m not sure where to start: it seems to me most lua scripts are designed to respond to keybindings on specific images, but I would need to essentially process my entire library, which doesn’t seem practical, even from a user interface perspective (can i even select fifty-five thousand images at once?!).
Is there something that does this magically already? Could I do this through the sidecar files, perhaps? I might be a little more comfortable parsing XML (i can’t believe I just said that) than dealing with the Lua API…
So I guess part of the question is: how to keep the problem from getting worse? As things stand now, I still see rejected RAW images don’t have their associated JPG image also rejected, so I’m still setting ratings only on the group leader instead of the entire group. I see there’s a rate_group script, but i don’t understand why it’s necessary if others say that the whole group is rated anyways.
If you don’t notice the difference, it’s because there really isn’t that much here: we went from 17875 to 17172 unrated images, only about 700 images, or 4% of the work, were processed.
That kind of mass data mangling can be done in the library.db using sql statements.
You can use a SQLite browser to investigate the data structure to identify groups and then query for minimum rating > 0 for each group and update those with rating is unset or 0
You forgot to tell us whether the script did not handle all groups, or there were only about 700 unrated images in groups?
If the latter, I don’t really see how any kind of script could help (for me, rating is an inherently manual task, as (again for me) there’s no relation between ratings for different images, not even within a group).
Oh, I thought i did: my script found about 700 images in group that were unrated but had another image in the group that was rated. So I reduced my number of unrated images by 700.
Oh yeah, my work here won’t help with the fundamental question of “is this good”, that’s a subjective process, naturally.
I have considered this. I found the rate_group script and found it was easier to start from that than reverse engineer the SQL database. I was also not sure the changes from SQL would apply back to the sidecar.
By using the Lua API, I’m pretty sure Darktable did the right thing, including in the SQL database (because I saw the changes in the UI) and the sidecar files (because I saw the changes on disk).
You probably could by putting a “%” in the filmroll name which should display all the images and they you could use select all. However, using scripting there’s a better way…
local dt = require "darktable"
for _, image in ipairs(dt.database) do
-- process each image
end
Since scripts can be started and stopped from script_manager, you could just have the script run until completion when you start it. If you leave it “started”, then the next time you start darktable the script will run again and catch any groups that aren’t rated correctly.
How grouping works in darktable, from a Lua API perspective…
Each image has an id and a group_leader field. The group_leader field is set to the id of the image that is the group leader. When you import an image the id is set and the group_leader is set to the id, so each image is a group of 1. So in code a TRUE group leader is
image.id == image.group_leader and #image:get_group_members() > 1
If I was doing this I would keep track of the groups I had processed
processed_groups = {}
-- set the group_id = true when you've processed a group
processed_groups[image.group_leader] = true
then go through the images looking for group members (image.id ~= image.group_leader), check to see if it’s been processed (if not processed_groups[image.group_leader] then) get the group members, find the highest rated, then rate all the group members. Then set the processed_groups to true for that group_leader value so that you don’t waste time reprocessing the group.
Interesting algorithm! As I mentioned above, I have already implemented something. I don’t check for group leader, and i don’t modify anything in the images other than rating, if possible.
I do not check for the highest rating, i check for the first. Maybe the highest would be better, but in practice I always have only two images in a group anyways, so that doesn’t matter much to me.
