After five years of off-and-on trying, I still can’t successfully rearrange the digiKam tags on the digiKam tag tree. Changes that I make to the tag tree aren’t saved to the image files, aren’t saved to the sidecar files, and disappear as soon as I close and reopen digiKam.
Does anyone have a step-by-step procedure that actually works to rearrange the tag tree and save the changes to the image file, or preferably to the sidecar file?
My apologies, I did also ask this question on the digiKam mailing list, so I know I’m breaking rules of protocol with respect to asking for help.
I’ve often had to rearrange my tag tree, but they’ve mostly been minor changes. I’ve done major reorganization only once, and it was a tedious chore. If I’d had the stomach to re-tag several thousand images I probably would have removed the tags from the files with exiftool and started over with a new database in digiKam. But that’s not the path I chose, so I’ll share my limited experience with you.
First some background: I only set tags on jpg images, not raw files. I have digiKam set to write the tags directly to the files, but only on confirmation; if you have it set to write changes to files immediately you might try changing that setting, at least until you are done reorganizing your tag tree. I also don’t use sidecar files, although I don’t think that probably matters much.
I make heavy use of the Tags pane on the left-hand side to select which tags I want to work with and tackle them one at a time. I did not try using the Tag Manager, although in retrospect I think it may have been easier to reorganize tags there.
The general procedure I use is to make a list of the tags I wanted to change and tackle them one at a time (or one sub-tree at a time). If you have hierarchical tags, I found it best to work from the lowest level first and then back up the tree to the parent tag(s). I start by selecting the tag I want to work on in the left-hand Tag pane so that digiKam will show me all images with those tags. If I’m just moving the tag, I will use the drag-and-drop method to move it. If I’m renaming the tag or want to move it into an entirely different branch I’ve found it easier to:
create a new tag
select all images with the old tag
add the new tag to the images using the Captions pane on the right-hand side
remove the old tag from the images
apply the changes (and wait for the filesystem to catch up, my desktop is a bit slow when I’m working on many images)
delete the tag from the tag tree
For simply moving stuff around, I move the tag, select the images, remove and add the tag in the right-hand pane to make the “Apply” button available, then write the changes to the files.
I don’t usually move whole branches around, but if I do it’s easier for me to create a new tag for at least the parent; that way I can move the child tags under it, remove the old parent tag from it, add the new parent tag, and apply the changes. That way I don’t inadvertently tag all the images with every child tag.
You might also consider using the database maintenance tools to clean up your database both before and after making changes to the tree. I had to do so (albeit manually, as that functionality was not yet part of digiKam at the time) the first time I made large changes to my tag tree.
I did finally manage to get my hierarchical tags back in order, though it took many hours of experimenting to figure out the procedure. If anyone else is faced with rearranging tags, @ashurbanipal describes the procedure much better than I could have done.
The only thing I will add to Steve’s “how to” is that when rearranging tags it’s a good idea to run digiKam from a terminal. What tipped me off as to “why” just moving an entire branch back to where it belongs was failing every time, was the following terminal output:
digikam.general: Delete Shortcut assigned to tag 56
digikam.dbengine: Failure executing query:
"UPDATE Tags SET pid=? WHERE id=?;"
Error messages: "Unable to fetch row" "UNIQUE constraint failed: TagsTree.id, TagsTree.pid" 19 1
Bound values: (QVariant(int, 81), QVariant(int, 56))
Whenever a message abut “UNIQUE contraint failed” appears, even though the digiKam user interface “looks” like the tag has been moved, it really hasn’t.
That’s a good idea, something I haven’t done yet. My usual way to “clean up the database” is to simply delete the existing database and then recreate, after using exiftool to spot-check to make sure that the proper tags are actually in the image files. I still don’t trust digiKam to write metadata, especially to raw files. But I do trust exiftool, which is why I have digiKam set up to only write to sidecar files. But maybe the “clean up the database” tools are working properly in the current version of digiKam?
I actually set digiKam to not write anything at all while I was rearranging the tags. And then I wrote them all at once when I was done rearranging. And I was very careful to make backups before writing anything at all, which was a good thing to do as it took several trials (and many hours) to figure out how to get digiKam to write things out properly.
It used to matter quite a lot when writing to sidecar files. I don’t know about now, stiill testing.
This issue with not being able to rearrange the tag tree without risking severe corruption of the hierarchy has been a problem with digiKam for a long time.
The recent version of digiKam has an option that seems to help. Under “Settings/Configure digiKam/Metadata/Advanced” you can configure digiKam to only write selected metadata items. I never did add useful ratings/labels/captions to my images (maybe some day). So the only metadata I was concerned about was the image tags. So I set digiKam to only read and write one item, which is “Tags/Xmp.digiKam.TagsList”.
I also took the precaution of using exiftool to remove all non-camera-written fields, so that literally the only user-added metadata in the image files was the digiKam TagsList, which was only written to the Xmp field. Before resorting to this rather drastic measure, after allowing digiKam to write to the image file (contrary to my usual practice but other efforts had failed), upon closing and reopening digiKam the tag tree was littered with a huge number of single-level tags that would have taken forever and a day to get rid of, except that fortunately I was working on a copy of the images.
Steve’s “how to” seems to indicate that even after getting the tags back in order “the first time”, any effort to do future rearrangement still isn’t a simple case of drag and drop. Sigh.
I’d be interested to hear how your testing works out. At some point I’d like to start using sidecar files for my raw files. I currently leave the raw files untouched (in a subfolder) in digiKam and tag the corresponding jpeg, but that makes the raw slightly less accessible.
It’s probably worth noting that I’m one release behind on digiKam; 5.6.0 is the latest packaged release for my distro and I haven’t tried the appimage of 5.7.0 yet. My tag tree is still a bit of a work-in-progress, so I was hoping the newer version might work better. I’d also missed the settings to control where digiKam writes tags, so thanks for pointing that out. When I get some free time I’ll probably set up a small collection using the latest version and do some testing; limiting where the tags are written will make the verification with exiftool much easier.
I think it’s not possible to move a tag branch from under another tag to instead be a top-level tag branch. Here’s what I’ve tried so far. Do you see anything I might have done incorrectly?
I moved a tag branch that previously had been under another tag, to be a top-level hierarchy. DigiKam clearly showed that the former branch was now a top-level hierarchy. So I tried four different ways to write the new TagsList to the sidecar xmp file:
After moving the branch I did “Tags Manager/Sync Export/Write Tags from Database to Image”. The time-stamp shown in dolphin was updated, so digiKam wrote “something”. But exiftool showed no change in the tagslist tag
Then I selected the affected tags and wrote them out by themselves: “Item/Write Metadata to Selected Items”. The time-stamp shown in dolphin was updated, so digiKam wrote “something”. But exiftool showed no change in the tagslist tag
Then I did “Tools/Maintenance” and asked to write the “Whole tags collection”, “Sync Metadata from database to image metadata” and to “Perform database cleaning”. The time-stamp shown in dolphin was updated, so digiKam wrote “something”. But exiftool showed no change in the tagslist tag.
Then I closed digiKam, and restarted it. The “moved” tag branch wasn’t really moved. It was right where it was before I tried to move it. So I tried moving the tag branch using “Tags” from the left-side panel, and selected the affected images and asked digiKam to save the metadata to these images. Again the time-stamp was updated, so digiKam wrote “something”. But exiftool showed no change in the tagslist tag. And after closing/reopening digiKam, indeed the tag branch had not been moved.
None of the four methods of moving the tag tree and writing to disk actually worked. Before closing digiKam, the tag tree in digikam shows that the tag branch was “moved”. But after closing and reopening digikam, the tag branch is back in its original location. All the moving and rewriting steps were a total waste of time.
I was monitoring the digiKam terminal output, and as far as I could tell no errors were ever printed to the terminal to show that something might not have worked.
At least when writing to an xmp sidecar file, I’m guessing the only way to successfully “move” a tag branch is to create a new top-level hierarchy and move just the bottom-level tags, that is, the individual “leafs” on the tag branch. This workaround wouldn’t be so bad for moving a small branch with only one sublevel of tags. For branches with multiple sublevels or with a whole lot of “leaves” at the end of a single branch, this is not a useable solution. FWIW, I haven’t tried this workaround because the branches I really want to move have too many subbranches and too many leaves to be moved “one by one”.
I did a quick bit of testing and it seems that moving any tag tree fails. I’ve evidently been lucky enough that I haven’t had to move whole sub-trees. (Only because my sub-trees were so poorly designed I abandoned them entirely, but I digress…)
Digikam writes to various existing namespaces (configurable) and to its own one: Xmp.digiKam.TagsList (I don’t know whether that is an “official” notation or just one used in digiKam). Your proposal should work fine. I would do the manual intervention without digikam running (it scans on changes), otherwise it might get confused, and then run the maintenance job to update db from metadata.
Can anyone confirm that mysql databases are not affected? That rearranging the tag tree is possible to do when using mysql? Or did I misunderstand the bug report?
I moved a second level tag with one subtag to another root level tag. The moved tags were newly created without any images assigned to it. I get an error on the command line about not being able to prepare a statement, but the move persisted after a restart.
digikam.dbengine: Prepare failed!
digikam.dbengine: Failure executing query:
"LOCK TABLE Tags WRITE;"
Error messages: "QMYSQL3: Unable to prepare statement" "This command is not supported in the prepared statement protocol yet" 1295 2
Bound values: ()
Actually after trying this out I had another look at this bug report: https://bugs.kde.org/show_bug.cgi?id=368734
There it is already reported that it works for mysql and doesn’t for sqlite, because of an SQL trigger that only works for a single tag but not for a tags tree. Probably mysql doesn’t use this trigger and thus does not suffer from the same problem. However the error above still makes it seem like not everything is just as it should be for mysql either.
Hi @rasimo - I’m a bit confused by what you said. Did the second level tag that you moved have any images assigned to it before you moved the tag?
After moving a tag branch to a new location, I would expect the image tags to follow the move. So all the images that originally had tags on the branch before the move, would still have the same tags after moving the branch. In other words, moving the branch should not just create new branch with no tags - that would mean all the original tagging was lost.
Hi @paperdigits - what is xslt? That is very, very kind of you to offer to help script an xslt solution. I can provide some xmp files, but it will take a few days to put together. Maybe in the meantime it will turn out that the answer is to switch from sqlite to mysql, though on the other hand that might be asking for a whole additional layer of problems . . .
Argh, the far too frequent issue that I make long confusing sentences struck again (I am not native English speaker, but that’s a poor excuse). What I did was the following: First I created a new second level tag and one additional child tag. I did not assign these tags to any images. Then I moved the new second level tag together with it’s child to another top level tag. This move produced the command line output I posted earlier. Then I restarted digikam and the two tags I moved remained at the new position.
So I repeated the same procedure but now I first assigned the tags to be moved to some random images. Again the move persisted. However I can also confirm that it does not change the tags in the image metadata - which clearly is a bug. This can easily be resolved by either writing the image metadata to file or using the maintenance tool - nevertheless it is definitely a bug in itself.
And I can say that I am using internal mysql for a long time now and didn’t have problems (which doesn’t have to mean anything, that I use just a subset of the functionality present).
@elle, xslt is a templating language that is XML aware, more or less. So I can read in XMP files, which are XML, manipulate the structure or content of the XMP tag.
XSLT generally runs using java and the Saxon library.
Yes, that’s the way I read it as well. Thanks to @rasimo for testing it! I haven’t yet tried mysql for the database as it’s still labeled “experimental” but it seems that the risks are small, so I might give it a try when I get a chance.
@rasimo - thanks! for filing the bug report and for testing with a tag tree that had images assigned. As soon as I finish organizing my image files (a task independent of digiKam), I’ll try making a mysql database.
Does it matter if one uses mariadb or mysql? Neither is currently installed on my computer. I’m not even sure if mariadb is still around, come to think of it.
I am by no means an expert, but from what I read mariadb is the more actively maintained/progressive/free variant of mysql. At least it is what I have on my system and I think it also is the default for debian.
