Can someone please answer my DAM questions? ;)

Software
1

Glad to see this site is back up! I was worried there for a bit!

It seems to me that digital asset management (DAM) takes on a variety of meanings in the realm of photography–depending on who you are talking to–including issues relating to archiving, organizing, data backup, metadata, tags, copyright registration/issues, publication, etc.

At this point, I am a bit confused about all that this entails, and where I need to look in order to get up to speed. I have been an avid photographer for 40+ years, and started dealing with digital images in the late 90s–at that time, mostly for research (I am a microscopist and biologist). Early on, I decided to organize everything by date, using ISO 8601 (YYYY-MM-DD) format, with a short descriptor thrown in at the end. This has carried over to my other work (i.e., with dSLRs, mirrorless cameras). Of course, after so many years, I have hundreds of folders, all backed up on numerous drives in different physical locations (have not settled on Cloud storage provider yet–but it is on my list!). I have a very good memory (for now at least!), and this system has rarely failed me.

I used both Bridge and Lightroom for a while, and played around with ranks (stars), keywords, and the like. Honestly, I never found these all that useful, but maybe this is attributable to my lack of patience with anything other than shooting, post-processing, and printing/publishing. I also found Bridge’s (and LR’s) system of importing files to be a big annoyance. Since then, I have left Adobe and have been using RawTherapee for most of my postprocessing. I am STILL looking for a good pixel editor (I have GIMP but know almost nothing about how to use it).

In any case, my memory is not infallible, and I somehow managed to lose a week’s worth of gorgeous photos from a vacation we took in 2018! I still have some wallpaper jpgs and images selected for a calendar, but it still is very disappointing to lose all the originals. Also, manual backups (relying on my memory) on physical media is likely not the best system–good memory or not.

I am also way behind on copyright registration, having last done it about a decade ago. It’s a long story, but the last five years have been hard (COVID, deaths in the family, endless estate hassles and time-sucks).

Anyway, I am hoping ot get my website online by the end of the year and want to have ducks in a row concerning copyright registration, metadata for online images. I am also concerned about backups and settling on some type of system that will augment my old folder-based/ISO 8601 system.

I now also have 5000+ images from a recent month-long trip to Spain and want to use this set as a springboard for getting serious about these issues.

What are your thoughts on these topics, and can you point me in the right direction? Do you use Bridge or a similar app? Do you think RawTherapee is sufficient to handle much/all of this? What would be a good complement to RawTherapee? How should I get started educating myself on these issues?

Thanks!

2

Hi, take a look at digikam, I have been using it for many years now and I find it very good for my photo organising needs.

3

I’ve used Bridge and LR which are great as DAM. I use Digikam now which is also great, once you’re used to it which doesn’t take long it’s just simple to use and does everything you could need. It’s a good choice if you’re not settled on an editor because you can open editors with a right click within Digikam.

4

Main tool for my own DAM (when it comes to photos) is DigiKam. It serves me well in managing >100k images, stretching over a period of 40+ years. The DigiKam database is on a SSD in my computer and the photos are stored on a NAS (with regular back-up on separate external HDDs)
The process I’m using is quite simple:

  • Loading all new photos into a sub-folder created by DigiKam, automatically named yyyy-mm-dd (all folders of one year are in a folder named yyyy)
  • Adding to the folder name one or two words describing location, event or similar
  • Assessing the images in DigiKam using green or red flags and rating keepers using stars.
  • Deleting images with red flags
  • Adding at least one keyword to all keepers (over the years I have built up hierarchical indices for people, locations, types of events, plants, animals, architecture etc.
  • All meta data are also stored in XMP sidecar files managed by DigiKam.
  • Development of RAW images is performed in Darktable and final retouching and printing is via GIMP

Using this approach I have never missed to find an image in DigiKam within a few seconds.
Only weak point of this approach is me. I should be more strict when culling images.

5

Great, thanks for the recommendations guys. I will certainly check out DigiKam.

Can I ask how you incorporate copyright registration in your DAM workflow? As I mentioned, I have not done this in ages. I registered several thousand images in 2012, but need to do everything since then. This is another whole issue, but at this point, I’m wondering whether it is even worth the hassle. I understand all the issues surrounding copyright and registration–but am still wondering whether it is worth bothering. Another thread maybe if people are not sick to death of this issue…

6

In most (all?) countries copyright is automatic, although registering a work may give further protections and simplify legal disputes.

1 Like
7

Some thoughts on your post and questions. Firstly it seems your DAM needs are quite limited but your storage and backup needs more important?

Secondly Rawtherapee has some serious limitations when it comes to metadata, ones that really balloon the amount of work required. This will be largely addressed when the exiv2 metadata branch gets merged. Currently RT won’t pass on metadata stored in xmp sidecar files to exported files. Which means all tagging etc has to be done again for every export… extremely inconvenient! RT is still my software of choice though.

That’s imho an extremely good starting point for managing photographs.

The above makes me wonder what you mean by “backed up on numerous drives” and how your memory plays a part of your backup strategy?

Some questions

  1. Is your actual collection of photographs scattered across numerous drives?
  2. Are your backups simple copies of your collections.
  3. Do each of your backup (drive) contain only parts of your collection (at time of backup)? or does the last backup contain all photographs you’ve ever taken?

My collection still fits on one large drive. When it becomes to large I’ll have to adapt by splitting into two drives (at some round date) or adopt logical volumes. Keeping it contained and structured really helps me with overview, management and backup of my files.

I backup using https://restic.net/ . I choose restic because it allows multiple computers to backup into one deduplicated repository. I often have the same files (raw files and such) on different computers for mobile work, presentations etc. so I take advantage of shrinking the storage requirements for the backup by deduplicating across multiple devices.

I think it’s important for backups to be automatic and that they contain the whole photo library. My priorities and mind would ensure failure if backups were to be manual or scattered in parts on different drives. If you can’t spend the money required on large drives and need to ad hoc what you have and come across https://git-annex.branchable.com/ is a good tool for managing multi drive collections. For many it’s not easy to use though.

Spending the time setting up automatic backups is well worth it imho. There are quite a lot of software options unfortunately so some research will be required.

What do you mean by copyright registration? If it’s just tagging with your name and some license that should be very straightforward in a software like Digikam. You should be able to bulk apply the metadata with only a few clicks assuming you are the photographer of all images.

My workflow is

  1. Import/rename images with Rapid Photo Downloader to **/home/user/img/YYYY-MM/MMDD-jobcode/YYYYMMDDTT-XX.[RAW|JPG]
  2. Cull obvious bad files using https://www.geeqie.org/
  3. Open folder with http://rawtherapee.com/ star select files and process them. I only star files to signify which to work on and then filter by star to only view the files I have to work on. This also means F3/F4 will move to the previous/next image to work on and skip the ones I haven’t starred.
  4. Export to subfolder of /home/user/img/YYYY-MM/MMDD-jobcode/
  5. Tag exported files using https://www.geeqie.org/
    5.1 Second cull where I select a subset of exported images for publishing (web or otherwise)
  6. Occasionally use https://www.digikam.org/ for deep diving into the collection and identifying/tagging faces.

I’m hopeful that the RT exiv2 branch will allow 5 to move before 3 in the above list. Resulting in a much more robust workflow.

In the above it’s only really step 6 but also perhaps 5 that I consider DAM work. Unfortunately step 5 is a huge job. Tagging is nasty business and work I truly dislike. It does pay off for step 6 which for me is when I need to dig up disparate files for some reason such as a presentation involving photographs from many different locations or times or say a “historical” slideshow/book for important birthdays weddings etc. Honestly though the “jobcode” system (appending a short keyword to image folders) means a simple linux locate search often helps me find the correct photo near instantly.

2 Likes
8

My workflow is pretty similar to yours, except I don’t do tagging. Instead, I try to make the jobcode I set in RPD as keyword-rich as possible. RPD’s ability to save jobcodes helps to eliminate variations in names for the same place, etc.

Thanks for the link to plocate. I haven’t tried it yet, but the performance looks impressive. I’ve been using find, and it can be pretty sluggish.

9

Thanks for mentioning this. I need something like this and restic looks interesting. :+1:

10

The tagging is most important (critical) for images that are published. It slots into other organisations image handling and simplifies credits etc. My website also populates it’s tags and information from metadata. It really makes it easier to add the info to the file so that it follows the image rather than having to type it in for every publication.

Searching is only a secondary concern but tagging has saved my butt a few times when a subject is hiding in a different folder. I tend to split into jobcodes but sometimes the ‘classification’ is dual or more so a folder structure can’t do the job.

It relies on an index so the computer has to chug through your files. Compared to most desktop style indexers it’s pretty light though and doesn’t eat to much power/time/noise. The search is also not as refined as find but works very well for my needs.

1 Like
11

Restic plus some S3 storage makes for some good off site backup. There are a bevy of S3 providers such as Backblaze, wasabi, cloudflair, amazon, and others.

2 Likes
12

I have two DAM questions:

  1. Why does darktable not see standard tags? I have tried adding tags in both Digikam and fotoxx. When I examine the tagged image in darktable, the tags are not shown.

  2. Why does darktable not search for tags in all film rolls? If I search for an image by its tag, and I am currently in the film roll that contains the image, it will find it. But if I’m in another film roll, it finds nothing, even if the file is is a subdirectory of the current film roll.

I am not understanding how darktable tags are very useful (unless everything was in one film roll, of course).

PS - Yes, I have looked at the darktable documentation. It does not say very much at all about using tags.

13

Thanks again for the detailed replies. In response to some questions, HD space is not an issue for me–my entire collection occupies about 2 TB (and drives are cheap). Each of my backups contains my entire collection; the collection is not scattered across numerous drives. Auto backups are something I will need to look into.

Embarrassed to admit I’m not too familiar with XMP sidecar files, eviv2 metadata, and what the issue with RT is… I need to find a “for dummies” level book and/or article and/or video(s) on these topics.

14

What do you mean by “standard tags”?

You’re adding them to the xmp file? Do you have darktable set to look for updated xmp files on startup? There is also some configuration necessary in digikam.

What are you using to search? FWIW I usually browse tags in the collections module. Select the tag there and you can see what is tagged with that tag, regardless of film roll.

15

Summary:

Rawtherapee is not a DAM focused software. To manage tags, find images etc you’re better off using complementary software. Software used for reading and writing tags and other metadata avoid writing directly to raw files to prevent accidentally corrupting them. Instead they read and write to the .xmp file that has the same name as the raw file. A xmp file is valid for all image files with the same basename that resides in the same folder.

img001.raw  <---
img001.jpg  <---
img001.xmp      ^
                tag: London, John Smith
                creator: Alice Jones

So when the xmp files contains tag and creator that metadata applies to img001.raw and img001.jpg. If the two files have different content you are expected to keep file with the same name in different folders. There is a complication in that darktable and since then other free software has introduced a img001.raw.xmp option that breaks the standard but this is a separate issue and often a configuration option.

If you send out img001.raw without img001.xmp none of your added metadata will follow along. Therefore the common behaviour is that when every you export a new jpeg from the raw file it will bake all the metadata from the xmp file into said jpeg. Say you make a monochrome edit for some magazine and export the file

img001_bw.jpg  <---
                tag: London, John Smith
                creator: Alice Jones

Then a crop version

img001_crop01.jpg  <---
                tag: London, John Smith
                creator: Alice Jones

Then you need a low contrast one because it will have text on it.

img001_pale.jpg  <---
                tag: London, John Smith
                creator: Alice Jones

The tags should reside inside the new jpeg file so that you have a single file including metadata.

Instead Rawtherapee exports

img001_bw.jpg

img001_crop01.jpg

img001_pale.jpg

Witouth the tags. Meaning you have to add them back to this file after export and before sending it out. Note that Rawtherapee does copy all the metadata from the raw file into the exports. So date of capture, lens, aperture etc. will be baked into the jpeg (unless configured not to do so)

Rawtherapee breaks this by not reading the .xmp sidecar file and by not copying that metadata to the exported file. The exiv2 branch of Rawtherapee fixes this behavior (amongst other things)

1 Like
16

Ok, that solves that question. I was editing the tags of the actual image files. If darktable uses tags only in its xmp files, that explains why it doesn’t “see” the tags in the image files.

As to standard tags, “Metadata is typically stored in an image file as an EXIF file, but some image file formats may also include additional metadata stored in separate files, such as XMP files. These files can contain additional metadata that is not stored in the EXIF file, such as keywords or descriptions of the image.” from Image Meta Tags Explained » A Beginner to Expert Metadata Guide!

I did not understand all of this before I asked my questions.

Thanks

17

In the Selected Images module, select the metadata tab, then select “refresh” or something like that. It’ll reread the file metadata.

1 Like
18

Thanks. I’ll try that.

PS - Ok, that worked, but it took me about 15 minutes to find it. It’s on the right side panel; I kept looking on the left, where I do the rest of that stuff.

Anyway, it is Selected Images - Metadata - Refresh EXIF.

A downside is that I can’t just refresh all of the EXIF data, i.e., for a Film Roll, instead I have to select each image file to do it. I can, however, select multiple images at a time.

19

I’m not familiar with fotoxx but usually when you add tags to a raw image file the software actually writes to the xmp file. The gui will never show the xmp file. This is usually a setting though and at least Digikam can be configured to embed metadata in the raw file.

If you’re adding tags to jpg files however they should only show up if you’ve configured the software to use xmp for jpeg’s as well. The most common defaults are .xmp for raw and embedded for jpeg.

Since darktable use it’s own filename.raw.xmp convention instead of filename.xmp you may have issues with ensuring metadata shows up both ways in fotoxx unless it’s been programmed to be dt compatible. dt will re-read filename.xmp files but depending on configuration you may have to tell it to do so. Metadata added to raw files in dt will never show up in most mainstream dam solution.

There’s a setting to check xmp files automatically
see look for updated XMP files on startup below

2023-08-04-205935_679x457_scrot

1 Like
20

Well, as I said, my outboard software embeds tags into the image files. So, this is sort of an interoperability problem for me, but I will get used to it. Now that I know to select the images in the Film Roll and Refresh EXIF, I’ can do what I need.

Thank you.

PS - A year or so ago, I tried Digikam for this same purpose, but it brought far too much infrastructure onto my system for this simple purpose. I removed Digikam. I’m pretty sure I could also use exiftool for the same purpose.