My XMPs are gone 😨... any Idea?

Software
,
1

Not sure what to even write here. I’m just curious, if anyone has an idea what might have happened. For myself, I have no idea :man_shrugging:

As background information: I’m running my (windows) system for weeks, leaving evrything open and the pc is 24/7 powered on. On one desktop I keep digikam and darktable open. Darktable runs with --library :memory: and is used only for editing single pictures.

Yesterday I wanted to edit a play raw and when collecting the image and the xmp file (to post here in the forum), I realized, there is no xmp file(*). I searched for a few minutes changing darktable to lightroom mode checking my edit is stil there, yet no xmp file. Then I called it a day.

This morning I was looking into the issue and realized, after doing a recursive file listing via powershell, that there is in fact not a single xmp file on this drive left. This are about 10000 files from this year with hundreds of edits. Let’s assume I might have a somewhat recent backup(**), I’m still curious what has swepped away all the xmp files.

I have no idea what might have happened.

I’m pretty sure that I did not somehow delete the stuff recursively in more than 50 folders, and only the xmp files. I can’t think that darktable is the culprit here, because it has only two or three folders at max imported (because I’m using only for temporary edits.) Maybe some shit in digikam I did not know about? But I don’t remember doing anything special there - just F5 from while to while, to see my automatic from the camera written photos.

Or it my be some kind of corruption on this disc, which is aexternal ssd drive 2TB. It’s not even a year old and sitting on my shelve beside the pc.

As I said, not sure what to expect here - but maybe someone has seen something similar and has an idea. Thanks for reading so far!

*) I have not tested this extensively, but darktable still writes no xmp file. But I have to investigate further

**) Maybe not, but thats completely my fault

2

Depending on your settings, the editing metadata may also have been saved with the output images (JPEG, TIFF, …) and can be imported from there. Also, the darktable database may still be intact and contain the editing information. Otherwise, I have no idea. Are you sure that windows does not hide the xmp files in any way?

1 Like
3

Thanks for caring. I can probably restore a good part of my edits and even if I loose some, thats no big problem - I’m generally not much attached to, err, stuff.

Pretty sure it’s not a windows hiding things from me problem, as a search via powershell on my internal hard disk reveals all xmps and only on the external the ones are lost. I had only the pictures of 2023 there.

Also wanted to add, that up until - hmm let’s look into the play raw section - at least Nov 13 everything was ok.

Yeah, I realize that there is not much information to base an assumption on… I just hope, anyone has a clever idea what might have happened :man_shrugging:

4

Further investigation revealed, that the xmp files where still in the trash can. Deleted yesterday at about 8:19. I think I did a system reboot shortly before, because of windows updates.

In the eventlog I can see at the same time an error from the application ‘mylio’ which I had installed as a test version. I think it might have been trying to do some stuff - in my tests it did crash, but it certainly was in autorun and it also had saved the folders i have been scanning.

grafik
grafik1587×989 112 KB

I will probably not investigate further, as I don’t believe, that darktable or digikam have anything to do with it. And ‘mylio’ is already deinstalled.

1 Like
5

Just wanted to add, that I don’t blame mylio for this incident, as I probably did’nt know enough to use this application correctly. I just started it and clicked my way through…

6

Backup, backup, backup…

I’ve been using Kopia for a while and backing up with snapshots to backblaze. Around $1usd/month for 250gb.

2 Likes
7

I hear you :smiley:

Kopia does look nice, I will read into it. Thanks for mentioning it :+1:

Any thoughts about adding git into the mix (for the XMP files)? Actually … forget that, it’s probably overkill. Just came to my mind, coming from a programming background.

8

Kopia + Backblaze is really intriguing.

I did evaluate it and set up a test scenario, which worked like a charm.

I have two follow up questions, if you don’t mind.

Considering, that I have about 6TB of data to backup, that would be at least 400$ per year (the transaction costs seem minor, I didn’t include that, but I can’t really tell).
I mean, thats a significant cost for me, that I’m probably not willing to invest.
Do I miss something? You are apparently willing to pay this, or have significant less data?
That said, aside from the cost factor, its very intriguing!

The other point is regarding Kopia. My problem is with my photo workflow. I’m currently importing pictures right into the final folder (in my photo directory, something like photos/2023/11/2023-11-18/). The pictures are lying around there, until I get around to cull them. That means, Kopia snapshots them unculled and the 50% I’m discarding will chew up backup space (they won’T get released, because of incremental backups). I would be wasting quite some space this way? This might be a misunderstanding on my side or may be configurable in Kopia? Otherwise I could of course change my workflow to fit the situation.

Just asking if you have additional insights to share. Would really appreciate this!

Understandable, if you don’t have the time for a long answer.

9

I don’t have 6TB of data to backup. You can always setup a local backup solution (inside your network). I backup locally and remote thru my nextcloud.

1 Like
10

Hetzner would be about half the price, and no transfer costs: https://www.hetzner.com/storage/storage-box

1 Like
11

It is certainly Mylio. It happened to me a week ago, about 90,000 xmp were sent to the trash. Fortunately they were there and I could restore them. Mylio does not like *.ext.xmp files. It only understand what they say are standard *.xmp. The idea is that xmp should only be used for RAW files and jpg should have the info in the file. While that can be correct, and darktable is doing something non-standard, it seems not wise to silently erase anything they do not understand… :frowning:

1 Like
12

Not sure if thats relevant for anyone… @Donatzsky @g-man

There are also lifetime storage offers (currently on black friday sale)

and also

https://landing.pcloud.com/de/BlackFriday2023?label=BF2023&name=&channelid=4342

I mean, for a backup solution you want to have for many years, seems like a good deal, I think?

13

I’m skeptical of anything lifetime. It only last until the company goes bankrupt or runs out of cash and starts implement limits. Also, some of these solutions are for one machine.

1 Like
14

Yeah, actually I’m sceptical too.
If it sounds too good to be true, it probably is.

On the other hand, 10TB for 750€ lifetime against 10TB=60$ per month or 720$ per year… I mean, money wise its a no brainer. They just need to last one year to catch up.

But then again, I’m all for a robust and long term solution and cheaping out on traffic or other soft conditions isn’t fun either.