It’s a few years of different cameras, different phones, spread to multible folders in several HDDs, some on not connected HDDs in the drawers, probably lots of doubles. Could be well over 10,000. And that is just me, there are more people around.
I have a NAS, used for other stuff right now. But I like the idea to have ONE central point where I put everything. The main NAS is connected to fiber (fast) - at home I have ADSL (means I can D/L stuff, but not really upload).
I am trying now Digikam on my PC only with folders on the NAS right now - and it’s painfully slow. Like 1% progress in 30 Minutes. I wonder if I can really use Digikam on remote folders. Or is that because of new install and Digikam needs a day to scan everything?
And how would that work using different PCs with Digikam on the same remote folder? Can the db be shared or synct?
If I get more speed I would probably go for USB3 HDD storage.
That’s the case. Digikam uses a db, i.e. it needs to scan everything and add it to the db. Once that is done browsing is much faster (thumbnails are also in the db). The speed of scanning also depends on how you access the NAS, e.g. smb is much slower traversing and stating huge file trees compared to nfs on linux (sshfs probably in between or even worse than smb, don’t know).
Talking about the db: Did you switch to mysql/mariadb? For large collections that superior to sqlite.
No unfortunately the db is not multi-user ready. I think it would be possible to share a db, if you make sure only one digikam instance is running at the same time (needs some tweaking of collection paths if I remember correctly). However that’s probably not worth the effort - one db per client is probably easier.
In my opinion, using commercial software to organize an image archive is not a good way. Nobody can guarantee that the data base can still be used in 10 years (yes, I am thinking about my children and grandchildren, who might want to use my images). E.g. Google has stopped support for Picasa.
I work on the assumption, that plain text files live forever. So I rely on the metadata, exported into text files, which in turn are imported into an EXCEL spreadsheet. The latter can, if necessary, be exported again into a plain text file.
The EXCEL table allows me to select images according to time, events, geographic coordinates … and I can open images from within EXCEL and I can copy the selection into a separate folder, e.g. to copy the images into a DVD or stick. That is all I need.
There’s just no clear cut there, it’s a transition between here longevity and usability on the other side. I mean you could spin your principle farther by saying you are relying on spreadsheets or heck even computers → print and catalogue everything on paper.
I do agree with the general sentiment, that’s why I like that Syncthing writes tags/geolocation/… to the metadata of the pictures. So if it disappears I could fall back to somethin like what you describe (or to whatever other solution that supports xmp metadata, which is a standard).
The first scan after digikam installation can take a while. I would attach the harddrive with the images directly on the computer with digikam and move image and database to the NAS afterwards.
After the first import digikam just searches for changes, that does not take long
I rely on a simple directory hierarchy. First level is a place. Inside each place, there is one directory per year. Inside each year, I have a directory for raws, where the RT PP3 profiles are also stored; a directory with the processed JPGs; and a directory with camera JPGs in those cases where I want to keep them.
I also have a few top-level directories for major events that I want to keep separate.
This approach is not without drawbacks but it has served me well so far, and I have close to 20,000 images organized like that.
[quote=“pphoto, post:7, topic:15953, full:true”]
I would attach the harddrive with the images directly on the computer with digikam and move image and database to the NAS afterwards.[/quote]
That sounds very doable! So I will get ALL image folders I have around on the my 1Tb USB3 drive.
Here comes the first question, how would I arrange the folders? Just leave them as they are? Doing any topic structure would be impractical. Some folders (like from iPhones) have random folder names, such as /gsdwevd/.
Getting all image “flat” and then putting them by creation date in folders should be easy. But then I destroy existing folders, such as i.e. /food/ or /california2006/.
So when I do the first scan I would go straight to MySQL/Maria, right?
The db will be on the NAS (it has MariaDB) or on the desktop? If on the NAS using it with different PCs should be no problem. Two devices accessing at the same time will be unlikely.
Last question, are there Android/iOS apps that can work with the image folders, or the Digikam created metadata?
That sounds very doable! So I will get ALL image folders I have around on the my 1Tb USB3 drive.
Here comes the first question, how would I arrange the folders? Just leave them as they are? Doing any topic structure would be impractical. Some folders (like from iPhones) have random folder names, such as /gsdwevd/.
Getting all image “flat” and then putting them by creation date in folders should be easy. But then I destroy existing folders, such as i.e. /food/ or /california2006/.
So when I do the first scan I would go straight to MySQL/Maria, right?
The db will be on the NAS (it has MariaDB) or on the desktop? If on the NAS using it with different PCs should be no problem. Two devices accessing at the same time will be unlikely.
Last question, are there Android/iOS apps that can work with the image folders, or the Digikam created metadata?
Just a quick suggestion, as you proceed to get all your eggs into one basket…I’m sure you have already thought about this, but make sure you properly backup that big basket elsewhere.
I use Darktable for organizing my original photos. The photos and their xmp-files are stored on a FreeNAS server, axcessed over NFS from my Linux workstation. The thumbnails are stored in darktables database-file. Since the database is stored on a SSD in my workstation, searching for photos is very fast. Accessing individual photos on the server is also fast enough. With NFS I can max out the gigabit connection to my NAS.
I then export photos to Plex mediaserver for viewing in the browser, on phones and tablets.
Benefits to me with my solution:
I use the same photo database for archiving and for editing photos.
Metadata I add in Darktable are either found in the exported images or in the xml-files Darktable saves. It is easy for me to access this metadata with a python script, which makes the solution futureproof for me.
With FreeNAS I get the ZFS filesystem, with snapshots, replications and data healing using RAID-Z1.
Having ONE location for all my familys photos makes finding and backupping photos SO much easier.
The only thing I’m missing is off-line facial recognition. I wish someone would add it to Darktable. I don’t have the skills to do it.
I use YYYY/YYYY-MM folder structure with some YYYY-MM-DD ‘Name of event’ for some major events like birthdays, vacations, weddings, etc.
I also use yyyymmdd-hhmmss.ext filenames.
exiftool, digiKam or Rapid Photo Downloader can do that for you.
I write metadata to jpegs themselves and xmp sidecars for raw and video files.
It seems to be a pretty bullet proof technique since I have already migrated from Picasa to WLPG and from WLPG to digikam.
XLS file format is owned by Microsoft. If I were I would go with ODS in case you absolutely have to stick with spreadsheets. XMP sidecars are text files though and I would trust them more rather than ODS.
I tag activities e.g. skiing, hiking; people (family members), geolocations, etc. in digikam writing into jpegs and xmp sidecars.
I also use digikam’s reverse geocoding which converts GPS coordinates to places, e.g. Canada/Alberta/Banff or France/Paris, etc.
I think folders by year, plus some with year/event makes sense. Details I get by tagging later anyway.
After that I try some test with MariaDB. I though I have a few images, but so far I’m at 60000, and not finished yet. I will do the indexing (will probably remove 10000+ doubles) the next few days and then test with local and remote db settings.
I have just recently migrated from Lightroom to Darktable. I was using nested collections in lightroom; need to figure out the best way to do this in darktable.
I had a few top level collections - 1)Family 2) Trips 3) Photowalks etc…Inside these I had further collections based on event or time. Like under “Family” I had “2010 Kid’s 1st Birthday”, “2011 Kid’s 2nd Birthday” etc. One photo could be in multiple collections, like in “Family” and also in “Trips”.
Then I used a LR plugin to export all these as jpgs, making folders based on the collection hierarchy. Any changes could be easily updated with only the relevant files getting updated.
With all my jpgs organised into folders based on the collection, I did not need any DAM software. I just used faststone image viewer and browsed by folder.
There are two ways of emulating what you want. If it is not important to have the same photo in several nested collections you could put your original photos in a folder structure that mimics your collections. Ie Persons/Greg/ etc
You could then use the folder structure when exporting. Set the export path to dollarsign(FOLDER)/dollarsign(FILE_NAME)
Only works with one level deep nesting.
Another way is to create hierarchical tags and then use the subtags and sub-subtags when exporting.
Ie assume you create a tag “Persons|Family|Greg” and tag all your photos of Greg with that tag.
If you then use the following export path Darktable will export to
