I use sophisticated folder structure for wildlife photos, with sub-sub-sub-subfolders for animal classes, families and species. So I have folders like ~/igor/Photos/Animals/Birds/Sheathbills/Snowy_Sheathbills/2020.08.17_Place_name.
Some less sophisticated structure for other photos. Eg. ~/igor/Photos/2020/Portrait/2020.07.11_Session_name. with subfolders like …/Processed_BW or …/Processed_Bright etc for different purproses.
So I prefer to sort photos in external software - copy from camera’s card by filemanager and view/sort in Rawtherapee as RT’s view/sort tools (especially “inspect” module) are more convenient for me. I “only” use DT as RAW processor.
I have set Settings > Import > Import > “Recursive directory traversal when importing” to OFF , to not show all processed JPGs alongside RAWs in Lighttable and Darkroom.
After sorting, I choose folder in my external software (or in filemanager) and choose “Open folder with > DT”. So I just get DT opened with freshly imported filmroll containing only RAWs from main folder, without processed JPGs from subfolders.
Until this moment, Easy and clear. BUT. I get big mess in “Collect Images” module. I cannot just click on “filmroll” and delete from database.
Is there any method to purge collections automatically on exit?
Thanks!! It’s even better than I expected. I can add rm /home/igor/.config/darktable/library.db to my startup script or to crontab. Now I can have all today’s imported folders (few) in “collections” module, then start with non-messed DT tomorrow.
Or see the startup parameters in the manual, especially “–library :memory:” which creates the library in memory (so it disappears on program exit) Not as suitable if sometimes you want to restart dt with the previous directories…
But keep in mind that the library not only stores the film rolls. It also stores:
history stacks (so make sure you write xmp sidecars if you remove the database)
user presets
styles
tags and ratings (also stored in xmp sidecars if you set dt to write them).
Meaning that you can lose edits on database removal and can’t use search facilities on older images…
Using styles and user presets also becomes impossible. So no easy addition of a basic group of modules with one click (e.g. lens correction + filmic + exposure + contrast equaliser) or on import.
I wonder why you can’t/won’t use the “recently used collections” tab, or the “folder” view in “collect images”. The latter gives you a file system view of (only) the imported images.
Oh, and if you want to delete the database through a crontab entry, be very careful. I don’t think dt will be very gracious when the database disappears in the middle of a session.
Yes, it’s ok until one day (night) I work a bit longer… Thanks for remembering.
And whole “magic plan” fails…
Yes. I like I idea of sidecars and use them.
I prefer my folder structure over “libraries”, “collections” etc. Folder structure is “obvious, consistent and the same” for any software I would ever like to use. No need for database. Last month I made slideshow about my 10 years in mountain guide association. Did not have problem finding eg. 8 year old pictures I wanted to include into presentation. Just went into right subdirs and picked what I needed. (ok, up to septemper 2019 I only shot JPGs).
Any view that doesn’t reflect that structure, is for me more confusing than helping.
I like the way Rawtherapee sees filesystem. RT has just file browser. User enters the folder and sees images. In this folder, but not including subfolders. Picture-related info stored in sidecars, settings in confog file, styles/profiles in XML plain text files. Simple, usable, easy to backup/divide/share_one_style_to_friend/whatever.
DT’s “collect images” module shows all (imported) images in folder and also in subfolders, no matter I choose “folders” or “film roll” view. Ok, I know about “grouping” etc but it doubles what I just have in mind or paper notes.
When I bring a bunch of newly taken pictures, I use ratings. I rate 4 and 5 for best pictures, “delete” bad ones and rate 1 for “maybe keepers”. Then I leave them for some time. Next time I re-rate. (I sometimes view other (much better) photographers’ pics on the same topic before re-rating). Then I permanently delete all pictures below 5. Since this moment I don’t need ratings/tags for these “keepers” anymore.
Very sometimes I keep pictures that don’t deserve 5. Only if I really don’t have better ones and want to keep these “ok” ones as memento. In such case I keep JPG in 1600px for longer side and 80% quality and delete RAWs. Such procedure keeps from trials of wasting time on processing pictures “other than best” in future. And saves disk space.
Ok, then we have a completely different approach. For me, disk space is cheap (relatively speaking…) which means that I tend to keep raw files around whenever a picture is worth developing. One reason for this is that screen sizes keep increasing, tools keep improving and my knowledge about how to use them keeps increasing. And my ideas about how a picture should look change as well.
That means that I want to be able to revisit an older image and re-do the editing. That’s impossible without the raw file…
Wrt. not being able to remove film rolls from “collect images”: that doesn’t work for me either. However, I can remove folders from the database through the “collect images” tab (i.e. switch to ‘folders’ mode, select the folder you want to remove and right click.). Afaik, that removes the images from the data base, but leaves the xmp files in place
Yes, when it’s worth developing. I’m talking about “phone-quality” memento pictures, badly composed, a bit outfocused etc. for which any development is pure waste of time.
Pictures I keep (and rate 5) aren’t in fact 5-star ones, as I’m beginner - for now, it’s one year of serious interest in taking photographs (not “point&shoot”). So really no worry about my today’s <4 ones in future. They are just bad and wouldn’t be any better with master-class processing. Maybe when I become a photographer, I would take more pics worth keeping.
Disk space. Of course 10 GB is ultra cheap today. But 10GB of 18Mpix RAWs mean 500 images. 500 separate things with some content, story behind, and requiring 500 processing operations. THIS makes every 10GB really much and worth saving.
You can start darktable with the parameter: --library :memory:
Then it just creates the image library in RAM which will be erased after exiting darktable.
There’s a feature request to make this an option in the program https://github.com/darktable-org/darktable/issues/2386
If rawtherapee --library :memory: does save presets/styles but doesn’t save collections/imports, it’s the point. Sidecar files are just what is needed to keep picture-specific history stack, but if “library.db” also keeps presets/styles, it’s a bad news.
Styles from dtstyle.net are just plain text XML files. Can DT use/store styles and presets in this form? Eg. in ~/config/darktable/styles/ directory. I know that it can export/import styles as *.dtstyle files.
The perfect option would be to start DT, browse into folder, see/view/rate/sort/delete RAW files (files, not “pictures”) just inside this folder (folder, not “film roll”). Then switch to darkroom and start processing. I tried but don’t see any option that can reflect my workflow in DT.
Yes, I have tried both filmrolls and folders (and other sorting options, thru I don’t need them). But it still requires importing folders before they can be visible in DT, so very partly usable. I can’t just browse into eg. card reader/pendrive. I have to import its content, and I get inconsistend “pics library” when I disconnect a reader, because DT remembers folder but doesn’t see pictures inside. And importing a folder is not enough, when I move card back to camera, make few shots and insert a card back into reader, DT doesn’t see new pictures until I re-import the folder on card. I know the reason but it’s just not my way of managing files.
Last but not least - no matter if I chose filmroll or folder. If folder contains subfolders with picture files, DT’s lighttable shows all pictures from folder and its subfolders as they were all in main folder. To make DT not show pictures from subfolders, I have to import with “import from subfolders” option disabled.
The same way I treat my ebooks and music files. No “library/collection” in player/reader/manager software (I use Calibre only to convert book files), or (even worse) “sycning” with Pocketbook device. Files are files, I can manage them using filesystem (and folder names, and file names) and that is intuitive for me.
So, I find DT being powerful RAW developer but its file managing capabilities are counterintuitive for me. If I cannot use regular “browse&open” way, I would rather use external file manager. That’s what I do - click on folder, “Open with > Darktable”. And this is ok and comfortable for me. I only want to get rid of “collection database” inside RAW processing tool.
If possible without deleting profiles/styles.
Normally I just copy new RAW files from SD card to ~/Pictures/temp/yyyy.mm.dd_session_name folder, then select, process and move to destination folder somewheer under ~/Pictures. But sometimes I view/select directly on card, eg when there are some not very important test shots and only want to see effect.
Why use these tools to copy files from card to harddisk? Do they make something special/better/faster than filemanager during copy operation?
Thanks, Todd. Sometimes I forget that FOSS programs can be well documented also for people that don’t analyze source code
So, main problem I had, is solved by auto-removing library.db, as @paperdigits said. But @rvietor mentioned that by deleting library.db I can loose styles and settings - fortunately not. @rvietor thanks for warning, @priort thanks for confirmation that it’s safe
Now if DT can work on files/folders direcly, then it’s perfect. If not, I just can load folder into DT by invoking with folder name (by file manager) and it’s also ok.
Ya I think with memory is the closest but it will open with nothing so then you have to import folders each time to see them. This gets wiped when you close DT but at least in the move dialogue you can create folders and bookmark folders and select go to location so if you were to upload to an import folder you could create a bookmark to your base directory and then move them. I also use this syntax for my exports it might be useful to you. So my raw files are in a folder called DTPhotos with a subfolder structure based on year and event or project I think DT likes to call it etc etc but …I have a preset that uses this syntax when I export… (FILE_FOLDER/otos/otoEdits)/(FILE_NAME)-DT so this will automatically save my exports in to exactly the same folder name but in a folder called DTPhotoEdits…This keeps my jpg edits separate and with no xmp files in the directory but he directory structure remains exactly the same as the raw folders…Now I can browse the exports with other viewing software of share and no xmp files are created as I don’t open these folders in DT to look at the files…I add the -DT as sometimes I am editing older jpgs and that clearly gives it a different name than the original. DT has a new variable $timestamp that is recording when the xmp was last revised so that can be used to create a sort of recent files query by specifying a date you can select files that have been edited in a date range or recently…so lots of tweaking but its not the same as directly having the folder structure…
So, you automatically double your “Source/RAW folder structure” to “Processed” folder tree?
My way (I learned it from DT’s default setting, then extended) is to keep processed files in subdirectory of source files.
For example I have folders:
~/Pictures/2020/2020.07.11_Session_name/
~/Pictures/2020/2020.07.11_Session_name/Processed_BW
~/Pictures/2020/2020.07.11_Session_name/Processed_for_Marie
In “processed” folders there are also sidecar files.
Your method separates source files from processed ones, such separation can be usable some day. Worth considering now, while I have small amount of pictures.
Would be a nice trick, but working directly on physical folder structure is more important to me.
I’ve been using exiftool in a script for a long time. It renames the images based on exif information and copies them safely to disk. Its a command line application.
Rapid Photo Downloader is a GUI application to manage your downloads from SD Card and can manage the downloads from the card for you, add job codes and many other things. I don’t have a lot of experience with it, but a lot of people swear by it.
BUT. I get big mess in “Collect Images” module. I cannot just click on “filmroll” and delete from database.