Context:
I’ve recently switched from Lightroom + Photoshop to darktable + Affinity Photo 2 and have loved it so far. My typical workflow is as such:
Export RAW files to TIFF for retouching
Import new TIFFs into darktable, adjusting the photo stacks
Work on retouching the TIFFs
Bounce back and forth between darkttable and Affinity Photo 2 to see changes reflect in darktable (for any additional darkroom tweaks that I might want to try
Need Help With:
I can’t for the life of me find or figure out how to force darkttable to refresh a file that I’m working on. I end up having to reopen darkttable to get it to show the latest version of the file.
Is there some kind of “Force Refresh” button somewhere that I’m missing?
Or does this feature not exist?
Since darktable is primarily a raw editor, it assumes its input files never change. You can try to remove (not delete) the file and reimport it. I’m not sure if that will invalidate all caches, though.
But why bounce back and forth, and not just use darktable? Is there a missing feature?
There are scripts to integrate external tools, see here:
It might be worth sharing an image in PlayRaw category with a licence to see how others would edit with just DT. I don’t understand the benefit of bouncing back and forth between the two. DT for me does the editing process for most images. I do stitch panoramas in a second program but that is about it.
I also don’t understand this workflow, the more as darktable is not the best option for detailed retouching (with pixel precision).
And you have an additional problem: any retouching you do in darktable will be based on the file as it is at the moment of editing. What is stored are the operations, not their result. Replacing the image file basically makes any earlier work in darktable invalid (depending on what you do in each program).
It might help if you can clarify why you use this combination of programs in this order, i.e. what you can do in darktable and not in affinity. I suspect part of the reason is that Affinity feels more like Lightroom…
I guess I didn’t clarify some key details of my workflow so I’ll try again:
Use darktable to manage my RAWs
Export flagged/selected RAWs to TIFFs via darktable
Do retouching work on exported TIFFs with Affinity Photo 2
Re-import exported/edited TIFFs into darktable so I can see the edited photos in the lighttable view
Occasionally bounce back and forth between darkttable and Affinity Photo 2 as I make further retouching tweaks
Use darktable to export TIFFs to JPEGs (much easier for me to manage multiple photos this way)
I use workflow is for portrait/headshot photography where I do significant pixel-retouching in Affinity Photo 2, but use darktable to continue to manage my RAWs + TIFFs in the same collection (I group TIFFs with the source RAWs).
Right now, whenever I make changes to the TIFFs, the only way I know to get darktable to update the cache for them is by restarting the program.
I don’t know how much retouching you do. I do very little, and simply use darktable’s retouch module (which, according to many, is not intended to do many changes). Other than that, it’s rather capable, with multi-level wavelet-based editing.
Notice that it was possible to greatly reduce the dark spot while keeping the hairs (not really that important when zoomed out, but there may be situations when this is desirable).
I am aware of the retouching tool and appreciate you pointing it out.
However, my needs require way more than what it can provide - hence the separate usage of Affinity Photo 2.
Here’s an example of the amount of work that I do for a portrait (before and after):
As I get closer to finishing retouches on a photo, I tend to jump back and forth between Affinity and darktable so I can view them in the same series of other photos I’ve taken from the same shoot.
I normally have darktable open in the background and I don’t see my changes reflected in it when I make changes to the TIFF (I’m not going to blow out my entire cache to regenerate just for a single photo - it’s way less time consuming to just restart darktable).
While this may be a weird workflow for some, I’m really effective with it so I wanted to know if there was a way I could shave myself off some seconds if there was a handy “Force Refresh” button I may have missed.
I appreciate the link to the lua script examples and will look at implementing something like this myself!
Darktable is just not designed to notice/accept external changes to image files during a run. Not too surprising given that it has as a principle to never change input files (raws don’t change usually).
@rych : I don’t think there’s a way to get ‘vanilla’ darktable to do what you want.
I wonder whether an external DAM program might be better suited to what you want? Still using darktable for raw development (and perhaps jpeg exports) and Affinity for ‘fine’ editing.
If you don’t want to do that, there are a few paths to look at:
disable the disk backend for the thumbnail cache, that at least forces darktable to recreate thumbnails at each run. It still won’t update them during a run (not in a predictable way, in any case), and it will slow down lightable browsing;
perhaps a lua script could help. I never used lua, so I can’t help there. I expect you’d have to start such a script manually when needed, as constant monitoring of the collection can get expensive in CPU time.
I don’t want to use another DAM - I’ve tried a lot of them and the one thing that darktable does extremely well vs the competition is its speed. And from what I’ve read on their developer’s guide, I can see why and really appreciate that the devs pay attention to performance.
I don’t want to disable the disk backend thumbnail cache because that’ll hamper the primary reason I like darktable over the other DAMs. I also don’t need to build something that complex that requires constantly monitoring (which again, performance).
All I want (and will need to build) is a module that shows up in lighttable and/or darkroom similar to how the export module works for both, gives me a “Refresh Selected” button, hence removing the step of me having to wait about 5-10 seconds for darktable to restart.
Thinking out loud, but there might be a way to force a refresh with a LUA script. If you go to the darkroom and move a slide so the pipe executes and go back to the light table, does it refresh the TIFF?
I’m aware that there are other programs that have more tools in the retouch department than what dt may have, but have you taken a look at what @s7habo does with portrait retouches in dt here: Editing moments with darktable - #576 by s7habo ?
@rych here’s a lua script that will do what you want.
It adds a button to the actions on selected images buttons (and a shortcut that you can assign). Select the image (or hover) and hit the button (use the shortcut) and the cache will be dropped and regenerated for the image so lighttable will reflect the latest changes.
I’ve installed and used your script but it only seems to regenerate the cache once and every subsequent update of my file doesn’t work anymore.
I thought it might be mip related and tried changing the mip size range with image:generate_cache(true, 0, 8) to no avail.
I couldn’t find anything obvious about why the script wouldn’t work. I checked the mip cache folders and they weren’t being generated from the full range of mip sizes specified (0-8). Any idea what might be the issue?
You have to force the update each time you return to darktable. If you weren’t on Windows, then we could set up the script to watch for when the file was modified and automatically refresh the mipmap. Unfortunately Windows has no easy way to do that.
@EspE1, took a look at the video process and while helpful, it’s not the way I work when retouching my photos. I work in quicker, smaller areas as paint around my images, which darktable doesn’t really excel at.
For example, in the video, if you look at the output for his first image:
This generally works well for portraits of people with smooth skin and minimal ridges on faces, I usually end up losing a lot of detail in face shape when targeting large swaths to retouch with frequency separation - it also sometimes produces a ‘plastic’ feel to a person’s face.
I try my best to keep a more natural retouched look as much as I can and find that working in smaller areas, patching and painting as I go, yields the best results that I’m looking for.
For portraits, this means I primarily use darktable for initial RAW processing (i.e. lens corrections, white balance tweaks, color correction etc) while continuing the rest of my work in Affinity Photo 2. I am much more used to the layered and painterly workflow there, especially with a pen+tablet.
I appreciate the link though! There’s some useful information there that I didn’t know about and learned some new things from it.
@wpferguson Thanks for looking at it and thanks for your time!
Are you one of the contributors to the project?
I was using your script to reference the C code to see if I could glean anything from it (and also just to understand the code a little more). Outside of the terminal output, does darktable output logs anywhere I can look at?
