Potential improvements to import and copy in Darktable.

That’s not correct. The library contains the editing instructions. You can (at your own risk, I would never recommend this) even disable writing XMP files. Also, there’s an option to check for updated XMP files (updated compared to what you have in the library database – again, not something I use). See darktable 3.8 user manual - sidecar files :

In addition to the sidecar files, darktable keeps all image-related data in its library database for fast access. […] If an image is subsequently re-imported, the database will be updated from the contents of its XMP file.

Once an image has been imported into darktable, the database entries take precedence over the XMP file. Subsequent changes to the XMP file by any other software are not visible to darktable – such changes will be overwritten the next time darktable synchronizes the file. On request, darktable can be configured to search for updated XMP files at startup, offering a choice to update the database or overwrite the XMP file where changes are identified.

It was my understanding that the database had tags/colors and such but not the edit of each image. I will need to see what the code is actually doing.

You can try this:

  • disable writing XMP files in preferences
  • import an image
  • changes some value (e.g. bump exposure by 5 EV, that’s highly visible :-D)
  • exit & restart darktable

Don’t forget to turn writing XMP files back on.

LR is a brilliant digital asset management system (DAM). I bought a perpetual licence for LR6 and love the DAM capabilities. I never thought of DT as anything more than a great editing program, but now it is really improving in the DAM capabilities. I do not want DT to become a free LR, but I can just see how a small improvement would really improve the copy and import capabilities,

For example, I am currently scuba diving in Thailand. I downloaded the images from one days diving to my computer for safety and backup. I then went out the next day and returned to the hotel and wanted to download only the images taken that day. I was able to do this, but the task would be so much more efficient and powerful if I had the option to just copy and import new images from the camera.

Thanks for your response Kofa. I do not have any issues with DT database or xmp files. I have set my preferences to only write an xmp file when an image is edited. This cuts back on many unnecessary xmp files. I have never ask the developers of DT to replicate the DAM capabilities of LR, but DT has become so good at DAM now that I throw up this one (hopefully) little improvement which will let me say good bye to Adobe for good. I am even switching some of my computers to Linux to try to break Microsoft’s grip on my life.

Keep in mind that traditionally, Linux uses specialised tools for each task. Extreme examples of this are programs like find, sed, awk, grep, which are meant to be used (also) as components to do one sub-task within a larger task.

In this case, that would mean using a different program to download files, and another tool to keep track of new files to be written to a backup location. MS-Windows software is much more oriented towards “one program does all”.

And darktable is originally a Linux program, with MacOS and MS-Windows builds provided when/if there are volunteers to do so. That’s why you are likely to recommendations for different programs to do what you want.
A big advantage of that approach is that each team only deals with what they know best. Disadvantage is that the user has to learn more interfaces/GUIs…

Now I understand the request and I think it is useful. Your example of traveling is a good one since I dont like to delete my images from the SD cards when I travel in case the laptop gets stolen/lost.

I think it would be best for you to type it into Guthub under issues with a Feature Request tag. I would avoid saying: like LR.

Request:
DT to track previously imported images to avoid importing them again. This should be a similar functionality to the Add Library Select only new images.

Rational:
When traveling, images from the SD card can be imported each day without deleting the images from previous days from the card. The ability to only import new images would make it easier to track and avoid duplicates or missing importing some.

While this doesnt mean the feature would be added, it provides a clear message on the functionality you are suggesting.

@Terry
Yep, I think a feature request is in order; but make sure you emphasise that you would like the copy & import functionality enhanced, otherwise the developers, who are often quite overloaded, may quickly dismiss your request as already implemented (since, for the simple import functionality of add to library, that’s the default behaviour).

The reason is that import means different things to different people; long-time users will remember that import used to mean add to library (see dtdocs/content/module-reference/utility-modules/lighttable/import.md at 2ab4634979632225eefa9849c0a5e62f6facfaf4 · darktable-org/dtdocs · GitHub)

Maybe:

copy & import should check if files are already pesent in the destination directory, and exclude them from copying and importing by default, like add to library excludes files already present in the database.

I can’t ever surrender trust to these sort of things…I need to see all the files physically transferred. I always manually move them to the computer and then organize and then as DT now calls it add to the library later after from there…In fact I don’t really even use the library as I invoke DT using the temporary virtual library … I can just not get comfortable letting something upload and rename my files… what can I say I have trust issues :slight_smile: or more likely I’m a dinosaur…

I also use a file manager to manage files. Often a Norton Commander clone (aka Orthodox File Manager): Multi Commander on Windows (yikes! its website is currently gone! – update: it’s there, but is only accessible via plain http, not https), and Krusader or Midnight Commander on Linux.

Thanks for the suggestions. I actually do not rename the files, but I like to have them put in folders based upon date. DT is doing this very well and I have saved it as a preset.

I was surprised to see the developers improving the import functions of DT. I primarily see it as an editing program, and a very good one at that. I have no desire to see DT become a LR. LR is too simple, but it is easy and fast for those unable or unwilling to learn the far more sosphicated editing options in DT. I also like RawTherapee, but the ability to do masking so well in DT has won me over.

Why? That is how an active open source project works. Things get a little bit better each time until its great. We don’t need a budget or to satisfy a market, “because we want a better application” is enough.

RawTherapee and ART also support local adjustments. See

https://rawpedia.rawtherapee.com/Local_Adjustments

1 Like

Greetings Terry,

I also agree with you that DT is killing LR and I think your request is much needed but as an “until then work around” what I do is sort the import files by date and pick the day in question. I do rename the files based on Date and Time taken and I end with a sequence number in case my rapid shoot takes two a second. If I then accidently import a photo twice it will create a new rendition of the photo instead of realizing it has already been imported. So, I understand your request is very legitimate.

Thanks for your response Kofa. I haven’t explored ART. But when I last looked at Rawtherapee when it mentioned support for local adjustments I felt the local adjustments did not compare to the great masking system in DT. I feel Dt has the best masking system I have used. I often use a drawn mask with the path tool to roughly select the subject. No need to be too critical. I then apply the adjustment such as a new instance of exposure. The transition is terrible at this stage but clearly reveals what I have done with the adjustment. I then add feathering and blurring radius for the mask until I achieve an invisible transition. I just love the way DT lets me do this. Of course there are also occassions when I use the parametric masks. DT’s real strength to me is the masking and that is why it has won me over from both LR and Rawtherapee. This is based on my experience and I respect that different people will have different experiences. BTW, what impressed me with Rawtherapee was the ability to tackle noise by picking different demosaic algorithms. Now I see a similar option in DT and I am very impressed.

Hi Paperdigits,
As a humble user who is incapable of coding I never felt the right to demand something from the developers or to criticise the developers or DT. I certainly felt no right to ask for a replacement to LR. I have used DT since about 3.2 and the developers seemed focused and motivated at improving the editing side, which they have done very well with. I yes I was surprised a developer felt motivated to improve the DAM capabilities of DT, because that did not seem to be a primary focus for them. So my suggestion here is what I hope is a small task for a developer (since the feature exists for the add to library). However, it would have great benefits when I put my memory card in to copy just the new images. As ever, I am so appreciative of the hard work and dedication of the developers.

Not necessarily hopeless:
image

But as there are a lot of waiting PRs in the pipe, you may have to be a little patient…

3 Likes

Does this screengrab indicate that someone is working on it?

Yes, I do.
The identification of the image is done with its original name and its timestamp. This should work for the memory card you mount on your computer.
EDIT: should work too for mounted cameras.

2 Likes

Will it work if you rename the images after import? That was my main concern when I was thinking about how to implement this.