Importing Lightroom XMP sidecars

Before discard history:

dt_history-1

After discard history:

dt_history-2

Everything after step 11 are operations that darktable tried to emulate from the Lightroom XMP sidecar file.

Step 1-8 are essential ones, 9 and 10 are being applied due to your settings in preferences. And as you say, in 11-16 dt tries to emulate lightroom modules.
If you want to get rid of all modules except for 1-8, select 8 (orientation) in darkroom and press “compress history stack”.

I tried importing a raw file with Lightroom XMP sidecar using all 3: scene-referred, display-referred, or none. Each one looked quite different. Using none was the worst, at least for the image I used to test.

I am confused now by your reply. Before you said to use discard history, but now say to use compress history stack. I have about 120k photos to import.

Yes, that is to be expected. You should read some documentation or watch some videos about the different modes to understand the differences. As a starting point I would recommend some videos of Bruce Williams. Choosing “none” and disabling “auto-apply sharpen” and “auto-apply per camera basecuve presets” in preferences - processing" you will apply no modules at import, except the essential ones, and, in your case, the modules dt uses trying to emulate lightroom.

I’m trying to find a way for you to get rid of all settings imported from your lightroom xmp, because

We have seen that discarding history does not do the job, so we have to find another way. Compressing the history stack does the job, but only for one image.

Actually I don’t see a solution to do this with your prerequisites (saving all your tags and metadata) in an automated way.

I don’t know why discard history isn’t working though. If it’s not removing your Lightroom edits it feels like this might be a bug.

I don’t know how it is intended to work in this special case, but I have an equal opinion. For me it makes no sense to handle “emulated” modules in some way other than every other module applied by the user.

1 Like

in the documentation you mentioned in your initial post you can see that there’re only a few modules that can be initialized with parameters ‘like lightroom’.
Basecurve or filmic can’t receive paramters from lightroom since these are quite different ways to process the raw file.
So you can use scene or display referred workflow or even no predefined workflow - in each case only the mentioned modules are affected by lightroom xmps.
So if you want to get exactly the lightroom result - you’d better use Lightroom or spend a lot of time to tweak basecurve or filmic or the whole bunch of darktables modules yourself.

MStraeten, please re-read my initial post.

The manual says that dt will automatically use these development steps in a LR XML sidecar file:

crop and rotate (crop and rotate)
black level (exposure)
exposure (exposure)
vignette (vignette)
clarity (local contrast)
tone curve (tone curve)
HSL (color zones)
split-toning (split-toning)
grain (grain)
spot removal (spot removal)

One of the raw files I tried in a test has spot removal, but dt gets it wrong. Seems to be in the wrong place in some places, doesn’t totally get rid of some spots, etc. Worse than doing nothing.

I think that when importing with a LR XML file then we should have the option to only get the metadata (tags, star ratings, etc.) and not get the development stuff. Even better it would be good to be able to choose which development options to get because some may work well. Maybe crop and rotate, for example, works well.

Well, if those sidecars are indeed XML files (i.e. including the required XML header), XSLT should allow you to select only the tags that are useful to you (or alternatively, only discard the ones you don’t want). Once you have the XSLT script, you can apply it to all the sidecars you want to import.

However, some XMP sidecars are not valid XML, as they are missing the XML header. Then this will not work, unless you can add the header “on the fly”.

Adding all of this in darktable might be a bit of overkill, unless there’s a significant number of users that regularly need to import from lightroom (what is significant in this context is up to the developers, as they furnish the effort).

Maybe because you expect something that is not granted. In no way the import will give you a good result, it does 70% to 80% of the work. You then need to review all pictures, so if on some images the spot are wrong just press the reset button on this module. No big deal.

(Note: I migrated from Lightroom to darktable years ago.)

The parallel to Lightroom’s spot removal in darktable is actually the retouch module, not spot removal (which is older and simpler). Sadly, the importer does a direct port to the older module, not the better one. I’d recommend dropping spot removal and switching to retouch.

It’s true that the retouch module can be complex… but in most cases, you can safely ignore the wavelet stuff. It’s super-powerful, but optional.

The basics you should know:

  • shift-click on the image to set a source point (optional, but useful to avoid adjusting it later)

  • control-click on an icon in the retouch module to stay in that mode, so you can continually retouch the image (such as dealing with multiple dust spots in the same go)

  • use your scroll wheel to adjust the size; hold down shift and scroll to adjust the feathering

  • you can turn on the highpass module to help identify dust spots in the sky more easily (and then turn off highpass when you’re done retouching) — this is useful if you’re not using wavelets (as you have other methods when using them)

  • right-click to remove source-destination pairs on the image (just like masks and other things on the image in darktable)

  • like most other modules, you can duplicate this one too (from the icon that looks like two overlapping rectangles in the header bar) — duplicating helps you use different modules for different targets on the image (such as one for sky, one for ground), so you can adjust each differently (such as with masking)

You should consider spending some time to learn about the wavelet features in the module, especially for skin retouching. Again, it’s not necessary to use the module successfully — but it’ll help you get better results, especially with trickier removals.

Here’s an amazing video on the retouching using the retouch module and wavelets. It’s around 40 minutes, but totally worth your time:

(Note: He’s using an older version of darktable, as this was filmed over a year an a half ago, but the technique is the same and the module hasn’t really changed since, so it still applies fully.)

And, of course, here are the official docs on the module: https://darktable.gitlab.io/doc/en/correction_group.html#retouch

1 Like

I fully accept that importing the LR development stuff doesn’t work well in dt. Then why is it mandatory? Why can’t I choose not to have dt import all that stuff since it cannot do a good job with it? That is the whole point of this thread: It doesn’t work well so I don’t want it. How can I turn that off?

That would destroy all my LR XML sidecar files. I don’t want to destroy them. They are used by LR.

Why not just have an option in dt that says only import the metadata stuff and ignore the development stuff?

If you think this is an important feature, best thing is to raise a feature request on github:

You could version control your XMP files or work on a duplicate set of XMPs.

The XSLT to rid yourself of the LR processing steps is likely to be straight forward, if you’re interested in help

Thank you to everyone.

I tried this again with 3.4.0 and the result is the same. The extra, incorrect development steps put in by importing a photo with the Lightroom XMP file are not removed.

Here is an update on using the discard history command after playing around with it some more. It works for only editing steps I explicitly do myself. It unfortunately does not discard mandatory and unwanted ones done by the Lightroom XMP file importer.