lens correction preset with dynamic focal length

So it’s not possible. OK. Thanks for clarification.

Interestingly, it does not store all current values, if I don’t touch any of the parameters. So if I save the preset in the first place, coming from some random picture where the focal length is set to some random value. In this case, if I save a preset, this random value is not saved in the preset.

1 Like

There is already a bug report open at github:

I also suffer from the problem of darktable not correctly autodetecting my lens and thus relying on something like dynamic presets.

This and a ton of other missing features regarding presets and styles would be a few keystrokes away if the settings were not packed into that dreadful data-blob inside the xml.

And please no one tell me it is faster … the xml and settings are fractions compared to the actual image data that gets rendered over and over again.

We await your patches!

No, you don’t… :wink:

The internal XML structure is not likely to change, sorry.

yay for tech debt.

btw … if I win the lottery this is really on my list to throw money around.
ah, well, one can dream.

Two years later and the decision to put un-editable binary blobs into the sidecar files is still one of my favourite software-design failures. Really a shame for an app that tries to be technical.

But well, I also have not won the lottery. :woman_shrugging:

foss isn’t about demanding but contributing …

2 Likes

What blobs are you talking about?

     <rdf:li
      darktable:num="3"
      darktable:operation="colorin"
      darktable:enabled="1"
      darktable:modversion="7"
      darktable:params="gz48eJzjZhgFowABWAbaAaNgwAEAOQAAEA=="
      darktable:multi_name=""
      darktable:multi_priority="0"
      darktable:blendop_version="11"
      darktable:blendop_params="gz14eJxjYIAACQYYOOHEgAZY0QVwggZ7CB6pfNoAAE8gGQg="/>

The darktable:params and darktable:blendop_params blobs, presumably

Y’know, I just had to implement a “blob” format for G’MIC scripts embedded in the rawproc toolchain, so they could get past command line shells. Didn’t really want to, but sometimes the mix of tools and environments compel such…

1 Like

Pretty sure it’s a hefty amount of work to move all the params to human-readable, and I assume no devs consider it worth the effort for the small reward. If it were me, money would not change that (unless someone wants to split their lottery jackpot prize with me).

Martin, we might have a misunderstanding.
I was talking about a favourite thing of mine. :sunglasses:

Well said.

It is definitely not one of those changes one can make on a whim because it effects every. single. one. of those million-billion-trillion sidecar files out there.

For most of the use-cases of hand-editing styles I have found work-arounds, but when you search and find your own answer that is not an answer then you just have to answer yourself again, because lens-correction is the one tool where I still haven’t found what I’m looking for¹.

Anyway, sorry for resurrecting a zombie-thread.

:woman_zombie:

¹) Time for some music from the past

We’d still need to support the blobs as well, which I guess would be its own nightmare.

Might as well summarize the issue here again, since it keeps coming around.

Introspection information describing the data format fields is available internally (for the current/last version of each module) so writing human readable files to xml and reading/verifying them (for the current version!) isn’t a hard task. @houz famously was there 90% of the way, without showing his workings, thereby discouraging anyone else from trying and getting rejected for not reading his mind on how to do it “correctly”.

The “problem” is with supporting previous versions. Because they are not introspected and therefore we just blindly trust that the blobs we receive are the right format (and size!). Human readable/editable means easier to mess up, so we’d get crashes. Or we’d have to strengthen all the legacy version upgrade paths, which we don’t want to touch because we have no test infrastructure to catch the inevitable breakage. Going forward, rework the legacy support to ease maintenance · Issue #14608 · darktable-org/darktable · GitHub wants to make this more robust (no idea why that is marked “good first contribution”; you really want to understand dt in-depth before touching that).

So switching to human-readable-only might be a no-go. But exporting both, and using the human one if the blob has been deleted/is absent (or some other indicator) and if it is the current version, should, again, not be too hard to implement. And might be sufficient for automation purposes, as long as scripts are immediately updated when a new module version has been introduced.

1 Like

Thank you very much for that concise insight into the situation.

Do you know if the introspection can deal with partial settings?

If yes, one might do it through a separate commandline util like darktable-generate-cache. Because although possible, editing files en masse is always a dangerous thing. But taking a style, output to unblobbed format, edit, repack to a valid blob seems like a viable way.

This way pretty all legacy treatment can be ignored. Maybe the blob-reader would need a little hardening against incomplete settings, but even that could be limited to the “apply style” path.

To further explain; “introspection” just provides a description of the parameter fields of each module (for the latest version only). How this is used is up to whoever wants to use it.

So I was imagining one would want to have a complete XMP, but with the info for some (or all) modules’ parameters in “human readable” form. But instead one could also have a separate XMP/XML/csv “adjustment” file that could be read (via a script? whatever) and “overlaid” over the current values (just overwriting the fields it contains with the values specified; multiple instances would need to be dealt with carefully). Or treat any additionally specified field values in an XMP as adjustments to the blob that would also be provided (a “blob” would always be “complete” “by definition”). Or whatever other approach might be more generic and widely useful. Even when using individual fields, one has to be careful that their values or interpretation might change between versions. Or they might even be completely removed, so staying up to date with version changes would still be important.

If adjustments are to be provided in separate files, of course finding and handling those (matching them with the images/styles that need to be adjusted) is a separate topic that needs to be carefully designed. If everything in the same xmp, this is simpler and changes would only be needed in the saving and loading routines.

I guess with this article Timelapse photography and darktable I am kind of the orignator of that issue :slight_smile:

I also realized the blob should be untouched.

I once also got the idea, a little helper program could read the editabl/edited values and write a new blob., maybe the crawler could even initiate it, once he finds, the editable values are different from the blob

If I could code and would have the time, I would have started it already.

Would it make sense to update my FR #4250 accordingly or make a new one?