Direct reading of an XMP file

[grubernd]

@paperdigits

When your rhetoric is pedantic, you get pedantic rhetoric in return.
[…]
Instead of rehashing the same tired and condescnedinly

Pedantic and condenscending? Made. My. Day.
If I am anything, then it is highly opionated. I know that doesn’t sit well with a lot of folks.

Anyway, I’m not here to pick a fight … even if Mica’s well oiled forum machinery sometimes takes on the appearence of a helicopter parent. @paperdigits if you ever happen to be in Austria near Vienna or Graz - drinks are on me.

Back to topic.

I think @Bordwall 's approach can work really well if the exports and associations of the hashed tool identifiers can be fully automated. Which - if feasable - might be an interesting route to take. But @hannoschwalm already pointed out the biggest problem: if not automatable, you’d have to follow every. single. change. in darktable tools with utmost precision or that tool will fall apart really fast.

For one-offs and personal hacking it is probably the best and most simple option there is. Kudos for showing a quick proof-of-concept.

Don’t get me wrong, I totally get the internal simplicities of dealing with struct data directly. Skip dealing with lot’s of extra code to maintain or alternatively all the translations, introspections and reflections - I heard a rumour C/C++ isn’t really good at that. My C/C++ foo is a bit rusty and never was too advanced to begin with.

Going the lib route would very likely be the most elegant way, but very likely also the most work - you’d have to bring all the xmp and tool handling into a single place.

I wonder if something reasonable could be done through the Lua interface. Sadly my time is rather limited taking on such an endeavour.

[priort (Todd Prior)]

Lately when I am curious about a question I just check one of the AI chats…

THis came up…might be old outdated or useless??

Yes, it is possible to convert a darktable XMP file to an XML file. Here are the steps to do so:

1. Open a command prompt or terminal window.
2. Navigate to the directory where the XMP file is located.
3. Type the following command and press Enter: python dtstyle_to_xmp.py <filename>.xmp
4. The script will generate an XML file with the same name as the XMP file in the same directory.

Please note that this script is provided by the darktable team and is not officially supported by Microsoft or Bing. Also, please ensure that you have Python installed on your system before running this script.

I hope this helps!

[kofa (István Kovács)]

@priort

Yes, it is possible to convert a darktable XMP file to an XML file.

I think you misunderstood something. The script converts a .dtstyle file to .xmp. The .xmp files, as well as .dtstyle files, are in XML format already; the trouble is that the parameters are not encoded as ‘human readable’ values (like <exposureAdjustment>+1.33</exposureAdjustment> – fictional example), but rather as <op_params>0100000000000000080000000000803f000000000000803f0000803f0000803f0000803f00000000000080be000080be000080be000080be00000000</op_params> (actual content from a .dtstyle file). While it is ‘readable’, it is not ‘understandable’ or ‘interpretable’.

[paperdigits (Mica)]

@grubernd

Anyway, I’m not here to pick a fight … even if Mica’s well oiled forum machinery sometimes takes on the appearence of a helicopter parent. @paperdigits if you ever happen to be in Austria near Vienna or Graz - drinks are on me.

Do you really think I’d meet for anything after you’ve just been insulting?

@grubernd

If I am anything, then it is highly opionated. I know that doesn’t sit well with a lot of folks.

Opinions are fine, but not when they’re stated as you often do. I’ve realized this is a problem of self awareness, I’ll drop it now.

[priort (Todd Prior)]

No I understood. I was just on my phone and couldn’t check it out… I thought it might decode to plain text from the description the bot gave… clearly it doesn’t. PS Starting at “Yes”. was the chat bot…

[priort (Todd Prior)]

Looks like someone took an initial stab at it some time ago…

GitHub - wmakeev/darkroom-xmp-tools: Helps to modify operations params from Darktable xmp files

[MStraeten (MartinSt)]

Or be inspired by Files · master · Jochen Keil / dtlapse · GitLab
and Annoucement of dtLapse

[davidvj (David Vincent-Jones)]

Yes, it is possible to convert a darktable XMP file to an XML file. Here are the steps to do so:

Open a command prompt or terminal window.
Navigate to the directory where the XMP file is located.
Type the following command and press Enter: python dtstyle_to_xmp.py <filename>.xmp
The script will generate an XML file with the same name as the XMP file in the same directory.

[david@david-W530 ~] cd /home/david/Desktop [david@david-W530 Desktop] python dtstyle_to_xmp.py 20150213_0097.RAF.xmp
python: can’t open file ‘/home/david/Desktop/dtstyle_to_xmp.py’: [Errno 2] No such file or directory

Did anybody else try this?

[paperdigits (Mica)]

@davidvj

Did anybody else try this?

No, its some chatGPT nonsense.

[priort (Todd Prior)]

I didn’t…

I found it here if you need to add it

https://github.com/darktable-org/darktable/blob/master/tools/dtstyle_to_xmp.py

[paperdigits (Mica)]

Ah my bad.

[priort (Todd Prior)]

It might still be non-sense … I don’t really have the skill to look at the script and see what it does… but it does exist…

[kofa (István Kovács)]

OK, just to satisfy your curiosity, I ran one of my styles through this script. Here are the dtstyle and the corresponding XML (renamed to .xml.txt, because that is what the forum supports).
000-filmic standard.dtstyle (3.4 KB)
000-filmic standard.xml.txt (2.4 KB)

Plugin params (for a single plugin) from the style:

<plugin><num>22</num><module>5</module><operation>colorbalancergb</operation><op_params>gz04eJxjYCAVNNij0ghw9oyPHVB8H1TeDpvuiW9t7JBpRiAGAIEVCY8=</op_params><enabled>1</enabled><blendop_params>gz10eJxjYGBgYAFiCQYYOOHEgAZY0QVwggZ7CB6pfOygYtaVAyCMi08IAAB/xiOk</blendop_params><blendop_version>12</blendop_version><multi_priority>0</multi_priority><multi_name></multi_name><multi_name_hand_edited>0</multi_name_hand_edited></plugin>

Indented for ‘readability’:

<plugin>
	<num>22</num>
	<module>5</module>
	<operation>colorbalancergb</operation>
	<op_params>gz04eJxjYCAVNNij0ghw9oyPHVB8H1TeDpvuiW9t7JBpRiAGAIEVCY8=</op_params>
	<enabled>1</enabled>
	<blendop_params>gz10eJxjYGBgYAFiCQYYOOHEgAZY0QVwggZ7CB6pfOygYtaVAyCMi08IAAB/xiOk</blendop_params>
	<blendop_version>12</blendop_version>
	<multi_priority>0</multi_priority>
	<multi_name></multi_name>
	<multi_name_hand_edited>0</multi_name_hand_edited>
</plugin>

Plugin params from the XMP (nothing to indent, it’s one XML tag):

<rdf:li
    darktable:blendop_params="gz10eJxjYGBgYAFiCQYYOOHEgAZY0QVwggZ7CB6pfOygYtaVAyCMi08IAAB/xiOk"
    darktable:blendop_version="12"
    darktable:enabled="1"
    darktable:modversion="5"
    darktable:multi_name=""
    darktable:multi_priority="0"
    darktable:operation="colorbalancergb"
    darktable:params="gz04eJxjYCAVNNij0ghw9oyPHVB8H1TeDpvuiW9t7JBpRiAGAIEVCY8="
/>

Both are XMP, and the parameters, as it has been pointed out, are binary encoded. The only difference is that the style uses XML tags (such as <module>5</module>), while the converted version uses attributes (darktable:modversion="5").

[rvietor]

While the change from tags to attributes is perfectly acceptable (in this case, as each tag should only appear at most once), I’m not so sure about the two tags that have disappeared without a trace: <num> and <multi_name_hand_edited>…

Nitpicking perhaps, but that omission could cause problems with the converted file (and drops information)

[genesius (Genesius Jaromsky)]

Putting my analytical two cents in.

1. I saved the default sidecar file to a new name in a separate folder.
2. Made a change to the Exposure module.
3. Changed the name of this sidecard and moved to the same separate folder.
4. Compared the two sidecar files in WinMerge.

Original
darktable:params=“00000000000080b9 3333 333f00004842000080c001000000”
Decimal: 13107
Setting: 0.700 EV

Change
darktable:params=“00000000000080b9 c074 333f00004842000080c001000000”
Decimal: 49268
Setting: 0.701 EV

Increasing the exposure EV by only 1/1000th EV the value in the parameters blob changed by a decimal value 36161.

???

I don’t know if this helps anyone.

Thanks and God bless,
Genesius

[forestbeasts]

Looks like in this case it’s not an integer – it’s a float!

You’re working with the raw binary data here, and while integers are pretty simple, floats are more complicated. But it turns out 0.7 converted into a float is 0x3333333f:

perl -e 'print unpack("H*", pack("f", 0.7))'
(gives 3333333f)

And going the other way, with c074333f

perl -e 'print unpack("f", pack("H*", "c074333f"))'

gives you 0.701. Bingo.

But then you’ve got to figure out what it all means for every single parameter… for every single module you’re interested in. Sheesh.

Thankfully the original code is available, e.g. in the file @Bordwall linked to upthread. So it’s “just” digging through the code, not reverse engineering! But still a Bit Of An Undertaking, for sure.

(I’m here 'cause I’m trying to get exact values in the color zones module. There doesn’t seem to be ANY way to do it in the UI.)

– Frost

[hanatos (jo)]

you guys know houz’ introspection code, right? darktable/tools/introspection at master · darktable-org/darktable · GitHub