Blender AgX in darktable (proof of concept)

I haven’t had the issue either.

Dave G.

2 Likes

So I reverted back to commit eba1876 from master and unfortunately I have the same issue. I am now pretty sure it has to do with the latest msys2 update. The saved build based on commit eba1876 that I created prior to updating msys2 works just fine.

Bottom line: This issue has nothing to do with the agx module and if you are building with msys2 on windows be aware that updating msys2 might cause slowliness with darktable in general (not just agx branch).

2 Likes

my last macOS arm64 build here in this thread
darktable-5.2.0+59~g8cc82d3a08_arm64.dmg

next builds in current OSX Build will contain the agx-pull request even it’s not merged …
then also in an x86_64 variant.

3 Likes

@kofa, It’s been a while without any post in this forum. Just to keep the conversation going :grinning:, how’s the PR developing? Anything to feel elated about and the converse?

So far, only the very first round of the review has been done: formatting, coding style. No deep look yet. I’ve addressed most of those comments. Now we have an unrelated build error that will be fixed soon.

My promise to avoid breaking changes may not hold, as the team is somewhat reluctant to ship the module already in a version 2 (which I introduced towards the end).
I think we can still save your edits: if I remove the conversion logic and just change the version number back to 1, you can issue some SQL to make the database think that you edited with this ‘new’ version 1. But let’s see if the migration and v2 are allowed to remain.

Once the more detailed review comments start coming in, I’ll have to return to the code.

For the primaries params, I had an heuristic optimisation process when I created ‘my’ primaries adjustments. I’m now revisiting that experiment, to tune for relative error instead of absolute (having an error of 0.0001 for all matrix components may not be as useful, when one of them has a value of 0.89, the other 0.013). I’m now testing what happens if I measure the error by the ratio of the target and actual coefficient rather than by the difference. If that gets us closer to Blender, I’ll send a pair of demo pictures like above.

7 Likes

No improvement (the difference between ‘my’ previous settings and the new ones was less than what a 16-bit sRGB PNG can represent, so we did not come closer to Blender) – but you can always tweak the params if you want.

Do you all have access to Python?

I’m asking because I’ve found a way that would allow me to change the parameters without always (I actually only want to do this once) providing C migration code, so we could ship with version 1 and less code, but still avoid breaking your edits.

I’m not planning new params, but it seems it would make sense to unify certain parameters that use 0…1, and others that use 0%…100%. The structure wouldn’t change, but some values would have to be multiplied by 100.

I’ve fed the C struct that holds the data, as well as the C code that reads/writes the DB into Gemini, and asked it to create a Python script that can decode the agx parameters from the DB. That works, so if I update the code to use percentages, I can have Gemini extend the script to multiply the affected parameters by 100, re-encode and write the parameters into the sqlite DB. I will test this, of course.

Would that be an acceptable path? Since the purpose is only to update those fields, it would be a one-time procedure that would look like:

$ python3 /tmp/update-agx-to-percents.py /path/to/darktable-agx-config-dir/data.db

@Pascal_Obry: would this be an acceptable route? Also, would this break anything? There’s the table history_hash. Would it be safe to keep it?
An alternative would be to update XMP sidecars the same way (I also have code to read that), and turn on ‘look for updated XMP sidecars’, and have the DB safely managed by darktable.

In case anyone is interested, these are the scripts to dump agx params from an XMP and from data.db. They look safe to me (no write/delete operations, except for writing to the console), but run them at your own risk.

decode-agx.py (5.5 KB)
decode-agx-sqlite.py (4.1 KB)

2 Likes

@kofa, well if the migration path is important as it was proposed here I have no problem to change my mind and accept to keep the legacy params migration in the first Agx version.

2 Likes

AgX was (and here still is) marked as “proof of concept”.
For me, that means, there are no guarantees of compatiblity or even continuation of the module,
let alone inclusion in “official” darktable.

So anyone using AgX for production work should know they were taking a risk…

I’d say, keep the included code as simple as possible, and if there’s a script to convert the old edits to current AgX, some are lucky…

6 Likes

OK, thank you, then we don’t need the Python hacks. :slight_smile:
Do you agree with standardising on using percentages?

I think the following would make sense:

  • Look:
    • offset: since black and white display target are set in %
    • slope (maybe): it’s basically a multipler, so a plain number is OK, but maybe a slope of 120% looks better than 1.2. Contrast-like values are in plain decimal multipliers in sigmoid and filmic, but in color balance rgb contrast is a 0-based percentage (a modification), and so is gain on the 4 ways tab.
    • power: keep as a number
    • saturation: as percentage
    • preserve hue: as percentage
  • Curve params:
    • basic:
      • pivot x shift, pivot y linear: as percentage (ratio of the distance to shift the x coordinate towards the left/right edge). Also, probably better names? ‘pivot input brightness shift’ / ‘pivot input shift’, ‘pivot output brightness’ / ‘pivot output’?
      • contrast around the pivot: a plain number; keep, as that’s how filmic and sigmoid use it
      • toe/shoulder power: keep
    • advanced: already uses % values, except for gamma; gamma is just a plain number, keep
  • Primaries: already uses % values for insets, degrees for rotations

Sure, but I also use it as my daily driver (so I gain experience not only with test images :slight_smile: ), and would like to keep my edits – so I’m being somewhat selfish here. :stuck_out_tongue:

2 Likes

Please keep implementation discussion in GitHub’s PR.

I would keep slope as number.

For the pivot names:

  • input brightness pivot shift
  • output brightness pivot shift
    ?

A bit long though…

2 Likes

I also wanted to get the opinion of current users. I’ll keep the rest on the PR.

1 Like

I switched to AgX as my production version. But I have already written that I take the risk, that edits will break. So I fully support @rvietor with his statement.

4 Likes

I’m emotionless regarding numbers or percentage.
Coming to the names of the curve params, I think it’s useful that the pivot is linked to the curve, so I would keep the x and y in the names.

I have been running Agx with its own config dir. If I wished to switch to primary use would just running the appimage without a unique configdir pick up the old DT database etc? Would “Agx” tags added when I use Agl also be added to the DT 5.2 database (XMP files are written on the fly). Apologies if this is a numpty question but I would like to add my old images / tags to the Agx version without breaking that db etc etc.

A new build:

  • rebased on top of darktable master, includes commits up to and including b67d715c7;
  • I’ve moved the primaries from preset combo + apply button above base primaries, as the loaded values apply to the base, as well
  • some parameters are now percentages, as mentioned above; migration is provided by the new version.

Most of the other changes should be invisible; I’ve addressed review comments and did a bit of clean-up.

@MStraeten , @priort , @Dave22152 , please update your builds.

Since version hashes (current: 5cba1e41) may change because of rebasing, this is tagged as agx-2025-07-06-16-43-09
(Just in case you need to report issues, or we make comparisons. g5cba1e41 is visible on the window, based on that, I should be able to find the tag and the exact version again, if needed.)

Linux AppImage: https://tech.kovacs-telekes.org/dt-agx/Darktable-5.3.0+37~g5cba1e410c-x86_64.AppImage

I’ve also updated the WIP documentation page, maybe half of the controls is done: dtdocs/content/module-reference/processing-modules/agx.md at agx · kofa73/dtdocs · GitHub

5 Likes

Yes.

I’m not sure what tags you mean. The database structure did not change since 5.2, as far as I know. My branch only adds a new module. Image history for older edits will not be changed in any way. Edits using agx will be recorded in the DB as normal. Darktable will record the usage of the module just like that of any other; you will be able to filter on it if you wish:

agx parameters will also be recorded in XMP sidecars, and included in the EXIF metadata if you normally include editing history on export.

AgX build 5.3.0+37 for Win 11:

https://www.dropbox.com/scl/fi/1eo8p6npwqtr6dluodikc/darktable-5.3.0-37-g5cba1e410c-dirty-win64.exe?rlkey=qj9q2372puaxwhkiyo6aj3u24&st=phfyekem&dl=0

4 Likes

My error. I had manually added an AGX tag to images that I had used it on. Those ended up in the limited AGX specific DB. I had failed to realise that I could utilise Collections/ Module/ AGX to find them again. Thank you for clarifying that for me. Also a very big thank you for all the work that you have put in. DT is in a better place for that.

1 Like