I don’t want to trigger another anti-AI rant here by those people who 100% oppose the use of AI in editing or Darktable. I request people to avoid turning this thread into that and just ignore my post if they are anti-AI.
This is a genuine technical question about if it would ever be possible to use a local AI model on Darktable to remove reflections from glass. AI is proving valuable for creating masks and in some instances helping with denoising.
Here is a not very important or interesting shot taken of a terracotta warrior in Xian, China. The statue is placed behind glass and reflections could not be avoided on this occasion since I didn’t have a polarizing filter with me.
So just for the heck of it I ask Lumo AI to remove the reflections. It did a reasonable job but smoothed the image too much for my liking. It also was incapable of leaving the English text on the sign unaltered.
There are occasions like this when I would like to see Darktable offer a local AI model for retouching and restoring. I am not wanting to create fake images from scratch or add stuff that was not there but rather solve issues like this reflection. Maybe the developers who are currently working with AI could comment if this is something that may one day be practical in Darktable. This is not a feature request.
I’m wondering if such couldn’t be just trained into a application-specific neural net? I’m messing with doing such to take images of brick or stone walls and puke out a black-n-white image where the bricks/stones are white and the interstitial mortar is black. Goes into another program I wrote to make textured walls. Pytorch would be used to train such, and OpenCV::dnn would run them.
That might be a palatable implementation for dt, the run-them side. A ‘zoo’ of image modifier networks would be useful, and no more objectionable than matrix metering in your camera.
That was a mistake. I have loaded a smaller one now. Thanks.
I am using this as a possible example of when traditional editing can not fix a problem in DT and we may need to resort to a local AI model. I just don’t know if with the current state of play if this is practical for a local model. Hopefully a knowledgeable person like @Masterpiga or @anry can tell me if this might one day be possible or not. It is not a feature request but I could see its usefulness.
Prompted generative models can certainly do that. Current darktable policy is to not include such models in its arsenal, but you can already use them via external tools and Lua integration, e.g., Krita or my own IOPaint integration (which currently is broken, but I am planning to fix it eventually).
It depends on the dynamic range and reflectivity. Normally the target image is there underneath the reflection, if an algorithm could figure out exactly how much blue glow to subtract. Ideally, AI would figure out the parameters (of a retouch module?) rather than writing a fresh image. But when the only tool you have is an LLM, everything starts to look like a prompt.
Thanks for the replies so far. I plan to look at what Krita can do. I have never explored Krita. I used to use and teach GIMP a lot when I was shooting JPG images, but now I shoot RAW and the masking system of Darktable means I rarely need GIMP anymore.
This isn’t a high priority of me but this image highlighted a rare case scenario for AI intervention in my workflow. The actual image is not that interesting or important but I don’t see normal editing tools handling it. I am not that keen on AI image generation in general. The example I posted shows how AI hallucinated with the English Text and tried to make everything as smooth as a Barbie Doll.
This strikes me as the same problem as removing a local tint or fringe. Because you don’t want to remove it everywhere, only where the color exists relative to the neighboring pixels. And it’s not necessarily the same blue everywhere, but it appears the same blue relative to the neighboring pixels. I guess the operation could be done with various modules, but we have no masking capability to select colors in local color space (for example, a pixel is green if it’s more green than its neighbors, rather than if it’s 00ff00).
