Darktable AI Edge Detection

Luminar NEO and Lightroom are both investing in AI-driven edge detection to mask areas precisely. Darktable v3.6 has the ability to mask based on focus detection. Have the developers contemplated adding AI-based edge detection algorithms to enhance masking?

I have ON1 and it has a lot of those features touted and they work and they don’t so you end up tweaking quite a bit …they have a new sky swap but honestly when I tweak parametric masks in DT and if need be restrict or protect with a drawn mask I can mask trees building etc pretty much as well and as fast as it can get done in ON1…I can’t speak to the two you mention as I don’t use them…

2 Likes

No. I don’t think anyone wants to add that.

1 Like

AI is just useful if properly trained - training isn’t a job for just a few developers spending some spare time

1 Like

NLP and object recognition require large sets of training files, but edge detection is by now an easy method to integrate. GIMP already has 5 methods for edge detection that require no training files.

then thats not AI :wink:

3 Likes

The new version of Lightroom separates faces with hair in one click and it is AMAZING.

Advances in software quickly become the norm. Most of us by now can run Lightroom or Luminar in docker containers at close to bare metal speeds on Linux. Darktable should not lag behind.

you might compare the size of adobes developer team and the darktable team - but of course you can join the team an do all that fancy stuff. :wink:

8 Likes

I think this is a classic case of people ‘expecting’ or ‘wanting’ software to go a certain way, but I have a feeling - because I can’t know for sure - that the developers simply don’t want to invest their time in that.

It’s open source, people code what they want to code. If the few DT devs don’t want to do it, they don’t want to do it. But maybe they do? It’s not like every coder in the planet is suddenly an AI-algorithm-expert.

If someone somewhere starts working on it and contributes, I’m sure they will not refuse it.

4 Likes

Darktable only lags in marketing. Parametric and drawn masks have made the selections I want to make relatively easy. The selections that I’ve made that took me a while aren’t going to be solved by AI.

Again, someone has to code the stuff, train the model, and keep it updated. Unless you’re going to do that, I don’t really see it happening.

10 Likes

And they probably had a 100 guys and a million(s) dollar budget…… just for that one feature :blush:

1 Like

Selecting faces is easy with Neural Net methods IF you are satisfied to find a bounding box.

Selecting a pixel perfect bound around the face is extremely difficult and requires both a ton of properly labeld training data (where exactly is the pixel boundry between a face and it’s surrounding?) and lots of expensive compuation power to really train the net…

4 Likes

I just saw a video someone made raving about the new adobe masking yet when they used select person, the person was holding a string of balloons and all those were selected…no person has that shape so I am not sure how “AI” it even is… I think like all these tools…some test images will give a perfect result and others will not…

…and then the problem (as with a lot of “user friendly” software) is there’s nowhere to turn when the “intelligence” is dumb. Perhaps they’ll provide a “mensa” version for extra money.

Maybe all the “experts” here should try the software in question before throwing up NIH vomit.

And you should not be so salty when you don’t get the answers you want.

7 Likes

Steven I don’t have the software and you do…so really I can’t comment from an informed position…But I have seen a few video’s for example this one

and this one

In the first video select subject selected a rock and some sky in addition to the person. Later the way the skin was masked was not particularly AI…but this might not be what you are talking about.

In the second video the presenter used the select subject on a black monkey that should have been easy to mask…it missed his rather large tail…selected the branch he was on and missed his hand…more over the presenter keeps saying how amazing it is and doesn’t even acknowledge a pretty obvious artifact…The sky that he chose to mask would have easily been done by any software with a luminosity mask…so he didn’t push it…

So its fair to say I don’t have the software and should not really comment. You do have it so you would be a better judge…but from many of the examples I see LR still has a lot of work to do and I have not seen a demonstration as clearly perfect as you mentioned. There seem to be a lot of artifacts that still need corrected and actually it makes me more impressed about how good the DT parametric masking is for a FOSS tool compared to a 305B dollar company.

Still I think your question has merits…are there areas that DT could improve masking and what would those be…given the software is written by people doing things that interest them as opposed to individuals paid to make a product for a consumer…what might those be…

The answer might come from a use case analysis rather than a comparison of what another software can do…so right now are there limitations to masking in DT that are holding people back or could be improved or made easier…some input from users towards that end might direct where any new improvements to DT’s masking might be focused going forward.

3 Likes

I encounter images where I know an edge must exist, but no local processing can detect it because the pixels from two objects have the same character. Using human intelligence looking at the area around the problem, I can see the photograph has two objects, and some areas show a clear edge, and for some pixels I can mentally assign them to one object or the other. By continuing clear edges into the problem area, I can choose an arbitrary line for an invented edge that may not be accurate but is credible.

So I can see that, in principle, AI could solve these cases. But I don’t have enough AI experience to design or implement this, or even estimate how much work by humans or computers might be needed, or how good the results would be.

That is an excellent point…here is the example from the video that the presenter raved about and you can see the subject is quite different from the background to that part should be easy but the the tree introduces a block to the algorithm when it processes the pixels and it misses the tail and the left hand of the monkey…and it includes the branch he is sitting on

1 Like