Darktable AI Edge Detection

I didn’t know where to put this comment, so I didn’t talk about it before.

My thinking goes: why not create the mask outside of dt and then import it. I am tired from work: can we import masks into dt? Takes time, but could be a stop-gap solution.

Not currently. There is raster mask facilities and you can use masks from other modules, so you’re not too far off from loading an external mask, I suppose.

People coming from commercial software value the export-to-other-apps-to-edit feature (or simply sharing rasters or vectors). If they could import and export stuff mid-workflow, that would be attractive to them. However, that requires knowledge, metadata, proper insertion and may break things or cause confusion if done willy-nilly.

I think the issue is if someone that is developing DT has the interest the time and the skill to implement this sort of thing I suspect it would happen. But we can’t commission people to include features we want in a FOSS product. We could of course make requests or suggestions but in the end this is why commercial software exists. We pay people to create or do things we cannot or do not have the time for. Its not the same with FOSS projects. Also many things that are suggested for DT ignore the way DT processes an image and the pixel pipeline and as such depending on the request its not often a simple matter of just adding something and having it work and or not break something…

So I don’t always think its necessarily a resistance to the concept or the request but rather an insistence that DT would be better if only x y or z was added… Its more about somehow " the insinuation that somebody should really get on this or the current thing is inadequate " etc etc …

The response most would likely give back is to say the code is all there have at it…

I think I have heard this before and it seems quite silly to me and I suppose its hyperbole. The camera makes a jpg for this reason because for sure to make anything that is much better from the raw file is usually going to take more than a min if for nothing more that thinking about what you might want to do with the image…

There is nothing stopping you from doing that except your own pocket book.

I was being generous and inaccurate… and even then the offer of money would still not be important to some who might work on a project for a variety of reasons and not want to be held to a deadline or outcome…but you are right… I could have said request although often its not requested even but more first a description of what is lacking and then what should be done or this is how product x does it why doesn’t DT … all flavors are there… luckily usually more than compensated for by grateful people very happy to benefit from the collective efforts of the team… myself included…

For edge detection, algorithms can do the job. AI is necessary to learn from User what is relevant. I expect Lightroom use the online data RAW —> JPG, the saved dev. steps from all of the customers to get a mega data base for training. Maybe they consider by (manual) selection / sponsorship only pro photographer.

I thought the following: Would it be feasible to implement a “lua iop module” that executes a lua script at its position in the pipe? The lua script would be called with the image and mask data at its position in the pipe, and would return image data as well. Optionally, it would be able to emit a raster mask as well. The module would not show up by default and only be available if some option in the preferences is set, to prevent its use by regular users. The purpose would be prototyping of iops rather than implementing something for production use, but could also be used for use cases that are needed by a very low number of users. In particular, exporting the current pipe state as image and reimporting an externally processed file could be easy that way, even at different positions in the pipe with two of these modules and respective scripts. An option to define if the module should be neglected in thumbnail rendering would probably be needed.

I know that the roi concept is something to deal with where I lack the knowledge which options are possible, maybe recent work on parallel pipes may ease this topic, but that’s for a later discussion anyway.

E.g., for me, it is feasible to write a bit of lua code if time permits (the last 9 years, at least one of my children was not autonomous enough to give me much time to work on such things, but these days I regain some free time), but my c knowledge is basically nonexistent, which makes it hard to prototype something (I tried but did not succeed). Such a module could help people to try out things with a much more gentle learning curve, and even the solutions that don’t make it into c code eventually may be helpful for some people.

What do you think, is this worth starting a feature request, or is it even possible already?

For me it’s the exact opposite. If I pay for a software such as lightroom, I still have almost no chance to get a feature request implemented unless thousands of people are crying for it. There’s no direct interaction with the devs. With f/l/oss I always have two options, i can implement it myself (or directly pay somebody to do it), or I can start a feature request. In that regard, darktable devs have been very generous, many of my feature requests made it into actual features, many of them back in the redmine era. That’s in particular one reason I am using f/l/oss software: The possibility that I can contribute one way or the other not only money but also ideas, and that there is a chance that they become reality. Lightroom would have never got a vertical waveform just because I had a need for it, just to make one example.

That’s IMO not true. First, when I shoot raw only, there is no jpeg. But even if it is there, the editing capabilities of the camera are very limited. When I shoot a soccer game, the light conditions do not change dramatically, and I can come up with a reasonable color edit in a couple of minutes (I even have a preset for the soccer photos as a starting point). This is then copied to all images of the match that I want to edit (culling comes before). Typically more than 50 pictures, as every player should be at least on a couple of photos. The edit per picture is then only cropping (sometimes very heavy cropping as 200 mm is my maximum focal length) and straightening (this in particular is not possible with the jpeg without massive quality loss), and maybe masking out some distraction in the background to make it a bit darker (e.g. sun reflection, but typically a very low number of images per session require masking, but sometimes I use it also for vignetting), plus some exposure, contrast and vibrance correction due to changing light or shooting direction (sun position relative to the shooting direction). All of this does not require much time, and I am thankful to have such a great tool for it :smile:. Using camera jpeg would be a no-go for me. I am doing this in my spare time to give something back to the team, as my son is playing in this team, and so far people like it. But typically I want to have the images out on Sunday if the match was on Saturday, so I cannot spend hours on a single photograph, getting one hour to edit the whole set of pictures is already luxury.

As I said, many of my feature requests made it into code and …

I am very grateful for this and I hope that this is clear. However, from your posts I get the impression that feature requests are in general a bad thing. I don’t think that this is what you intend, but that is what I read on my side. I think I understand which attitude you are arguing against, but we are not all native speakers, and sometimes things may sound more harsh than they are intended, also in feature requests.

Question to the experts: here, it is stated that a good mask is not good enough, but that a foreground detection is mandatory for reasonable matting result. How does this relate to the current gimp implementation of foreground selection?

I already looked once or twice inside the darktable code, and I wouldn’t say that it’s technically impossible to do.
But this comes with a lot of constraints and questions. If some peoples already find the current processing pipeline a bit slow, what would it become with a lua script in the middle ?
Let’s imagine a simple pipeline like: exposure → Lua iop → filmic
What would happen when you change the exposure step by step (like roll your mouse wheel). Do you expect the lua script to be called for each step ? if it’s the case, it could take a lot of time to process.
Another solution could be to only invoke the lua script on demand, once and have it cache the result. But that means you can ignore everything that comes before the module.

For the mask problem, it’s probably more simple to add a way to edit an existing mask from the mask manager. Because in that case, there is no update loop involved.

1 Like

Sorry I didn’t mean that at all I meant that you suck it up and pay for a commercial product that already does what you want say AI masks or denoising or what ever… for sure those projects/products are not nearly as responsive to feature requests and input unless it impacts the bottom line…

Not at all and my apology if it was conveyed as such… I think if it was it was in response to the use of the word should… and as you say that can be a language thing. I don’t think we have the right to tell developers of a project what they “should” do but rather could this be done or is it reasonably possible with the resources at hand… and you are correct again there could be what resembles push back at times but perhaps not quite at the level of hostile …

1 Like

I think that would be reasonable, those who are using this feature will know this limitation. Therefore I would hide it behind an option in the settings, such that one cannot stumble upon it by accident.

1 Like

That’d be. a UX disaster, and we already get hammered on UI/UX all the time (true or not).

Could you please explain why and where you think it would be a UX disaster? Do you mean the toggle in the settings (blender e.g. hides features by settings) or the module itself (it has in my imagination a file dialog button, a text field naming the file, a recompute button, and a toggle for consider in thumbs or not, and maybe also a roi vs full selector and maybe even a color format dropdown).

Btw., I don’t see any major UX disaster in darktable, I even like the oft-discussed collection filters very much, I was hoping for such a feature for years, but I also like the UX of blender, so it’s maybe my fault :wink:

^ Implementing it this way would be a UX disaster

I am a native speaker and I think your use of the word hostile is completely legitimate. It’s arguable, of course, depending on the judgement of the reader, but having read through this thread, the word hostile is within the bounds of reasonable interpretation of at least some of the commentary. Thanks

2 Likes

Not all image processing software is targeted towards the same population and the same needs. The same is of course true of most products. We would not expect, for instance, car manufacturers to slavishly emulate each and every feature that others do. We each select and make compromises with acquisitions according to our needs and availability.
Rather than asking the dt developers, with their meager resources, to add features … why not ask Adobe to upgrade LR to match the superior development of dt?
Maybe you should buy a copy of LR for the specific fast AI processing needs and simply use dt for quality processing.

Hi David. I think you make a good point on the limited resources and that may be argument enough. However, saying that choosing photo software is a matter of horses for courses doesn’t seem entirely realistic. Most people don’t often use multiple full-fat editing software packages, particularly, I suppose, pros who need to get the job done. I wonder (out loud) whether there isn’t a tendency, not so much from the developers but the users, that this is their niche and newcomers should do the hard work of figuring things out and not expect things to be too simple or straightforward or popular. Not directing this at you but some of what I’ve seen in this and other threads. It’s probably an unfair characterisation. Thanks

1 Like

Hello.

I just post a play raw with an image to see how users change the background using just luminosity masks. As I’m not a heavy user of darktable and don’t know the tools so well, I want to know how users separate the model from the background using luminosity masks. This is a easy one.

This is another image where I tried to select the model in darktable using luminosity masks but at the end I switched to GIMP to end the edition.

Basic edition in darktable

High end in GIMP

But I don´t changed the background color, and definitely not with AI :wink:

That’s my impression as well.
But I did not wanted to exclude the possibility up front :slight_smile:

I do think it would be a more interesting solution to be able to externally edit the masks