Not that well, you can see it in Adobe’s own demo video, or hit up the YouTube. Simple, obvious things work great, peoples’ faces, the sky, etc. But subtle things don’t work well at all.
1 Like
All the videos and examples I have seen on these automatic subject masking tools always seem to use examples with relatively uniform or simple backgrounds. I’ve played with some of ai tools for masking with a very noisy non uniform background and while they can mostly get th main subject isolated that’s the easy bit, I always ended up needed to manually tidy up all the complex edges of hair/fur/motion blur that the automatic tools can’t handle, they’re a long way from ‘intelligent’, but that’s marketing for you.
As humans we find it easy to isolate a subject from its surroundings and extrapolate/guess anything missing, our survival has relied on this and evolution has honed this ability over millennia. It’s likely going to take software a little while to catch up.
Need some AI sabre-tooth tigers
I don’t understand this, pls could you explain how.
nothing to explain in in addition to the reference:better try it yourself - the slider can be moved in two directions and depending on the setting the mask ist refined.
Use details mask to further refine the mask… I suspect is what it meant
/* How are “detail masks” implemented?
The detail masks (DM) are used by the dual demosaicer and as a further refinement step for
shape / parametric masks.
They contain threshold weighed values of pixel-wise local signal changes so they can be
understood as “areas with or without local detail”.
As the DM using algorithms (like dual demosaicing, sharpening …) are all pixel peeping we
want the “original data” from the sensor to calculate it.
(Calculating the mask from the modules roi might not detect such regions at all because of
scaling / rotating artifacts, some blurring earlier in the pipeline, color changes …)
In all cases the user interface is pretty simple, we just pass a threshold value, which
is in the range of -1.0 to 1.0 by an additional slider in the masks refinement section.
Positive values will select regions with lots of local detail, negatives select for flat areas.
(The dual demosaicer only wants positives as we always look for high frequency content.)
A threshold value of 0.0 means bypassing.
So the first important point is:
We make sure taking the input data for the DM right from the demosaicer for normal raws
or from rawprepare in case of monochromes. This means some additional housekeeping for the
pixelpipe.
If any mask in any module selects a threshold of != 0.0 we leave a flag in the pipe struct
telling a) we want a DM and b) we want it from either demosaic or from rawprepare.
If such a flag has not been previously set we will force a pipeline reprocessing.
gboolean dt_dev_write_rawdetail_mask(dt_dev_pixelpipe_iop_t *piece, float *const rgb, const dt_iop_roi_t *const roi_in, const int mode, const dt_aligned_pixel_t wb);
or it’s _cl equivalent write a preliminary mask holding signal-change values for every pixel.
These mask values are calculated as
a) get Y0 for every pixel
b) apply a scharr operator on it
This raw detail mask (RM) is not scaled but only cropped to the roi of the writing module (demosaic
or rawprepare).
The pipe gets roi copy of the writing module so we can later scale/distort the LM.
Calculating the RM is done for performance and lower mem pressure reasons, so we don’t have to
pass full data to the module. Also the RM can be used by other modules.
If a mask uses the details refinement step it takes the raw details mask RM and calculates an
intermediate mask (IM) which is still not scaled but has the roi of the writing module.
For every pixel we calculate the IM value via a sigmoid function with the threshold and RM as parameters.
At last the IM is slightly blurred to avoid hard transitions, as there still is no scaling we can use
a constant sigma. As the blur_9x9 is pretty fast both in openmp/cl code paths - much faster than dt
gaussians - it is used here.
Now we have an unscaled detail mask which requires to be transformed through the pipeline using
float *dt_dev_distort_detail_mask(const dt_dev_pixelpipe_t *pipe, float *src, const dt_iop_module_t *target_module)
returning a pointer to a distorted mask (DT) with same size as used in the module wanting the refinement.
This DM is finally used to refine the original mask.
All other refinements and parametric parameters are untouched.
Some additional comments:
- intentionally this details mask refinement has only been implemented for raws. Especially for compressed
inmages like jpegs or 8bit input the algo didn’t work as good because of input precision and compression artifacts. - In the gui the slider is above the rest of the refinemt sliders to emphasize that blurring & feathering use the
mask corrected by detail refinemnt. - Of course credit goes to Ingo @heckflosse from rt team for the original idea. (in the rt world this is knowb
as details mask) - Thanks to rawfiner for pointing out how to use Y0 and scharr for better maths.
1 Like
yes, I consider myself an advanced Darktable user, and as I said, have been using it for at least five years. I’ve used masking extensively in Darktable. Drawn masks. Parametric. Gradient. Ovals with/without feathering, etc. You name it! I shoot in a lot of low light environments and like to make edits that create drama – yet keep them realistic.
Darktable is amazing and powerful!! and for many use cases better than lightroom.
But I’m curious, have you used the current Lightroom (or Photoshop) with AI masking? It’s dramatically faster for 90% of masking use cases, IMO, at least my uses of masking of objects, people, skies. An advanced user saves time and either uses the AI mask or uses it as a faster starting point. So the mask doesn’t alwasy have to be perfect – just way faster.
So don’t let the hurdle of training a perfect AI prevent you from seeing that taking no action will eventually render Darktable antiquated.
AI masking would simply make Darktable better – and currently lightroom masking is now better than Darktable – which was probably not a true statement when this thread started, and partly why I posted above.
1 Like
That is one opinion. Putting in such dire terms seems overly dramatic and unenticing.
But masking is only half of it, what you can do with the mask in LR is still extremely limited compared to darktable.
I’m not a developer, and don’t know much about this really. But I recently tried the 7 day free trial of Lightroom, (just cos I could…) and while it works well (as you’d hope given the cost) the global options are way more limited than darktable, pushing more reliance onto masking.
For example, a sunset I processed in dt just needed a simple tweak in the tone EQ to bring out all the dynamic range in my camera. With Lr I had to resort to masking the sky (which was quick, but I wasn’t 100% happy with the result) to bring the highlights in. I’m obviously not an LR expert, but I thought this was relevant…
I haven’t bothered ever messing with LR. Did some photoshop back in the day but as a pure hobby or to work on images from the lab. I did buy ON1 a number of years ago when I was struggling with DT. I never really did use it that much but when the cheap upgrade options came up I did. I have 2022… likely won’t upgrade to 2023 as I said I don’t use it much… People complain about UI issues well ON1 introduced this AI denoise which for me always or generally produced artifacts with the default settings but the biggest thing was you could only compare it against the base raw image. So imagine some of the underexposed images we see in playraw and then hit original in the history stack and now imagine this is the reference image to assess the AI denoise and sharpening… in most cases its useless… what you have to do is take an image and edit it and them save that as a layer and then go back apply the AI denoise and put that in another layer and then toggle the layers to see what the difference is… Still it makes the adjustments hard to make as you can’t just change the setting strength etc without all that messing around… I don’t think they have fixed it at all in 2023. When I asked support they said because it works before demosaicing or as part of that then they can’t compare it to the current state of the image when you apply it…so you compare the full pipeline with the module enables vs the starting image equivalent in DT of original in the history stack… That’s how AI it is…
1 Like
Ha! Very intelligent… 
One thing I like in Lr is the function to stitch a pano from raws and and it makes a DNG which you can then treat like any other raw file. My computer crashed when I tried a big one - needs a massive amount of memory I suspect. Those DNGs can be opened in dt but the black levels are way off - all at 0 in fact. Doesn’t bother me as I’m very unlikely to proceed with Lr past the trial! Curious though.
I’ll have to have another go with hugin - IIRC one can do a similar sort of thing, albeit not as streamlined
Its a TIFF in a DNG container. Might as well just work with the TIFF.
1 Like
Ah. Right, that makes sense. What sort of TIFF can hold the whole dynamic range? Does it need to be a 32bit or anything like that?
It can be if you want. 16 bit is probably OK tho
1 Like
since darktable doesn’t support editable raster masks it’s not that easy to properly integrate AI even as a starting point for further refinement.
So to make use of AI there are 2 complex tasks to be solved:
- support for editable raster masks that can be used in combination with drawn and parametric masks
- the whole AI training shebang
So valid demand and of course soemthing that would boost darktables capabilities, but maybe hell freezes over earlier - maybe like windows support 
The original poster of this thread eventually threw up his hands with this sentiment before abandoning the thread 1.5yrs ago. I’m going to do the same.
At a high level I understand many of the roadblocks of bringing AI masks to Darktable. But I couldn’t more strongly advise any Darkroom developers to at least try Lightroom AI masks. That’s all I was getting at, and I think what @Steven_Adler was driving towards.
@MStraeten, You responded, but directly avoided my question: Have you used latest Lightroom masking with AI?
Agreed. Have you used masks in current version of Lightroom?
I love Darktable… masking is only a portion of why I use it, and then, only if I’m really editing. But that’s the point… masking is a key part of why Darktable editing has traditionally been more powerful than Lightroom – because you can target edits.
I’m out. Not interested in a flame war. Just felt a year and a half after this topic was posted, the original arguments against fell flat in light of actual use+results of new tools.
Anyone on linux can “try for free” by downloading free trial version of MS Windows and then into it, downloading trial version of Adobe Lightroom, etc. Also worth noting, Lightroom (and many other tools, such as Photoshop) are also web and app based, though sometimes in more limited form. (I didn’t realize that until I tried recently).
1 Like
there’s no “Lightroom masking with AI” module within darktable
so why the question?
it’s obvious that you can save a lot of time using AI masks - but you need to spend a lot of effort first to get that implemented. Is it so hard to understand, that a billion dollar company can do things in a year a couple of spare time developers can’t?
Which of those 4 VMs can run under linux?
[quote]
Anyone on linux can “try for free” by downloading free trial version of MS Windows and then into it,[/quote]
Note: I have only read a few posts. I don’t have LR.
Currently, LR appears to cover the following object detections:
- subject
- sky
- background
- objects
- people
It appears that you can detect each individually and then edit or overlay them.
That could be useful for speed. However, if the user knows what they want and to do, then the current implementation in dt is just fine.
Perhaps, following some tutorials would help! (Say for example @s7habo’s.)
If you look at the video you will notice that he does not zoom in to ensure the mask edge are correct. In my experience, these tools do an ok job if you intent to post on instagram or facebook. The google pixel does a decent job too. But, in my opinion, if you intent to print your photos to have it in a wall at a decent size, proper mask selection and mask edge definition matters. Maybe in a few year ai can get there.