Something WRONG with darktable 4.1.0~git272.5a1a1845-1 clipped highlights

Check AP’s latest video explaining LPHLR. It’s long but in the last 10 min he generates this editing the image and makes a comment that it can happen with the gamut mapping when you generate certain invalid values…

As an end user who is not an expert in mathematics or physics I want my editing software to behave in a certain way like if I crank up the exposure (call it brightness, brilliance luma etc.) to brighten the picture, may be at some point the whole picture becomes white but I will say something is wrong when a tiny spot becomes a giant white blob.

I request the developer and experts to analyse the problem and solve it in such a manner that the complexity is hidden from the end user.

Thanks

Can the OP or someone else open a github issue, attaching the examples?
Otherwise nothing will happen I guess

That might be where things went off the rails.

Those terms each have a specific meaning within darktable, and the modules (mostly) use the terms in that meaning.
So you should not use brilliance to increase the overall lightness of the image, that’s the job of the exposure module (unless, of course, you have a specific reason to do that). That the “perceptual brilliance” sliders seem to have the same effect does not mean they act the same way.

The sequence described in the manual (exposure, filmic, color balance rgb) uses those modules in that order for a good reason. That doesn’t mean that you should always follow that recipe blindly, but it’s a good starting point.

A bit like using other tools: you can force a screw in the wood with a hammer (most of the time), but the result will not be quite what you expect (and probably wanted)…

I can reproduce this behavior and have written a bug report:

If anyone wants to participate they are welcome to do so.

8 Likes

I totally agree with this. An average user dont know much about color science. I have told my photographer friends to use opensource software, especially darktable, but both of them said this cant be used as professional software. One reason is learning new software and they dont have time for this (darktable is not a easy one) and second was ugly colors that they were unable to fix. If i remeber correctly problems were with fixing highlights. To make highlights visually “pleasing” as lightroom can do with one slider, was too much difficult for them as they had to use multiple modules and have knowledge how to use them.

A “professional photographer” is not necessarily a better photographer, it’s just a person who get paid for his/her pictures.
In most cases a professional photographer needs to develop hundreds or thousands of pictures in the shortest possible time, with decent quality, for pure economical reasons.
For this purpose the “one slider” approach of LR is the best: fast and decent quality. Not excellent.

Instead, DT is for enthusiasts, people who practice photography for pure personal pleasure, who don’t mind to study a bit and take some time to get beyond the “good enough” quality, aiming to excellence.
DT is almost on par with the image processing techniques used by the cinema industry, who is the only one having the budget needed for hiring true image processing experts.
Photography, no, that is much behind, as there is simply too little money in it.

2 Likes

Arrélien Pierre is going to be happy reading that :stuck_out_tongue: Afaik, he is a professional photographer…

The average professional will not work at the ‘excellence’ level, if only because the market at the price levels associated with excellence is very small. Then again, how many professionals dealing with hundreds or thousands of images use raw? (starting frm jpegs, a lot of the advantages of dt are lost already)

The need for retraining and the perceived lack of support are probably more important reasons to stay away from dt. Whether those reasons are valid? No idea.

Another case where the blob appears
increase the brilliance to the point where white blob appears, just start decreasing to the point where the blob disappears, now open diffuse or sharpen module, select fast sharpen and try increasing the iterations by one and the blob will appear again, in my case it was around 3.

Also in some cases just zooming in will make the blobs to appear

Once again : you are pushing areas which are already clipped in raw data higher and higher using several modules. Several way to avoid this have been suggested (masking clipped areas, using curve modules instead of filmic, setting filmics parameters to avoid it’s reconstruction…) And the reason for the actual behaviour (artefacts) has been explained in the github issue mentioned/opened by @s7habo.

Driving a car with 200 km/h against a wall will cause artefacts on the car. :wink:

2 Likes

Perhaps a quote from Aurélien Pierre in this issue report helps explaining what the actual problem is:

I see what you did. The brilliance boost for highlights is far too high, which means you are making white “whiter”. Since we are working in a perceptual space, white cannot be brightened without voiding all the assumptions needed by the color space. I suspect you are shifting the design use case of color balance by using it for contrast, but it can only be used to make colors paler or darker. Brilliance boost up to +50% or so are fine in highlights, but more will create problems.

(bold added)

It looks like this is exactly what @Rajkhand is doing: using brilliance to increase contrast. There are better tools for that, which do not cause those problems, because they can be used within design parameters.

And that has nothing to do with understanding the science behind the tools, but everything with understanding what a tool is supposed to do as described in the manual.

Don’t be surprised a car doesn’t work very well if you want to drive it all the time in reverse speed…

1 Like

for me my blobs disappear when I disable the highlight reconstruct module

Which algo?

went back to my edits that I had this issue with and it seems to be gone. Not sure if it was an opencl or caching issue, but when it happened it didn’t matter what algo i used it only changed the shape and color of the orbs. Just to clarify I can reproduce the orbs by pushing the colorbalance rgb brilliance to far , but this was a separate issue which was fixed by disabling highlight recon (seems to be gone on a fresh start, but I will keep an eye out for it so if it happens again I can record it)

Sure, but nevertheless we have devices that try to mitigate the “artifacts” for people that insist having crashes: airbags, seat belts etc.
So why not trying to implement a “graceful degradation” in the image instead of these horrendous blobs?

Rajkhand does not use this devices (for example parametric mask) in the xmp he posted, and this is the problem.

maybe a simple ui change that turns the slider red after it reaches a certain level past white.

it’s not darktables way to restrict control just because someone might misuse stuff.
If you’re overdoing something you get a visual response - such blotches, halos, hue shifts, artifacts. Then it’s up to you to find the proper settings - but these depends on the image you’re dealing with, not with some generic rules.

1 Like

I agree with this philosophy, as I said before DT is for enthusiastic learners, not for slider pushers.
However here I see a gui design flaw.
Try to sort of force-disable filmic reconstruction by setting threshold to 6EV and transition to 0.25EV.
Then you can push brilliance as high as you want and you will get degradation yes, flat blown highlights yes, but nothing surprising.
So the problem for me here is that we can’t really disable filmic reconstruction from the gui. We can push its intervention threshold high, but in reality it is alway active and ready to kick-in in the sneakiest and unexpected way.
To me we should have a true switch-off widget for filmic reconstruction and off should be its default value.
Maybe the code would even be a little more efficient when reconstruction is off.

your scenario sounds like you want to overdo it → then darktable will do what you want → overshoot so you can complain :wink: