Something WRONG with darktable 4.1.0~git272.5a1a1845-1 clipped highlights

Perhaps a quote from Aurélien Pierre in this issue report helps explaining what the actual problem is:

I see what you did. The brilliance boost for highlights is far too high, which means you are making white “whiter”. Since we are working in a perceptual space, white cannot be brightened without voiding all the assumptions needed by the color space. I suspect you are shifting the design use case of color balance by using it for contrast, but it can only be used to make colors paler or darker. Brilliance boost up to +50% or so are fine in highlights, but more will create problems.

(bold added)

It looks like this is exactly what @Rajkhand is doing: using brilliance to increase contrast. There are better tools for that, which do not cause those problems, because they can be used within design parameters.

And that has nothing to do with understanding the science behind the tools, but everything with understanding what a tool is supposed to do as described in the manual.

Don’t be surprised a car doesn’t work very well if you want to drive it all the time in reverse speed…

1 Like

for me my blobs disappear when I disable the highlight reconstruct module

Which algo?

went back to my edits that I had this issue with and it seems to be gone. Not sure if it was an opencl or caching issue, but when it happened it didn’t matter what algo i used it only changed the shape and color of the orbs. Just to clarify I can reproduce the orbs by pushing the colorbalance rgb brilliance to far , but this was a separate issue which was fixed by disabling highlight recon (seems to be gone on a fresh start, but I will keep an eye out for it so if it happens again I can record it)

Sure, but nevertheless we have devices that try to mitigate the “artifacts” for people that insist having crashes: airbags, seat belts etc.
So why not trying to implement a “graceful degradation” in the image instead of these horrendous blobs?

Rajkhand does not use this devices (for example parametric mask) in the xmp he posted, and this is the problem.

maybe a simple ui change that turns the slider red after it reaches a certain level past white.

it’s not darktables way to restrict control just because someone might misuse stuff.
If you’re overdoing something you get a visual response - such blotches, halos, hue shifts, artifacts. Then it’s up to you to find the proper settings - but these depends on the image you’re dealing with, not with some generic rules.

1 Like

I agree with this philosophy, as I said before DT is for enthusiastic learners, not for slider pushers.
However here I see a gui design flaw.
Try to sort of force-disable filmic reconstruction by setting threshold to 6EV and transition to 0.25EV.
Then you can push brilliance as high as you want and you will get degradation yes, flat blown highlights yes, but nothing surprising.
So the problem for me here is that we can’t really disable filmic reconstruction from the gui. We can push its intervention threshold high, but in reality it is alway active and ready to kick-in in the sneakiest and unexpected way.
To me we should have a true switch-off widget for filmic reconstruction and off should be its default value.
Maybe the code would even be a little more efficient when reconstruction is off.

your scenario sounds like you want to overdo it → then darktable will do what you want → overshoot so you can complain :wink:

installed a new build and the highlight blobls popped back up on images that I didn’t any edits on. All algos accept clip highlights. If I turn off highlight module the blobs disappear and if I turn off colorbalancergb the blobs also disappear, but I could not get this behavior to reappear with the last build. the build I’m currently




the over exposed pics are with filmic turned off (it shows the warping in the area with the blob)
the one with the big blob is with filmic turn on and the one with no blob , filmic turned on and highlight module turned off.

On those pics guided lap does not cause the blob but on other pics it does. I can get rid of the blobs on all pics by dropping highlights but like I said this issues was hard to reproduce on the previous build.

Im using the new build by wpferguson darktable windows insider program 9/18

edit: If I turn off the high light recon module and every thing looks good , but if I zoom in on the picture other blobs start appearing. I also have another pic that shows the the blob with the module on but if I zoom in it disappears; the same pic will be inconsistent on showing the blob and how high I can raise the highlights before it reappears. I keep thinking opencl or caching because of the inconsistencies

Just curious what happens if you zoom to 100 % and or export…are the artifacts present or altered??

This is why I asked… I noted that using the image from the OP. The displays are not showing the same image and it can change with zoom in the center or main preview while not in the preview/zoom nav preview in the upper left that you use to pan. I used the image with only a small set of adjustments. I started by changing the raw white point to the value stated in the meta data…ie around 15383 I think. Okay so now filmic only at all v6 defaults and with exposure using auto exposure ( I think it calculated and added about 1 EV again by default) and then only activating highlight brilliance slider in the rgb CB module… At 67% no artifact…1 more percent creates an artifact. Things that I noticed impacting this were 1) lowering the filmic transition slider a small amount removes it as does 2) setting structure to 100 percent or bumping highlight saturation to 15% in this case while changing nothing else… so I think some of these have been mentioned but for me and this follows another artifact that I mentioned in a recent post that was dramatically affected by zoom level…it seems to me the previews are not getting the same data consistently and this can impact your evaluation if you are not keeping that in mind… note two things how the display of the artifact in the image changes in the center display and either jumps or disappears with transition images appearing momentarily that seem to more reflect the consistent artifact and also more in line with how it gets displayed up in the zoom navigation preview area (not sure of the name for that one…)

EDIT and what’s even more interesting is you export this image look what you get compared to the preview…

As far as I understand AP’s latest video about highlights reconstruction the result of guided laplacians is dependent on resolution (and filmics hl reconstruction uses guided laplacians). We have different resolutions in the pixelpipes for the small navigation preview in darkroom, the preview in darkroom and export at full size. Maybe that is the explanation?

Yea the orbs show up on export , even if I turn off the the highlight recon module. they appear when zooming in on the image , which they did not in the previous build. I wonder if I should install the previous build to retest this issue.

edit: install the last build and the problem is still there, I will install the official release to test.

edit 2: tested the official build and still happening. this is a problem that I have had before and all but went away. The only thing that really changed was installing a new build.
I will go ahead and adjust the pics that need adjusting, Will update if I figure out anything new

just a quick note, if you change the saturation method from darktable ucs to the old one you
can go highlight crazy. I can crank the highlight slider to a 100 percent and no orbs.
(this was not my issue though because I’ve been converted over to ucs)

The last insider build is 4.1.0+398~g94db38396, which means there have been 226 commits since the version that’s having a problem. Perhaps try an up to date version and see if the problem still exists.

I should have included that my comments above were using a build that was current that day so only a couple of days old…not the one mentioned in the post

1 Like

This orb issue was already reported in github, but it still needs resolution. Is this a different issue, if yes, then start a new issue in github. If not, then we need to wait for resolution.

I believe that darktable only calculates the region of interest (ROI) when you are zoomed in, so filters that need all of the image data (i.e. segmentation based highlight reconstruction) don’t necessarily show the true state of the image.