Something WRONG with darktable 4.1.0~git272.5a1a1845-1 clipped highlights

I totally agree with this. An average user dont know much about color science. I have told my photographer friends to use opensource software, especially darktable, but both of them said this cant be used as professional software. One reason is learning new software and they dont have time for this (darktable is not a easy one) and second was ugly colors that they were unable to fix. If i remeber correctly problems were with fixing highlights. To make highlights visually “pleasing” as lightroom can do with one slider, was too much difficult for them as they had to use multiple modules and have knowledge how to use them.

A “professional photographer” is not necessarily a better photographer, it’s just a person who get paid for his/her pictures.
In most cases a professional photographer needs to develop hundreds or thousands of pictures in the shortest possible time, with decent quality, for pure economical reasons.
For this purpose the “one slider” approach of LR is the best: fast and decent quality. Not excellent.

Instead, DT is for enthusiasts, people who practice photography for pure personal pleasure, who don’t mind to study a bit and take some time to get beyond the “good enough” quality, aiming to excellence.
DT is almost on par with the image processing techniques used by the cinema industry, who is the only one having the budget needed for hiring true image processing experts.
Photography, no, that is much behind, as there is simply too little money in it.

2 Likes

Arrélien Pierre is going to be happy reading that :stuck_out_tongue: Afaik, he is a professional photographer…

The average professional will not work at the ‘excellence’ level, if only because the market at the price levels associated with excellence is very small. Then again, how many professionals dealing with hundreds or thousands of images use raw? (starting frm jpegs, a lot of the advantages of dt are lost already)

The need for retraining and the perceived lack of support are probably more important reasons to stay away from dt. Whether those reasons are valid? No idea.

Another case where the blob appears
increase the brilliance to the point where white blob appears, just start decreasing to the point where the blob disappears, now open diffuse or sharpen module, select fast sharpen and try increasing the iterations by one and the blob will appear again, in my case it was around 3.

Also in some cases just zooming in will make the blobs to appear

Once again : you are pushing areas which are already clipped in raw data higher and higher using several modules. Several way to avoid this have been suggested (masking clipped areas, using curve modules instead of filmic, setting filmics parameters to avoid it’s reconstruction…) And the reason for the actual behaviour (artefacts) has been explained in the github issue mentioned/opened by @s7habo.

Driving a car with 200 km/h against a wall will cause artefacts on the car. :wink:

2 Likes

Perhaps a quote from Aurélien Pierre in this issue report helps explaining what the actual problem is:

I see what you did. The brilliance boost for highlights is far too high, which means you are making white “whiter”. Since we are working in a perceptual space, white cannot be brightened without voiding all the assumptions needed by the color space. I suspect you are shifting the design use case of color balance by using it for contrast, but it can only be used to make colors paler or darker. Brilliance boost up to +50% or so are fine in highlights, but more will create problems.

(bold added)

It looks like this is exactly what @Rajkhand is doing: using brilliance to increase contrast. There are better tools for that, which do not cause those problems, because they can be used within design parameters.

And that has nothing to do with understanding the science behind the tools, but everything with understanding what a tool is supposed to do as described in the manual.

Don’t be surprised a car doesn’t work very well if you want to drive it all the time in reverse speed…

1 Like

for me my blobs disappear when I disable the highlight reconstruct module

Which algo?

went back to my edits that I had this issue with and it seems to be gone. Not sure if it was an opencl or caching issue, but when it happened it didn’t matter what algo i used it only changed the shape and color of the orbs. Just to clarify I can reproduce the orbs by pushing the colorbalance rgb brilliance to far , but this was a separate issue which was fixed by disabling highlight recon (seems to be gone on a fresh start, but I will keep an eye out for it so if it happens again I can record it)

Sure, but nevertheless we have devices that try to mitigate the “artifacts” for people that insist having crashes: airbags, seat belts etc.
So why not trying to implement a “graceful degradation” in the image instead of these horrendous blobs?

Rajkhand does not use this devices (for example parametric mask) in the xmp he posted, and this is the problem.

maybe a simple ui change that turns the slider red after it reaches a certain level past white.

it’s not darktables way to restrict control just because someone might misuse stuff.
If you’re overdoing something you get a visual response - such blotches, halos, hue shifts, artifacts. Then it’s up to you to find the proper settings - but these depends on the image you’re dealing with, not with some generic rules.

1 Like

I agree with this philosophy, as I said before DT is for enthusiastic learners, not for slider pushers.
However here I see a gui design flaw.
Try to sort of force-disable filmic reconstruction by setting threshold to 6EV and transition to 0.25EV.
Then you can push brilliance as high as you want and you will get degradation yes, flat blown highlights yes, but nothing surprising.
So the problem for me here is that we can’t really disable filmic reconstruction from the gui. We can push its intervention threshold high, but in reality it is alway active and ready to kick-in in the sneakiest and unexpected way.
To me we should have a true switch-off widget for filmic reconstruction and off should be its default value.
Maybe the code would even be a little more efficient when reconstruction is off.

your scenario sounds like you want to overdo it → then darktable will do what you want → overshoot so you can complain :wink:

installed a new build and the highlight blobls popped back up on images that I didn’t any edits on. All algos accept clip highlights. If I turn off highlight module the blobs disappear and if I turn off colorbalancergb the blobs also disappear, but I could not get this behavior to reappear with the last build. the build I’m currently




the over exposed pics are with filmic turned off (it shows the warping in the area with the blob)
the one with the big blob is with filmic turn on and the one with no blob , filmic turned on and highlight module turned off.

On those pics guided lap does not cause the blob but on other pics it does. I can get rid of the blobs on all pics by dropping highlights but like I said this issues was hard to reproduce on the previous build.

Im using the new build by wpferguson darktable windows insider program 9/18

edit: If I turn off the high light recon module and every thing looks good , but if I zoom in on the picture other blobs start appearing. I also have another pic that shows the the blob with the module on but if I zoom in it disappears; the same pic will be inconsistent on showing the blob and how high I can raise the highlights before it reappears. I keep thinking opencl or caching because of the inconsistencies

Just curious what happens if you zoom to 100 % and or export…are the artifacts present or altered??

This is why I asked… I noted that using the image from the OP. The displays are not showing the same image and it can change with zoom in the center or main preview while not in the preview/zoom nav preview in the upper left that you use to pan. I used the image with only a small set of adjustments. I started by changing the raw white point to the value stated in the meta data…ie around 15383 I think. Okay so now filmic only at all v6 defaults and with exposure using auto exposure ( I think it calculated and added about 1 EV again by default) and then only activating highlight brilliance slider in the rgb CB module… At 67% no artifact…1 more percent creates an artifact. Things that I noticed impacting this were 1) lowering the filmic transition slider a small amount removes it as does 2) setting structure to 100 percent or bumping highlight saturation to 15% in this case while changing nothing else… so I think some of these have been mentioned but for me and this follows another artifact that I mentioned in a recent post that was dramatically affected by zoom level…it seems to me the previews are not getting the same data consistently and this can impact your evaluation if you are not keeping that in mind… note two things how the display of the artifact in the image changes in the center display and either jumps or disappears with transition images appearing momentarily that seem to more reflect the consistent artifact and also more in line with how it gets displayed up in the zoom navigation preview area (not sure of the name for that one…)

EDIT and what’s even more interesting is you export this image look what you get compared to the preview…

As far as I understand AP’s latest video about highlights reconstruction the result of guided laplacians is dependent on resolution (and filmics hl reconstruction uses guided laplacians). We have different resolutions in the pixelpipes for the small navigation preview in darkroom, the preview in darkroom and export at full size. Maybe that is the explanation?

Yea the orbs show up on export , even if I turn off the the highlight recon module. they appear when zooming in on the image , which they did not in the previous build. I wonder if I should install the previous build to retest this issue.

edit: install the last build and the problem is still there, I will install the official release to test.

edit 2: tested the official build and still happening. this is a problem that I have had before and all but went away. The only thing that really changed was installing a new build.
I will go ahead and adjust the pics that need adjusting, Will update if I figure out anything new