highlight reconstruction issue

Please do.

i f that doesn’t help you might also tweak the raw white point. but without the raw it’s just poking around in the fog

here is the RAW file. I have notice that this issue is not only on this file. if the file is clip in one or two channels even trying to recover highlights the mosaic pattern will show up. it is especially noticeable when you zoom in to 100%. In RT and ART this mosaic pattern does not show up.
_5290185.orf (15.5 MB) _5290185.orf.xmp (6.3 KB)

I just played with this one and darktable is rather bad at handling it to be honest (RawTherapee doesn’t seem to have problems with it, as you mentioned).

You might want to give the raw denoise module a try. If I turn it on and up the noise threshold slider to > 0.19 the pattern seems to be gone (tried both dt 3.4.1 and 3.5.0).

But you still need to adjust try and adjust using filmic and highlight reconstruction a lot to get a half decent result. Then again, most of the sky is blown (=lost).

EDIT: Even the new demosaic options in 3.5.0 don’t solve this.

I have given it a quick go in 3.4.1.1 on Windows with my normal scene referred workflow (filmic etc) so haven’t looked at the orig XMP

Just opened the image, dropped the exposure a bit and set ‘highlight recon’ to ‘reconstruct in Lch’ and ‘demosaic’ to ‘Amaze(slow)’ and I see no mosaic pattern.

What I see an issue with (but haven’t had time to work on it) is the blown highlight on the right shoulder.
That may take a bit of work to get rid. Would be interested in how this can be got rid of if anyone has a go

For info, a similar problem was raised in this thread: Highlight recovery artifacts

Sorry, this discussion is old and it has answers, the highlights reconstruction module is broken and darktable’s devs don’t want to fix this.

lch reconstruction
https://github.com/darktable-org/darktable/issues/2086

Using Lab/LCh before that correction is done is conceptually wrong.
Won’t fix, we can’t change the algo without losing compatibility. Checkout filmic reconstruction.

LCH

Filmic reconstruction (it just blur the highlights and doesn’t recontruction at all)

Color reconstruction
https://github.com/darktable-org/darktable/issues/5509

This is a limitation of the algorithm used, not a software problem. There is nothing we can fix in the program that doesn’t require to rewrite the maths first and use a different algorithm.

I’ve posted somewhere in the forum a comparison of raw editors, darktable doesn’t render correctly raw colors (neither arwtherapee), in my comparison it adds green bias in the shadows, however looks at the strange pinkish skin and gray hair.

I suggest you to decode the raw in Photoflow, it renders correctly the raw, the highlights reconstruction works great and it support dcp profiles .

Photflow

1 Like

Thanks @Jade_NL for the verification and possible solution. I really want to make darktable my primary raw editor. So I have been testing it with some of the most difficult shot through it.

Initially I thought so too. If you zoom in at different magnification the pattern will show up. But the most important is when you export it. It will show up in your exported files

@age yeah my findings is the same as yours. Your results from photoflow is as good as capture one. It’s sad that it won’t be fixed. I remember I read somewhere in the forum that one of the developer said that the goal of darktable was to produce raw results that rival capture one. Highlight reconstruction is a super important in my imho. Thanks again for the recommend will check photoflow out

1 Like

I could get some texture back in the sky with filmic reconstruction, by carefully adjusting the threshold and contrast parameters. Keep in mind that those depend heavily on the white relative exposure in filmic, and also on the colour preservation mode. Changing the threshold value by as little as 0.1-0.2 can (should) have a visible effect on the result.

I didn’t manage to get the blue in the sky, but then, was it a blue sky with some clouds, or cloudy (and Capture One faked the blue)? The blue and red channels seem to be clipped, so there’s really no colour information left in the sky

It was blue sky with clouds

Have you used filmic? The reason is that filmic doesn’t preserve the hue in the highlights , it removes all the saturation instead, even the per-channel rgb suffers from this weird behaviour because some under the hood “gamut compression”.

I’ve opened an issue on github with a request that works great for a lot of images.

I’ve used this method for the photoflow developed RAW ,and the blend highlights reconstruction (not available in darktable)

2 Likes

Try as much as I can to love and use filmic. It takes too many modules and step to reach what I want to achieve. I find that using base curves as a starting point and then tonal equalizer to pull or push the highlights or shadow. Anyway from my understanding the tonal equalizer will happens before the base curves in the pipeline which means information should not be clipped. I am not expecting miracles. But I do hope that areas in highlights that is not fully clipped should be reconstructed properly. I want to applaud the darktable devs that has produced such an amazing app. A class leading highlight reconstruction and moiré canceling would put darktable to the top. IMHO. Filmic has its place and is great tool.

i think the mosaic pattern can remove i have added a surface blur with a parametric mask to the highlight area for the display referred workflow and astrophoto denoise for the scene referred workflow attach is the xmp file can someone check to see if this works.
_5290185.orf.xmp (7.6 KB) _5290185_01.orf.xmp (6.7 KB)

1 Like

It’s possible to have good reconstruction in darktable. Just tested quickly your photo in darktable master (as I remember, no improvement on highlights reconstruction since darktable 3.4. See:

The XMP is inside JPG. Just use reconstruct highlight in LCH mode and default setting. Then I adjust with filmic white and black exposure (could be easily changed/improved with those sliders).

The main thing on my result photo I see to improve is the highlight in girl knee. Should be fixed by using a mask.

2 Likes

Can you describe what you did? Not everybody has dt git master. I think you used the color calibration module to make the sky look more blue. Because if you only use highlights reconstruction lch you just make the sky grey, but here it is blue. I tried to open the xmp but dt 3.4 does not do this properly.

The shoulder highlight is unnaturally blue/gray and more noticeable than the knee IMHO.

2 Likes

I don’t know about the current iteration of filmic but I recall lights and darks being desaturated because they “aren’t supposed to be” saturated.

3 Likes

_5290185.orf.xmp (8.5 KB)

darktable 3.4.0

4 Likes

i tried it. and yes the color calibration module made it more blue but its a monochromatic blue instead of blue and white.

yes, those clip areas do not have nice transition. i have tried both reconstruct LCH and reconstruct color sometimes LCH is better and other Reconstruct color is better. but reconstruct colors will always have a mosaic pattern what ever you do.

yes, that’s how filmicrgb works, I think. but there are times when you don’t want that

@wpferguson most of the images that i pass through filmic seem to lose contrast. It would take another module such as the color balance module to bring back the contrast. and the recovered sky seem to be much grayer. I feel the best showed here is the one by photoflow. It looks the most natural and the contrast is abotu right. I understand everyone will have a different taste here but the photoflow version seem to be the best to me. even better than capture one and its clean