highlight reconstruction issue

here is one image that i have use highlight reconstruction >color reconstruct, at 100% view there seemed to be a mosaic pattern that can be seen. if you compared to C1 there is no mosaic. is there a method to eliminate the mosaic when trying to recover highlights.

this one is by capture One

this is processed in darktable 3.4.1.1 in windows

2 Likes

You might try highlight reconstruction in lch or color or tweak settings in filmic highlight reconstruction tab

i have tried it but the mosaic pattern is still there. I just tried it again with ART and there is no mosaicing. even in filmic highlight reconstruction tab the mosaic is still there.
below image is from art

I see the mosaicing in the blue part of the sky, are you sure it’s due to the highlight reconstruction? My first thought was the demosaicing used, try one of the other options?

yes I have tried PPG, Amaze and VNG all will have the mosaic pattern.
here are my settings

i just tried it on ubuntu its the same
i can provide the raw and xmp file if you would like to try

Please do.

i f that doesn’t help you might also tweak the raw white point. but without the raw it’s just poking around in the fog

here is the RAW file. I have notice that this issue is not only on this file. if the file is clip in one or two channels even trying to recover highlights the mosaic pattern will show up. it is especially noticeable when you zoom in to 100%. In RT and ART this mosaic pattern does not show up.
_5290185.orf (15.5 MB) _5290185.orf.xmp (6.3 KB)

I just played with this one and darktable is rather bad at handling it to be honest (RawTherapee doesn’t seem to have problems with it, as you mentioned).

You might want to give the raw denoise module a try. If I turn it on and up the noise threshold slider to > 0.19 the pattern seems to be gone (tried both dt 3.4.1 and 3.5.0).

But you still need to adjust try and adjust using filmic and highlight reconstruction a lot to get a half decent result. Then again, most of the sky is blown (=lost).

EDIT: Even the new demosaic options in 3.5.0 don’t solve this.

I have given it a quick go in 3.4.1.1 on Windows with my normal scene referred workflow (filmic etc) so haven’t looked at the orig XMP

Just opened the image, dropped the exposure a bit and set ‘highlight recon’ to ‘reconstruct in Lch’ and ‘demosaic’ to ‘Amaze(slow)’ and I see no mosaic pattern.

What I see an issue with (but haven’t had time to work on it) is the blown highlight on the right shoulder.
That may take a bit of work to get rid. Would be interested in how this can be got rid of if anyone has a go

For info, a similar problem was raised in this thread: Highlight recovery artifacts

Sorry, this discussion is old and it has answers, the highlights reconstruction module is broken and darktable’s devs don’t want to fix this.

lch reconstruction
https://github.com/darktable-org/darktable/issues/2086

Using Lab/LCh before that correction is done is conceptually wrong.
Won’t fix, we can’t change the algo without losing compatibility. Checkout filmic reconstruction.

LCH

Filmic reconstruction (it just blur the highlights and doesn’t recontruction at all)

Color reconstruction
https://github.com/darktable-org/darktable/issues/5509

This is a limitation of the algorithm used, not a software problem. There is nothing we can fix in the program that doesn’t require to rewrite the maths first and use a different algorithm.

I’ve posted somewhere in the forum a comparison of raw editors, darktable doesn’t render correctly raw colors (neither arwtherapee), in my comparison it adds green bias in the shadows, however looks at the strange pinkish skin and gray hair.

I suggest you to decode the raw in Photoflow, it renders correctly the raw, the highlights reconstruction works great and it support dcp profiles .

Photflow

1 Like

Thanks @Jade_NL for the verification and possible solution. I really want to make darktable my primary raw editor. So I have been testing it with some of the most difficult shot through it.

Initially I thought so too. If you zoom in at different magnification the pattern will show up. But the most important is when you export it. It will show up in your exported files

@age yeah my findings is the same as yours. Your results from photoflow is as good as capture one. It’s sad that it won’t be fixed. I remember I read somewhere in the forum that one of the developer said that the goal of darktable was to produce raw results that rival capture one. Highlight reconstruction is a super important in my imho. Thanks again for the recommend will check photoflow out

1 Like

I could get some texture back in the sky with filmic reconstruction, by carefully adjusting the threshold and contrast parameters. Keep in mind that those depend heavily on the white relative exposure in filmic, and also on the colour preservation mode. Changing the threshold value by as little as 0.1-0.2 can (should) have a visible effect on the result.

I didn’t manage to get the blue in the sky, but then, was it a blue sky with some clouds, or cloudy (and Capture One faked the blue)? The blue and red channels seem to be clipped, so there’s really no colour information left in the sky

It was blue sky with clouds

Have you used filmic? The reason is that filmic doesn’t preserve the hue in the highlights , it removes all the saturation instead, even the per-channel rgb suffers from this weird behaviour because some under the hood “gamut compression”.

I’ve opened an issue on github with a request that works great for a lot of images.

I’ve used this method for the photoflow developed RAW ,and the blend highlights reconstruction (not available in darktable)

2 Likes

Try as much as I can to love and use filmic. It takes too many modules and step to reach what I want to achieve. I find that using base curves as a starting point and then tonal equalizer to pull or push the highlights or shadow. Anyway from my understanding the tonal equalizer will happens before the base curves in the pipeline which means information should not be clipped. I am not expecting miracles. But I do hope that areas in highlights that is not fully clipped should be reconstructed properly. I want to applaud the darktable devs that has produced such an amazing app. A class leading highlight reconstruction and moiré canceling would put darktable to the top. IMHO. Filmic has its place and is great tool.

i think the mosaic pattern can remove i have added a surface blur with a parametric mask to the highlight area for the display referred workflow and astrophoto denoise for the scene referred workflow attach is the xmp file can someone check to see if this works.
_5290185.orf.xmp (7.6 KB) _5290185_01.orf.xmp (6.7 KB)

1 Like

It’s possible to have good reconstruction in darktable. Just tested quickly your photo in darktable master (as I remember, no improvement on highlights reconstruction since darktable 3.4. See:

The XMP is inside JPG. Just use reconstruct highlight in LCH mode and default setting. Then I adjust with filmic white and black exposure (could be easily changed/improved with those sliders).

The main thing on my result photo I see to improve is the highlight in girl knee. Should be fixed by using a mask.

2 Likes