highlights very troubled

dafrasaga · October 28, 2020, 5:39pm

Hi All,
I have a lot of issues to make highlights decents…

The RAW and the PP3
_DSC4330_1.NEF (25.2 MB) _DSC4330_1.NEF.pp3 (13.3 KB)

When I apply the neutral profile ( as usually) I have pink highlights

Then I set highlights reconstruction with Color propagation ( the best method about Rawpedia)

but I have a lot over exposed

then I make some highlights compression, but artefacts come up

What would you do to those highlights?
Is there nothing to do because Raw histogram is clipped ?

Cheers
Gabriele

RichardRegal · October 28, 2020, 6:21pm

With something this clipped you might want to go for luminance recovery as the highlight reconstruction mode. Usually it just gives a grey or white solid area that is not particularly pretty. However, in this case where what you have is a cloud that might not be too bad.

ChasingShadows · October 28, 2020, 6:26pm

I agree with Richard, however, I started with setting the exposure to -0.67 … here is my result _DSC4330_1.NEF.pp3 (14.2 KB)

CarVac · October 28, 2020, 7:31pm

The hard artifacting is because the raw whitepoint is not quite correct; the real white point is a bit lower (as seen from that green channel spike). Try adjusting the raw white point on the raw tab.

dafrasaga · October 28, 2020, 8:07pm

Hi CarVac,
if I understood I moved the Raw White point down but the image is more darker but the artefacts are there …

Thanatomanic · October 28, 2020, 8:38pm

@dafrasaga To tackle this problem properly it would be very helpful if you could do the following steps and provide the raw overexposed shots: Adding Support for New Raw Formats - RawPedia

I could analyse them and make sure the pink highlights will be gone in the future (i.e. with the next RT update, or when using development builds).

CarVac · October 28, 2020, 8:40pm

It only needs to be decreased a tiny bit to benefit.

That said, there are limits to the recoverability of highlights. It’s just not made clear since there aren’t intermediate histograms except at tone curves.

dafrasaga · October 29, 2020, 7:05am

Hi @Thanatomanic ,
many thanks for your time I’ll shot the images as claimed in Rawpedia but one question

Can you direct or explain me what is raw scaling for (I googled but I didn’t find a clear explanation for men as me )?

Cheers
Gabriele

Thanatomanic · October 29, 2020, 7:17am

I am not very familiar with the details. The only explanation I know is here: RawTherapee/camconst.json at dev · Beep6581/RawTherapee · GitHub
Somewhere in the paragraph titled ‘How to Measure White Levels’. Hope it helps, and thanks for providing the shots!

dafrasaga · October 29, 2020, 7:33am

Thanks, I didn’t read it (camconst.json) ever…but …
what does it mean if my camera ( Nikon D300) isn’t there??
How RT can decode my raw if D300 isn’t there?

Thanatomanic · October 29, 2020, 7:42am

Decoding a raw file is a totally different beast. The NEF format for the D300 is well-supported, so RT can decode the image data correctly. However, it needs a little more guidance to display that data properly. That’s where camconst.json comes in.

dafrasaga · October 29, 2020, 7:49am

Thanks… Than We’ll try to give more guidance for D300 decoding… When I have time ( soon) I’ll send the images

Reggybe · October 29, 2020, 11:06am

Hey,

The Nikon D300 is supported for many years. There is no need for changes for this camera.

Find attached a pp3 file with the elements that tackle your problem.
_DSC4330_1.NEF.pp3 (14.1 KB)

Here a screenshot of the result

dafrasaga · October 30, 2020, 8:31am

Hi Reginald,
your PP3 solves magenta but not completely for color propagation… however good for now

Thanks

dafrasaga · October 30, 2020, 8:39am

Hi @Thanatomanic,
I made some photos as here but there is something I do not understand.
When I shoot for LENR=yes I enable the option in the menu but I have no LongExposureNoiseReduction tag in exif… I have only Noise Reduction which is OFF for ever even if I enable and disable the option… However I upload the images to Filebin | ukggmx9z2smbttw7 and I’m here for other informations.

Thanatomanic · October 30, 2020, 11:05am

@dafrasaga Thank you for uploading. I don’t know why the LENR didn’t work. I also don’t see any usual differences between the longer exposure shots and the short exposure shots, so I think it is safe to assume that no LENR has been applied.

Some of your lower ISO shots were not fully overexposed. I also see that the G-channels seem to be slightly non-unform, but nothing to worry about. More troubling is that after I enter the white level values obtained from my analysis, I still get a purple look in your original image. I need to look into this a bit further…

It might help if you could redo the shots at lower ISO and use an even brighter scene. In particular I would like shots of ISO 200, 250, 320 and 400. If possible, could you addtionally supply these at intermediate f-values of f/4.0 and f/2.8?

Daniel_Catalina · October 30, 2020, 11:47am

The fact that they are not in the exif should not worry you, as not all cameras write the tag in exif.
But a hint here: when the camera performs LENR, it will take 2 exposures. So if you hear your shutter go on twice that all is fine.
Also, most cameras will do LENR only if your exposure time is bigger than a specific value. So have a look in the manual to check for what values you should set.

dafrasaga · October 30, 2020, 3:08pm

Hi @Thanatomanic,
I redoed the shots here… I hope I made as you stated

Cheers

dafrasaga · October 31, 2020, 7:39am

Hi Daniel,
Yes, the exposure time have to be bigger than 8s…

Thanks for the tip

Thanatomanic · October 31, 2020, 9:35am

Hi @dafrasaga the additional shots were helpful, thank you! I have now also seen the ‘aperture scaling’ effect in action for the first time

I am a little stumped though, because even though I think I did my analysis right, and even with the additional samples, I still cannot get rid of the purple highlight in your original image. Somehow the whitelevels are still off…

But there are still a few tricks we can pull for which I hope you can oblige. Could you also provide the following shots:

Fully overexposed, f/11, ISO200, no LENR, please three shots like these for averaging (these are basically the conditions in which you shot your original image)
Fully overexposed, all intermediate f-stops from the lowest you have available up until f/6.3 at ISO200, no LENR (for more information on aperature scaling)
Same as above, but for ISO2000 (from this ISO onwards the camera scales the raw differently)
Shot with lens cap on (so completely dark), aperture irrelevant, focus irrelevant, but please two identical shots for all available ISOs.

I know I’m asking a lot, but I kinda like these analyses and it bugs me that the problem isn’t resolved yet

FWIW, this is my current change for camconst.json if you want to try it out yourself:

    { // Quality B, samples provided by dasafraga (pixls.us), intermediate aperture scaling not available
        "make_model": "Nikon D300",
        "dcraw_matrix": [ 9000, -1966, -711, -7030, 14976, 2185, -2354, 2959, 7990 ], // DNG_v13 (Neutral v4)
        //"dcraw_matrix": [ 9030, -1992, -716, -8465, 16302, 2256, -2689, 3217, 8068 ], // DNG_v13 (Neutral)
        "ranges": {
            "white": [ 
                { "iso": [ 200, 250, 320, 400, 500, 640, 800, 1000, 1250, 1600 ], "levels": [ 16383, 15292, 16383, 15317 ] }, // R, G1, B, G2 - R G typically saturate, G1 G2 differ slightly
                { "iso": [ 2000, 2500, 3200 ], "levels": [ 16383, 16323, 16383, 16347 ] }
            ],
            "aperture_scaling": [
                { "aperture": 2.8, "scale_factor": 1.025 },
                { "aperture": 4.0, "scale_factor": 1.011 }
            ]
        }
    },