Very Different Initial Rendering of RAW File ??? [possible corrupt file but renders ok in RT]

Perhaps Fix Corrupted Nikon NEF Images

2 Likes

Due to noise I assume.

@kmilos or @LebedevRI, any clues about this and why darktable brightens the image?

Edit: Reading the thread at Exiftool right now Nikon D800 NEFs ==> SubIFD Tags corrupted ==> Rewrite possible?

1 Like

I just tried that little application (cool btw, custom built for a similar issue) but it returns an error that the camera is not one of the models it supports. And it is a different issue - I think. But a step in the right direction!

@dnlyko it sounds like Phil Harvey over on this forum has successfully sorted similar issues in the past. Maybe post a question over there? https://exiftool.org/forum/index.php

Difference between a raw file from raw.pixls.us and the one above. Strip offsets and Rows Per Strip.

1 Like

I don’t know what happened to this file.
I have been playing with it.

700_5910.NEF.xmp (33.6 KB)

I only decided to move my workflow from Nikon to DT yesterday.
I started to import and discovered this issue.
I have continued to import, after I posted the issue, and it is happening on about 20% of my files.
Each time it happens I try to edit the file in all my other apps and they all open fine.
I can only reproduce this issue in DT.

I check with geeqie and other viewers and they are showing issues for me too. Do you have that image without Capture NX edits ? (Straight from camera?)

As it has been said, nobody should ever use any program
that modifies the original raw files for any reason whatsoever,
including “harmless” sidecar embedding.

That being said, in this case, you happen to be in luck.

6 Likes

That was a quick fix! Just to be clear, does this mean that if I download the next nightly Windows build it should handle the file that the OP posted?

I’m still not 100% clear on some Github aspects


At least not until it’s propagated into darktable,
since darktable does not (should not) track rawspeed’s develop branch.

1 Like

I see, thanks. I was mostly thinking of the OP - how he should proceed considering the issue with the modified NEFs. I presume the thing to do is wait till it propagates? :grinning: Or build oneself I guess
 something I have never tried


I have confirmed that it is indeed Nikon Capture NX2 that is altering the file
(It no doubt also applies to Nikon Capture NX as well. I could do a test if required. I do have the original CDrom but I would have to install it on an old 32-bit Windows machine)

I took a picture on my D700 today
I insert CF Card and copy the .NEF file to c: drive

File size is 25,331KB
I open it in Nikon Capture NX2
I adjust the exposure
I save/close file/app
File .NEF file size is now 32,668KB
No side car file or folder created
Nikon wrote the data into the .NEF file itself
I Import .NEF file into DT and it does not render properly
I open the .NEF file in all my other apps and it opens/renders properly
I delete the file off my c: drive

I re-copy same original file from CF Card to my c: drive
I open it in Nikon Capture NX-D
I adjust the exposure
I save/close file/app
File size remains the same
Side car folder created
Side car file .nksc created
I import file into DT and is A-OK
I delete the file off my c: drive

I re-copy same original file from CF Card to my c: drive
I open it in Nikon Studio NX
I adjust the exposure
I save/close file/app
File size remains the same
Side car folder created
Side car file .nksc created
I import file into DT and is A-OK
I delete the file off my c: drive

There is an easy solution. Dont open the .nef files in any of the Nikon softwares and just import them into dt.

Absolutely. I have faithfully used Nikon’s software trusting what they stated that my edits are non-destructive. Which it is. It’s just that Nikon added the new data to the .NEF file, instead of a side care file. All the original .NEF data is still intact within the .NEF file. My problem is that I have used Capture NX and then Capture NX2 on my original NEF files. This is a known issue that has been addressed by all the other software apps I use. Now that I had made my mind up to go with DT, it’s rearing it’s ugly head. I don’t write code but I would think if other open source apps have addressed this issue, it would take little effort to include the extra code in DT.

The code changes were pushed to the development branch. They should make it for the next dt release 4.4. Release 4.4 is scheduled for summer 2023.

1 Like

By the time you wrote this, the fix had already been created, as described here:

My apologies, I did not comprehend that that had already taken place.
That is absolutely wonderful! Looking forward to it’s release.
Thank you all for your responses and guidance. I am very grateful.
Truly a wonderful DT community. :ok_hand:

1 Like
3 Likes

A little late to the party, but I thought there might be some insight to be gained by stepping through the processing with rawproc.

First, libraw read the file just fine. Next, there’s no black point compensation numbers in the file, I think because Nikon was doing that in-camera to the raw data in this generation; my D7000 exhibits this.

Being a low-exposure image I first added a loggamma curve, then a control point curve to manually shape the tone. This exposed a distinct blue cast in the darkest regions, which is usually associated with the sensor dark current for which black subtract compensates. So, I went back to the subtract tool and dialed up a value until the blue went away.

I used a wavelet denoise to smooth the image out somewhat, then downsized to rendition format to take care of the rest:

1 Like