NEFs "has unknown format"

Oleastre · May 20, 2020, 10:37pm

RawTherapee and Darktable uses 2 different raw file reading libraries. That’s the difference.
A file being supported in one of the software does not imply it’s supported in the other one, but it could since there is another open-source implementation.

elGordo · May 20, 2020, 10:51pm

(If you still have the D3100) what happens if you take new photos with it and try to import them into darktable?

Soren_Garup · May 20, 2020, 10:55pm

The d3100 was my backup camera, and I sold it 5 years ago, but now I’m left with 32GB of imagenes…

But than for the tip

Soren_Garup · May 20, 2020, 10:59pm

Yes at least I can always process my 32GB of D3100 NEFs in rawTherapee, but I’d have to make a new workflow and in my experience Darktable has better watermark support, so…

But would it be wrong to conclude that it is not my NEFs that are broken, but Darktable?

Soren_Garup · May 20, 2020, 11:04pm

Thanks a lot

SO it is NOT my NEFs that are corrupted…

Cant express what a joy that is…

IT must be a v3.0.1 or a Linux version problem, I guess…

Any one tried it on Linux?

Soren_Garup · May 20, 2020, 11:06pm

Thanks for the tip, but @grsnovi managed to open my example file without any problems, so guess it not a file problem

Soren_Garup · May 20, 2020, 11:09pm

Sorry, i must have misunderstood

I thought you managed to open MY example NEF…

Please disregard all my comments about it being unbroken…

Soren_Garup · May 20, 2020, 11:12pm

I tried to download the NEF in the thread, and it didn’t even open in rawTherapee as my NEFs do…

elGordo · May 20, 2020, 11:17pm

I tried opening your file 2013-02-16@17-15-0802.nef in darktable 2.6.2 on Ubuntu 18.04 LTS. A thumbnail showed up fine in lighttable, but when I tried to open it in darkroom, I got this message:

failed to read camera white balance information from '2013-02-16@17-15-0802.nef'!

and the photo does not open. I’d suggest having a look at the thread that @Thomas_Do linked.

grsnovi · May 20, 2020, 11:19pm

No, your NEF would not open but others that I downloaded opened fine. I got the same error with your file that you did. Sorry for the confusion.

Thanatomanic · May 21, 2020, 2:45am

Ping @LebedevRI

Bbawt · May 21, 2020, 5:11am

It is a bit strange. The file opened fine for me in RT 5.8, Photoscape X, XnView MP, Adobe Bridge. But in darktable I can see the thumbnail in the import dialogue but then get “file has unknown format” message in lighttable mode.

LebedevRI · May 21, 2020, 8:04am

This is not a proper bug reporting method, please use issue tracker

Thanatomanic · May 21, 2020, 8:31am

I knew was a reasonable chance you might say that, sorry, I’ll go through the proper channels

@Soren_Garup I took the liberty to create an issue on the darktable GitHub. You can find it here: NEF's that were desecrated by Nikon Transfer fail to load · Issue #5149 · darktable-org/darktable · GitHub

Oleastre · May 21, 2020, 11:25am

I wouldn’t call darktable “broken”. There is a bug in rawspeed (the library that darktable uses ) and it deserve a proper bug report, as Roel just did for you

LebedevRI · May 21, 2020, 12:38pm

It would be really helpful to preserve such a raw on https://raw.pixls.us/
Can you find some such affected raw that has no people in it,
that you’d be willing to contribute to RPU?

Soren_Garup · May 21, 2020, 12:47pm

Thanks for the bug reporting

This was the conclusion from @LebedevRI :

#The problem is:

github.com

darktable-org/rawspeed/blob/708f3cdc7bf1e1a7cbe52c479a625bda6d08455c/src/librawspeed/decoders/NefDecoder.cpp#L113

    
      
          
          
if (34713 != compression)
            ThrowRDE("Unsupported compression");
          
          
uint32_t width = raw->getEntry(IMAGEWIDTH)->getU32();
          uint32_t height = raw->getEntry(IMAGELENGTH)->getU32();
          uint32_t bitPerPixel = raw->getEntry(BITSPERSAMPLE)->getU32();
          
          
mRaw->dim = iPoint2D(width, height);
          
          
raw = mRootIFD->getIFDWithTag(static_cast<TiffTag>(0x8c));
          
          
TiffEntry *meta;
          if (raw->hasEntry(static_cast<TiffTag>(0x96))) {
            meta = raw->getEntry(static_cast<TiffTag>(0x96));
          } else {
            meta = raw->getEntry(static_cast<TiffTag>(0x8c)); // Fall back
          }
          
          
ByteStream rawData(
              DataBuffer(mFile->getSubView(offsets->getU32(), counts->getU32()),

That looks for TIFF IFD that contains an entry with tag 0x8c ( ContrastCurve ?)
Clearly, it is not present in that raw, so we fail."

END

As I understand it Darktable will not handle the file due to missing information in the file, but rawTherapee will handle the same file despite the missing information.

Since Darktable is not broken, I guess there will be no bug-fix to the problem?

And I will either have to make a new workflow for rawTherapy or find some other work-around?

Soren_Garup · May 21, 2020, 12:51pm

I have 32GB of NEFs from my now sold D3100, so I’m sure there must be a few without humans in them…

How many do you want and where do i upload them?

Thomas_Do · May 21, 2020, 1:08pm

The internet is full of reports of corrupted NEF files. Often these files were changed (destroyed?) by the transfer software. The example file of @Soren_Garup also the following information in the header: “Nikon Transfer 2.0” and has obviously been edited by that software.
For the future, I can only recommend to use file transfer tools that do not edit the files themselves.

paulmiller · May 21, 2020, 1:11pm

I have this problem with NEF files from a D3100 that have been edited (even viewing them may be enough) in View NX2. View NX2 changes the raw file from a big-endian to little-endian TIFF wrapper (if you check the header it goes from II to MM). This is on the macOS version of View NX.

Not sure if this is the same issue as @Soren_Garup is seeing but I do still have the camera so I can get some sample files togther if that would help.