Nikon: a specific raw sample wanted.

FWIW, the original photo for this PlayRaw is Lossy type 2, 14 bits per sample. I donā€™t have time right now to throw up a fresh DT to see whether it has the problem.

Sadly (fortunately?), that raw is not one of these split raws, loads fine.

Youā€™ll have to be more specific than that.
Building. For fedora/opensuse there are packages by @darix

I have found some on my hard drive, the good news is that all of those RAW are opening fine with current master.

Thank you for checking! Hopefully someone will have worse news than that :slight_smile:

Up. Still looking :slight_smile:

I think that such splited RAW are obtained when using a slow flash card (for example, Š”lass 10) for long serial shooting. Maybe someone will try to?

@LebedevRI-
I have 2.2.5 installed.

I pulled 2.4.0 RC2 from git master.

I have at least two images that are 14-bit lossy Nikon RAWs that fail to load using 2.4.0 RC2, but load and have been edited in 2.2.x.

The images were taken with a Nikon D5500.

Hereā€™s an exiftool snippet:

======== ./DSC_0248.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
======== ./DSC_0258.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14

Here is a snippet of the errors I saw with 2.4.0 RC2:

[rawspeed] (DSC_0248.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16
[temperature] failed to read camera white balance information from `DSC_0248.NEF'!
allocation failed???
[rawspeed] (DSC_0258.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16
[temperature] failed to read camera white balance information from `DSC_0258.NEF'!
allocation failed???

@mikrom-
The images were taken with a SanDisk Extreme Pro UHS-I SDXC, so I can at least confirm that it doesnā€™t only occur with slow cards.

1 Like

@lptech
Hi!

Oh great, sounds just like what i was looking for!

Could you please contribute at least one of such raws to the https://raw.pixls.us/, please? :slight_smile:

@LebedevRI-

I uploaded one of the RAWs.

Do you expect to have this fixed before the 2.4.0 official release, as a 2.4.x patch, or in a later release entirely?

Thank you!

1 Like

Well, given that 2.4.0 has happened alreadyā€¦ :slight_smile:

Thanks, butā€¦ I donā€™t suppose there is some other raw without people in it?

@LebedevRI-

I donā€™t see any images in my library that appear to be 14-bit ā€œlossy after splitā€ Nikon RAWs that donā€™t have people in them.

While it perhaps isnā€™t the best for long-term public verification/testing (as I see the image uploader rule), could you use the image for development and hope someone uploads another sometime in the future?

Thanks!

Iā€™m afraid i canā€™t really give any credible time estimates. ā€œOnce itā€™s doneā€.

Ok then, this is not too bad, too.
Iā€™m sure this will bite someone else and there will be more samples :slight_smile:

Thank you for this sample!

I have a (probably) controversial suggestion - what about detecting these photos in darktable (or any other program using rawspeed and under your control), and showing a dialog once the user imports such a wanted photo?

And generally - what about checking if the combination of camera/lens has a record in RPU, and if not, kindly asking the user if he doesnā€™t want to share some photos? Of course this could be seen as a breach in anonymity of use of darktable.

By now there is enough (~5, including 2 elusive 14-bit ones) samples with this problem,
so there is no further need to actively look for these samples.

Believe me, i did think about it :slight_smile: I do not see any sane way how to do this.
Fetching the ā€˜missing camera listā€™ from RPU is, well, will access the network, and would need to be implemented in dt. And just bundling the current list of missing cameras will, at the very least, quickly become outdated.

So no, i donā€™t think it is feasible.

Yep!

@LebedevRI-

I have another likely 14-bit ā€œlossy after splitā€ RAW sample from a Nikon D750.

The image does have one person in it. Should I upload it?

Thank you, but not that one, there are two samples already (one without people at all)

Hi.

Iā€™m using a Nikon D500 and have encountered some files that fails to load in darktable, due to failure to load the cameraā€™s White Balance.

I have done some testing and it looks like I am able to ā€œforceā€ the failure of getting the camera white balance within darktable for images with 14-bit Lossy Compression if the ISO value is higher than 25600.
Lossless Compression, or uncompressed, does not seem to suffer from the same problem.

I can not say that this is the cause of the problem, but it could be worth pursuing.
Hopefully someone else can test with another camera and see if they have the same problem.

My darktable version is 2.4.1 (had the same problem with earlier versions too.)
D500 firmware: C=1.13 / LD=2.016 / W=3.00

4 images with different ISO-settings shows like this using exiftool:

$> find -iname *.nef -print0 | sort -z | xargs -0 exiftool -compression* -nefcompression -bitspersample -iso
======== ./_5001196.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
ISO                             : 64508
======== ./_5001197.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
ISO                             : 51200
======== ./_5001198.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
ISO                             : 25600
======== ./_5001199.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
ISO                             : 6400

Upon opening darktable this is shown in the console:

[rawspeed] (_5001196.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16
[temperature] failed to read camera white balance information from `_5001196.NEF'!
allocation failed???
[rawspeed] (_5001197.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16
[temperature] failed to read camera white balance information from `_5001197.NEF'!
allocation failed???

As we can see, ISO 25600 and below does not show an error. Higher than 25600 produces the error and the file will not open in darktable.
Is the same code, (rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&)), executed if the file is Lossless Compressed, or 12-bit?
Lossless and 12-bit seem to work with higher ISO-values, so the problem looks a bit inconsistent.

I can upload the 4 images referred to above if it will help resolve this issue.
Just let me know.

I just noticed that some of my files are broken. Definitely they worked before. Sample ā€” _DSC0395.NEF (33.0 MB)

Hi,

I have some, at least the first with no people in, D5500. From darktable 2.4.4:

[rawspeed] (_DSC6849.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16
[temperature] `NIKON CORPORATION NIKON D5500' color matrix not found for image
[temperature] failed to read camera white balance information from `_DSC6849.NEF'!
[rawspeed] (_DSC7011.NEF) void rawspeed::HuffmanTable::setCodeValues(const rawspeed::Buffer&), line 176: Corrupt Huffman. Code value 92 is bigger than 16

find -iname \*.nef -print0 | sort -z | xargs -0 exiftool -compression* -nefcompression -bitspersample
======== ./_DSC6749.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
======== ./_DSC6849.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
======== ./_DSC7011.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
======== ./_DSC7012.NEF
Compression                     : Nikon NEF Compressed
NEF Compression                 : Lossy (type 2)
Bits Per Sample                 : 14
    4 image files read

Is this still needed?