Darktable PNG ICCP chunks and display colour profile

Software darktable
1

Hello
I’ve been trying a new image viewer “avis-img” showcased on this very forum, coded in rust and hit kind of a problem with the testing …

Being a long time user of Darktable and used to export my edits in PNG format, I found testing that the files I exported before I calibrated my display and applied a display profile to my system are compatible with avis-img (using native rust png decoder to open PNG) but since I used display profile on my system, Darktable exports PNG with 2 ICCP chunks in the headers.

This render the file incompatible with rust standard png library.

My questions are the following :

  • Is this 2 ICCP chunk is a normal procedure of the PNG file standard ?
  • Why exporting a file from a system with a calibrated display render the file different ?

Maybe @Pascal_Obry could shed a light on this problem ?

Thanks for your time and amazing software :smiley: My only regret is myself not having enough time to participate in development or even using it to it’s full potential :smiley:

1 Like
2

Seems like its a bug somewhere in the PNG stack, as the standard says:

A file should contain at most one embedded profile, whether explicit like iCCP or implicit like sRGB.

3

What exiv2 version? This was fixed relatively recently: PNG: always strip the existing iCCP chunk (backport #2254) by kmilos · Pull Request #2256 · Exiv2/exiv2 · GitHub

4

OOk I run the 0.27.6 version of exif (wich carry the corrective) since fairly recently (08/03/23) and I just tested and image exported now is have a compliant header without duplicate ICCP chunks…

That’s nice ! First problem solved !
I have 2 questions non the less :

  • Why these 2 chunks to begin with?
  • Is having a display profile applied on my system is supposed to modify the files I export ? My comprehension was that it was just supposed to modify the way I display things and not in any other way the other way around …

Now I’ll have to find how to remove the extra ICCP chunk from all my PNGs :smiley: And to know wich one !

5

Said bug in exiv2.

They carry an identical profile, so doesn’t really matter. You could now try exiv2 0.27.6 on the command line to change something like exiv2 -M"set Exif.Image.Copyright foo" bar.png, it should keep only one…

1 Like
6

Wow, nice and thanks, I’ll do a quick test before bulk applying.

7

Beware that there might be a new exiv2 bug in 0.27.6 affecting Olympus metadata unfortunately… If you’re not an Olympus user, you should be good to go.

1 Like
8

I’m not but I have processed a handful of olympus raw. I’ll match names to avoid any problem. Thanks !

9

If you are really desperate to remove the duplicate iCCP chunk from those as well there is probably an alternative way using dd to skip it.

As far as the whole display profile and colour management goes, please search the forum, it’s been covered many times. I’ll just say, use only “standard” profile for export and “output profile” (sRGB, AdobeRGB, etc.), your display profile shouldn’t leave the editing/viewing app.

10

By all means. thanks !

11

You need to first use exiv2 -pS bar.png to see where the iCCP chunks start in each file. Then use those addresses and their difference to just skip one of them using a “triple” dd command like this one.

12

It’s been a while but as I’m may not be the only person impacted by this bug I’ll post my effort to build a script to scan and correct my whole library here given that not all my png are impacted (the old png files before having a profile for my display were not malformed)

Requirements :

  • detect malformed PNG files
  • process only impacted PNG
  • Bulk process file through a complex folder architecture

Design choices :

  • I’m more familiar with bash than any other quick scripting language so i’ll stick to that
  • I’m familiar with the find -exec command so I’m planing on using that
  • Use of a dd command to write a new file or overwrite (if I’m confident enough) with the correction

Anatomy of a malformed PNG :

TRUCTURE OF PNG FILE: ./DSC09484.png
 address | chunk |  length | data                           | checksum
       8 | IHDR  |      13 | ............                   | 0xd0a081cb
      33 | zTXt  |    8219 | Raw profile type exif..x..}i.. | 0xe3e0e856
    8264 | zTXt  |     155 | Raw profile type iptc..x.M.A.C | 0x420c1340
    8431 | iCCP  |    8522 | sRGB..x...WTU....Z;g...9or.9., | 0xe71b7543
   16965 | iTXt  |    3575 | XML:com.adobe.xmp.....<?xpacke | 0x951e085d
   20552 | iCCP  |    8522 | sRGB..x...WTU....Z;g...9or.9., | 0xe71b7543
   29086 | iCCP  |    8522 | sRGB..h...WTU....Z;g...9or.9., | 0xc9d10e63

Normal PNG

STRUCTURE OF PNG FILE: original/sanepng.png
 address | chunk |  length | data                           | checksum
       8 | IHDR  |      13 | ............                   | 0x98043999
      33 | zTXt  |     313 | Raw profile type exif..x..RA.. | 0x3e3724bf
     358 | zTXt  |     214 | Raw profile type iptc..x.UPI.. | 0x8d850d1b
     584 | iCCP  |    2384 | icc..x...gT.I...y..B ..C..H..@ | 0x98bb3170
    2980 | iTXt  |    4496 | XML:com.adobe.xmp.....<?xpacke | 0xd1963598
    7488 | oFFs  |       9 | ........                       | 0x62f8d78e
    7509 | pHYs  |       9 | .........                      | 0x679fd252
    7530 | IDAT  |    8192 | x...O.dI......{....Y.5=...)... | 0x39f0413a

There seem to be not one but 2 extra ICCP chunks on the malformed PNGs

Code snippet :
Detection ouputs 3 on malformed PNG and 1 on sane PNG :
exiv2 -pS DSC09484.png | grep iCCP | wc -l

PS : See bellow for script advancement

13

There could be many actually - depends how many times you ran an exiv2<=0.27.5 modify command on it (either as command or via libexiv2 API), it’ll add another…

1 Like
14

I know there’s many I even know how many ! All the PNG I exported with DT since january of 2014 !!!
The only PNG files of this period (01/2014 to now) not impacted in my photo folder are those exported with hugin :smiley:

That’s why I need a bash script to discriminate and therefore correct only the impacted ones :smiley:

15

I didn’t mean many impacted files - I meant many extra chunks in a file, not just 1 or 2…

1 Like
16

Sorry, I red this too quickly…

After analysis of my whole photo folder you’re right :

find -iname '*.png' -exec  sh -c "exiv2 -pS '{}' | grep -i iccp | wc -l " \; >> iccp-count
cat iccp-count | sort | uniq -c | sort -h
      1 8
      2 7
     10 6
     27 5
     97 4
    174 3
    187 1
    597 2
   1502 0

it just happened that on my sample of “bad” PNG’s they coincidently all had 3 …
Thanks for the tip ! I’ll dodge this one :smiley:

17

On the few seconds I manage to snatch out of my schedule here is the advancement of the script :

Step 1 - Sorting out OK and Malformed PNG files
I detect as malformed any files with more than one iCCP profile in the header (thanks for @kmilos for the tip). This is still a one-liner
find ./ -iname '*png' -exec sh -c "a=\$(exiv2 -pS {} | grep -i iccp | wc -l) ; if [ \"\$a\" -ge \"2\" ]; then echo {}.Malformed; else echo {}.OK; fi" \;

Step 2 - Extracting necessary information to repair the file

1 Like