A request for help: TIFF writing

I recently discovered that TIFF writing in Filmulator has stopped working as expected, despite the TIFF writing code not having changed since before there was any GUI at all.

When I open a Filmulator-output TIFF in GIMP, it apparently has four pages to the TIFF, and only the first one has the image data, the other three don’t work.

When running tiffinfo on the file, it gives the following:

TIFF Directory at offset 0x8 (8)
  Image Width: 2920 Image Length: 4100
  Bits/Sample: 16
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Planar Configuration: single image plane
  Make: Canon
  Model: Canon EOS 5D
  DateTime: 2017:08:01 16:12:15
  EXIFIFDOffset: 0xe4
TIFFFetchNormalTag: Warning, Incorrect count for "MakerNote"; tag ignored.
TIFFReadCustomDirectory: Warning, Wrong data type 3 for "PixelXDimension"; tag ignored.
TIFFReadCustomDirectory: Warning, Wrong data type 3 for "PixelYDimension"; tag ignored.
TIFFReadCustomDirectory: Warning, Unknown field with tag 40965 (0xa005) encountered.
TIFF Directory at offset 0xe4 (228)
  ExposureTime: 0.000625
  FNumber: 0.000000
  ExposureProgram: 3
  ISOSpeedRatings: 100
  ExifVersion: 0x30,0x32,0x32,0x31
  DateTimeOriginal: 2017:08:01 16:12:15
  DateTimeDigitized: 2017:08:01 16:12:15
  ComponentsConfiguration: 0x1,0x2,0x3,0x0
  ShutterSpeedValue: 10.625000
  ApertureValue: 2147483648.000000
  ExposureBiasValue: 0.000000
  MeteringMode: 0
  Flash: 16
  FocalLength: 0.000000
  UserComment: 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0
  FlashpixVersion: 0x30,0x31,0x30,0x30
  ColorSpace: 1
  Tag 40965: 75992
  FocalPlaneXResolution: 3086.925781
  FocalPlaneYResolution: 3091.295166
  FocalPlaneResolutionUnit: 2
  CustomRendered: 0
  ExposureMode: 0
  WhiteBalance: 1
  SceneCaptureType: 0
TIFFReadDirectory: Warning, Unknown field with tag 513 (0x201) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 514 (0x202) encountered.
MissingRequired: TIFF directory is missing required "ImageLength" field.

Nowhere does it seem to show that there are any extra frames or anything…

Anyone have any idea why this might be? Filmulator’s TIFF writing code is in this file.

If we can’t figure this out we may try switching to another TIFF library, like maybe TinyTIFF…

Are you sure the bug is in Filmulator and not in Gimp?

You might be right.

The TIFF works fine in RawTherapee…

Before you indict libtiff, you might comment out the exiv2 code at the bottom of the routine, recompile, and generate a new test image. I don’t see anything out of order with your libtiff code.

2 Likes

That worked. I suppose that our current strategy of copying all the exif over wholesale is what’s causing issues.

I’ve gone back and forth about the whole exif enchilada, but I’ve found I really only need the essential exposure/focal length information carried forward. If I want to pick at the image exif-wise, I just open the raw; I put in a exiftool-based dialog box for that purpose.

Edit: Oh, TIFFs vex that approach; the libtiff-supported metadata is not camera-oriented.

I guess your problem is coming from the thumbnail(s) in the RAW file. I had to deal with the same problem in photoflow, and came up with a pretty simple solution with EXIV2. You can check the lines of code starting from here.

Hope this helps!

2 Likes

Interestingly, it didn’t help. I guess I’ll have to list all the tags and see which ones cause problems…

You might want to look at https://github.com/darktable-org/darktable/blob/master/src/common/exif.cc and grep for dt_remove_exif_keys in there. We have a few places with lists of EXIF tags that may not be written to the exported images.

1 Like

I, uh, finally got around to implementing this exif cleanup from darktable, but it seems like when I write exif data to a TIFF it still ends up having 4 subimages of some sort.

Unlike before, where the first image was the good one and the other three weren’t, now the first two both contain the full image, the third one is a corrupt thumbnail (below), and the fourth simply crashes GIMP.

At this point I’m not sure what to do besides disable TIFF exif writing again. The TIFFs crash Photoshop, which is obviously not desirable behavior.

Right now, I just use the rather dodge-ily-documented libtiff extensions to write a limited bit of exif:

Are you using exiv2?

Yeah, I’m using exiv2. From what I gather it’s incompatible with libtiff.

I’ll try your method tonight probably.

There is also a line in dt saying you cannot write metadata via Exiv2 to an open TIFF file, you have to close it first and do it in a second pass. Or rather, you cannot write to a multi-page file, so you have to close after the first page, then add Exif, then append other pages.

I’m getting a few errors.

libtiff complains “cannot modify tag Compression while writing” even though I’m not modifying the tag.

The TIFFSetDirectory and TIFFSetField calls fail too, with libtiff complaining “unknown tag 34665” which is the TIFFTAG_EXIFIFD as in your code.

Any ideas?

Hmmm… I’m gonna guess right now that what you’re experiencing has to do with something you’re doing that I’m not doing before line 230… :slight_smile:

I offer very few options for writing TIFFs, and compression is not one of them. Just a guess.

Back to exiv2, my understanding of its operation is that you build a data structure with the tags you want, then you apply that data structure to the image file with code that looks like this:

Exiv2::Image::AutoPtr image = Exiv2::ImageFactory::open(file);
image->setExifData(exifData);
image->writeMetadata();

from https://www.exiv2.org/doc/addmoddel_8cpp-example.html

This is creating new subimages in the file?

Or something. I do not understand why it does this.

In any case, I got your method working to write the basic exposure info, thanks a lot!

After digging into the whole metadata thing a couple of years ago, I decided to rely on the raw file as the canonical source for all data related to the capture, and store only exposure information, if that, in my renditions. I used the various image libraries’ metadata writing functions, although jpeg and png required supplementary support to organize the tags in the IFDs.

The one bit of crucial metadata in my little world is the processing toolchain that describes how the rendition is produced from the source (raw) file, and I currently store that in EXIF:ImageDescription. One day I’ll explore using the XMP tags, but that’s a low priority right now.

I fiddled a bit with exiv2 in considering it for the upcoming rawproc 1.0. The main challenge was replacing or complementing all use of my image data C++ map structure, which also gets stuff like libraw-collected information and things I use internally. I’ll probably revisit when I resume consideration of how to embed the toolchain…