I would like to experiment with different ways to demosaic/denoise RAW files and would like to export a particular buffer in the image processing pipeline as TIFF files so I can experiment with the data using external tools (Jupyter notebooks and Numpy).
For this, I open a RAW file, select the “passthrough” mode in the demosaicing module, and disable any denoising module. This gives me (on screen, in darktable) data that I would expect, i.e. a monochrome mosaiced image.
When I try to export this image as 16 bit TIFF however, the resulting TIFF file only contains integer values in the range 0 to 200, suggesting that some type of transformation happens on the exporting step. I played around with different “export profiles” in the “output color profile” module to no avail (the values shift around but remain in the 8 bit range).
Therefore, my questions:
Is there a simple way to save intermediate image buffers in the image processing pipeline of darktable? (For example, when developing novel denoising or demosaicing methods.)
Is there an export mode that enables me to export the output of the “white balance” module, disabling all subsequent modules?
I don’t usually recommend my software, but you have a particular need for which it is well-suited…
rawproc is a raw processing toolkit - when you open a raw file, the first thing you see is the color management attempting to render the data read from the file, the only transform being a proportionate transform from 0-65535 integers to 0.0 -1.0 floats. At that point, you can save a TIFF of that, either 8-bit, 16-bit, or float, and even embed the camera profile if you desire. But, what you can also do is stack, one-by-one, the subtract, white balance, and demosaic to incrementally build to a RGB image. And, at each step, you can output to TIFF in the manner I previously described.
I use the librtprocess demosaics, so you have benefit of @heckflosse’s wonderful single-algorithm routines; if you want the dual routines, you’ll have to use RawTherapee. I’m also doing a bit of work on the other tools to more fully provide the necessary adjustments for a wider variety of cameras, e.g., per-channel black subtraction (in fact, my task today is to incorporate the Bayer image-walk loops for pre-demosaiced image).
The most recent version, 0.9, will probably work fine if you are using a mainstream camera, Nikon, Canon, Sony. But on Ubuntu 19.10, compiling according to the instructions in the README should be simple enough (famous last words… ) if you don’t mind compiling wxWidgets for a static link.
My motivation writing my own processor was to implement a workflow I like, and learn about the particulars of raw processing under-the-hood. But I think the approach would also facilitate your needs.
How do you know that you only get values between 0 and 200? I tried to do the same using a Canon CR2 file and 16 bit TIFF export and seem to get full range of values. I used imageJ to get the pixel values from the exported image.
This produces the full 16 bit range, so indeed the issue was not with Darktable at all but with the Python Image Library (PIL) reading the uint16 encoded TIFF file wrongly.
Check out also imageio for a simpler API, should handle 16b TIFFs just fine.
Btw, I had no problem getting the 16b data using Pillow, but only if it’s a real grayscale, single channel TIFF. It seems in dt case it is exported as a gray RGB, which indeed Pillow converts to 8b…
) if you don’t mind compiling wxWidgets for a static link.