I would like to generate some linearly encoded images for exploratory / pedagogical purposes, mainly for use in Darktable.
Specifically, I can write a computer program (in Julia) that fills a W x H x 3 array with the pixels I want. The question is, how should I save this so that it is understood as linear (ie scene-referred) input?
Recommendations for specific formats are also welcome (eg if you recommend TIFF, what tags etc should I use to achieve this?)
You either need to create a DNG w/ its required tags (i.e. color profile), which is a bit of a bother, or just write a 16b TIFF or PNG w/ the appropriate linear ICC profile embedded. You could also write a a (float) EXR file or a PFM file, which dt assumes are always linear if no profile is attached.
Note that you can always manually choose the linear input profile yourself in dt.
I think most easy way to make linear RGB files is using darktable. In darktable you can export linear tiff or png files selecting linear REC2020 RGB, linear REC709 RGB as export profile. They can be also used as linear input image data for darktable.
OpenEXR is a format specially designed for such task:
https://openexr.com/en/latest/SceneLinear.html
However the step of storing metadata for colorspace tracking is not the best defined:
chromaticities attribute which store the CIE x,y primaries of the encoded colorspace, but application usually ignore it (and it’s also usually not set properly)As a bonus OpenEXR is very flexible encoding wise, offering 16 bit half-float and 32bit float for disk size gain, and a whole bunch of different compression algorithm for different use case. DWAA is impressive as being a lossy compression algorithm but with no visible difference.
EDIT: my answer is assuming you want to store already debayered camera data. OpenEXR might be used to stored non-debayered data (“RAW”) but I don’t know if would be any useful over other standard format and how complicated it would be.
Luckily storing these is not really required for simple use cases like this one - linear Rec.709 is implied by the spec.
yes true that most application just assume all EXR files are “linear REC.709/sRGB” encoded.
And because the floating point format can store negatives values you could just always store them as linear sRGB and just reconvert to whatever colorspace you need during processing (not sure of the potential pitfall of this workflow).
Yes, sorry for not specifying this.
I think I will go with PFM, as suggested by @kmilos, they are trivially easy to generate.
Yep, just pay attention to the endianness and the bottom-up row ordering. Again, dt will assume linear Rec.709 for these.
thanks for the pointers. would you know whether there are any plans to include TRC in the standard attributes? sometimes it makes sense to record raw video in something log space/HLG etc so 10 bits would give you enough leeway to grade it later. i’m writing a custom tag so i remember what is in my files but obviously a standard attribute would be better.
How are you writing 10 bits in EXR? The 32-bit integer encoding is a bit of a niche (and not efficient), and almost everyone uses float (either 16b or 32b). The format is really made for linear data, so I don’t think a standard attribute like that will likely happen…
fair point. you’re proposing to always extract instead.
Sorry I am not aware of the “TRC” abbreviation meaning 
but you could try to search in the github repo. Or ask on the ASWF slack #openexr channel.
And yes it’s really not recommended to store log-encoded data in EXRs:
Integer format are far more efficient for this.
Tone Reproduction Curve. It’s an ICC profile term.
Thanks ! Ha yeah I saw that it is probably the most tricky component of colorspace to handle.