I’m planning on using RawTherapee’s camconst.json as a source of data for Filmulator, specifically for white saturation points for the time being.
Is the dev branch of RawTherapee on GitHub the canonical place for it, if I have a button auto-download it?
I am not saying that you have a library which could be the canonical shared source for it.
In RawTherapee the dev branch on GitHub is the canonical place for it 
Now to figure out how I’m going to make use of it…
I might have it be loaded into the SQLite database for easier querying instead of reading through it at startup…
You probably already figured out it is not strict JSON, as there are comments in the file, which is not allowed. Something to take into account when parsing the file…
Actually I had no idea, because I have never dealt with JSON before.
I was hoping I could just use Qt’s library for parsing it, but maybe not…
I include camconst.json with rawproc, and I use Dave Gamble’s cJSON library to read it.
It seems to ignore the comments just fine…
Edit: Oh, that’s also what RawTherapee uses… 
Happened upon this on Hacker News…
“To be safe allocate 5 bytes more than you need”
I don’t use that function, I think it’s only needed if you intend to print the data structure.
Here’s my code:
I’ve gotten the parsing working now, but now I need a little advice on how aperture scaling works.
Do I scale the white saturation point before I subtract the blackpoint, or do I scale it afterward?
Actually, I might not bother with aperture scaling. The whitepoints in camconst.json seem pretty close to the limit as-is.
In other news, it now works! No more purple highlights on 80D pics!
Also I discovered that while the 80D whitepoint was set very unsafely for intermediate ISOs, my 60D’s whitepoint was set extremely conservatively by LibRaw, and at some ISO values I have exposed a decent amount of highlight headroom!
Today I discovered that some Nikons have the camconst white level specified as if they were 14-bit, not 12-bit…
Yay for more complexity.
Not sure if it’s the greatest thing to do, but in rawproc I use an instance of the blackwhitepoint tool to scale white, and I have an option named ‘data’ that scales it to the maximum value of the channel with the largest maximum. There’s a option with ‘data’ to scale to the minimum-maximum, which just blows clipped pixels to white. So, essentially I scale ‘per-image’. I need to work this into the librtprocess hilight_recon() behavior, which I ran out of brain cells to do prior to 1.0…
I just detect when a) the camera is a Nikon, b) the LibRaw whitepoint is less than 4096, and c) the CamConst whitepoint is greater than 4096, and then convert the 14-bit value to the 12-bit value per the comment in camconst.
It’s a pretty simple fix, all things told. It does, however, assume that the LibRaw whitepoint is never less than 4096 for a Nikon 14-bit image…
Also, I wonder what’s gonna happen for future 16-bit raws (I think the a7s3 can write 16-bit)…
One thing of note: Fuji data in RawTherapee has no aperture scaling listed, instead saying “likely negligible”, but on this LibRaw page I stumbled across someone saying that the X-Pro2 has tons of aperture scaling, by a factor of 1.4 for f/1.2.
https://www.libraw.org/articles/Canon-5Dmk2-headroom.html
That’s a large enough change to make it definitely worthwhile to ask for people to gather data on, I’d say.
Unless the camera itself throws out the raw data, given that it tops out at 16100?
@CarVac It may differ greatly based on specific camera model. For the FUJIFILM X-T3 people supplied sample images for me to determine that there was no significant aperture scaling, unless I have not done my analysis properly which is entirely possible. I may have wrongly extrapolated this notion to other Fuji models…
In terms of highlight clipping, there may be no need for compensation for aperture scaling even if the camera does apply it. After all, the white point for the cameras is quite close to 2^14.
If it’s true (that person said they did testing of the photon transfer curve) then Fuji is just throwing away a half stop of dynamic range at f/1.2. I wonder how strong it is at f/1…
I’ve noticed a while back that the Panasonic blackpoints in camconst are offsets from a black level from exif, but I can’t find that exif tag myself, and the black points returned by LibRaw are inconsistent about needing the black offset. For example, the GX80/GX85 doesn’t seem to need it, the G9 needs it, the GH5 needs it, the GH5s doesn’t need it (and doesn’t have a camconst entry at all), the FZ1000 doesn’t need it…
Does anyone know what exif tag it is that I need to look for in the case of these Panasonic cameras?
With a handy GX80 file, I found the black level to be in three tags, BlackLevelRed, BlackLevelGreen, and BlackLevelBlue. The value for all three in this raw is 128, which we determined in another thread was not good. For this image, libraw provides a single value, 143, which looks like it comes from their file RawSpeed/rawspeed_xmldata.cpp.
I just use libraw-supplied values, which can come one of three ways 1) single value, 2) per-channel, and 3) a number for each position of the mosaic. Previous to the GX80, I found corresponding exiv tags for how libraw presented the data, but this camera broke that expectation…
Interesting, exiftool finds those tags but not exiv2.
I used to just use libraw-supplied values too, but that doesn’t work properly on some cameras, like the G9 and GH5.
I suppose that’s just a bug?
