The sun might burn a hole in your sensor. Oh yes, those fires. My cousin from the west (BC, Canada) says she hasn’t been able to see the sunset. She has an autoimmune disease, so it has been tough on her.
Apparently the smoke has now gotten high enough into the stratosphere to hit some eastward winds that are blowing it all the way over to the East Coast.
Yes, even OUR skies are getting hazy and reddish now!
Edit: Which makes a perfect use case for SSF measurement since we’re losing easy access to clean D50.
1 Like
Thanks for checking out the grating corrections!
It’s a bit disappointing it didn’t improve things and also slightly surprising, given the rather large correction (factor 2 over the range of interest). At the same time I don’t know what exactly goes into the DE calculation, so it’s totally imaginable that these smooth, continuous corrections have not that high an impact.
Anyway regardless of potentially open questions, the conclusion is nice: The cheap grating is just fine for good results 
I still haven’t exonerated my calibration operator in ssftool; it’s a simple division, but along with 1.0 normalization I’m not sure it’s all acting correctly.
That would be ssf_powercalibrate? And calibrationfile is something like a normalized version of source powerspectrum * grid spectral efficiency * diffusor power efficiency? That looks quite straightforwardly correct to me (though not too much stock should be given to such a statement from me ). Or what is there concretely that makes you doubt it’s correctness?
I don’t know yet, just looking at some normalized data out of it just didn’t look like the relationship was preserved. It’s something I’m going to pick at next week…
Now, how you express it, as a product of the three, I haven’t done; I’ve attempted to apply each individually to the SSF measurements. I don’t see how that’d yield a different result, but I’m not really a math guy and a lot of those implications escape me until I just go and try them… 
This just hit hackaday:
There’s some good discussion of calibration sources in the comments.
Most notably https://eprints.lancs.ac.uk/id/eprint/6736/1/inproc_326.pdf is linked from the comments
There’s a bit of discussion regarding line CCD vs monochrome 2D camera, TBH for our purposes a monochrome 2D camera is fine for the “reference” calibration, and obviously for the camera to be characterized itself, it has to be a non-monochrome camera
1 Like
Interesting line CCD monochrome sensor. I wonder if they compensate the CCD for angular dependence and QE in their project:
Should be relatively easy to drive directly from an ESP32 or similar via i2c? No need for a lens in our application I think.
Or from a Pi, although in this application:
Since the camera we’re measuring has a 2D sensor, most of the benefits of line sensors are lost.
The main thing there that was useful for this application is the discussion of calibration sources. Specifically that paper discussing using “consumer” tungsten bulbs for spectral reference.
In this case, we’re interested in the camera’s SSF, so we need to know:
- The SSF of our reference sensor (this could be hard without existing data, but perhaps for a monochrome sensor, the general SSF of silicon CMOS is close enough?)
- The spectral power distribution of our light source
- This lets us figure out the SSF of our optical system, which can be used to compensate it out when measuring
- The SSF of our actual camera
1 Like
Though it being a single line sensor is not a problem if we use it as a calibrator, assuming we trusted the QE curve in the spec sheet:
- Place inline sensor in box, turn on well behaved source (knowledge of SPD not necessary) and take picture of spectrum from grating with it
- Compensate for inline sensor QE = spectral photon distribution of source signal after grating
- Place camera to be tested in box, turn on the same source and take picture of spectrum from grating
- Normalize with the spectral photon distribution from 2) = CFA SSFs
There are much cheaper such inline sensors out there, such as this < $5 from Toshiba. I think they are used in scanners and the like. None of them include an onboard ADC (nor does the hackaday one) so an ADC-less Pi is probably less than optimal.
The key would be finding one whose ‘typical QE’ plot is repeatable and trustworthy. Do they exist?
This one appears to be spectral energy so it would need to be converted to quanta. And it obviously includes a coverglass. I wonder why they don’t show the response below 400nm.
I think you have replaced ‘estimate spectral photon flux from similar enough lightsource’ with ‘estimate spectral photon flux from similar enough sensor’.
Nothing wrong with that, it takes out the optical system in between the actual sensor and the lightsource. But it also relies on the published QE of the line sensor with some assumptions. Specifically any angular dependence of the line-sensor-QE or the actual-sensor-QE will lead to minor problems as well as line-sensor-QEs variance. Spatial ADC gain variances of both sensors not even touched.
At that point a calibrated photodiode that one scans through the image plane connected to a good enough voltmeter might make more sense.
The question is, which way forward is practical and might improve on the existing results?
Ha, buy an i1Studio! I know when I’m out of my league… 
I had the money set aside, then decided to upgrade my computer instead. Now, I don’t have to take vacation days to run nlmeans denoise (another technical bit where I’m out of my league), but I have to start socking away my allowance again for the xRite spectrometer. Actually though, the 1-line arrays present a decent electronics project, something where I am in my league…
Seriously, i do need to pick away at these smaller influences. Still, when I went to compare results for my D7000 with a larger training set (Munsell spectra set in dcamprof), the max DE difference was even smaller, 4.61 for a LUT profile based on my measurement vs. 4.33 for the rawtoaces monochromator-measured equivalent. I’m using my Z 6 profile from my Rube Goldberg spectrscope data to good effect now. What’s that old saw about perfect being the enemy of good enough?
1 Like
Yes indeed, hence the skepticism at the end. My feeling is that variance in the latter is substantially less than in the former, especially because of the difficulty in controlling DIY temperatures and drive currents inexpensively.
A chunky slice of the budget of the Spectron II went to stabilizing the source and the integrator. On the other hand I suspect that the QE curve of the inline sensor should be repeatable with decent accuracy, at least from similar batches.
1 Like
4.33 for a monochromator measured profile…I wonder if other monochromator based setups are better? that maxDE of 4.61 is very very respectable in that regard. Kudos.
I agree insofar as there ‘should’ be something gained from taking out optical system and drive voltages for incalibrated filaments. But when a monochromator setup does not really improve on the results, it’s somehow hard to justify a line sensor to calibrate against.
My Best guess is that the mentioned monochromator setup has still leeway to be better but was not optimized to deliver better maxDE values? A lot of people can see a maxDE of over 4! I would have expected a better result from a proper monochromator setup, and yet I don’t have any reference.
I know, this is what keeps me picking at it, even though the results i get in my raw processing are just fine.
Here are all the monochromator-based data sources I’ve assembled in this endeavor:
rawtoaces:
Here is a link to a post that I believe describes their measurement methodology: Results from an IDT evaluation - Tech/Engineering - Community - ACESCentral
See the link in the post to the actual report: https://community.acescentral.com/uploads/short-url/2kdAkrmO79OIr1bU1S9J4eDEctC.pdf
RIT camspec and Univ of Tokyo:
These links actually have pictures of their measurement setup:
https://nae-lab.org/~rei/research/cs/zhao/database.html
Now that we’re talking about it, I’ll probably take a few of these datasets and generate some munsell-based LUT profiles with them to see what DE they produce. Of note is that dcamprof produces a boatload of reports and images if you tell it to, and some of those are linear TIFFs that visually depict the difference for the various DE assertions.
I did a bit more digging behind the IDT_report_v4.pdf, found out it was a separate endeavor from the rawtoaces collection. Scott Dyer did four cameras, Canon 5D Mk ii and Mk iii, Nikon D810, and Sony A7. The thread I posted contains a post further down where he links a DropBox with 12GB of files supporting this measurement; I downloaded the tab-separated data files for each camera and added it to my collection.
A consideration in all of these endeavors is that the measurements were taken through a lens. Dyer’s data includes transmission data for the lenses; I plotted the Nikon D810’s 24-70 f2.8 lens and it looks like this:
Rather significant, on the order of the diffraction grating, which I’m still not including…
The “gold standard” measurement technique appears to be using the output of a calibrated monochromator to shine on the bare sensor, through an integrating sphere. Far past my budget, and I don’t even have example data from such a source for comparison…
This one is a nice complement to that one. For our purposes we only need working temperature to estimate SPD. They use the same resistance method as in the other paper to estimate temperature and even suggest that CCT could be used, if known.
1 Like
I like the tokyo dataset more for its 4nm bandwidth resolution.
Well with such significant contributions from a lens…I’d say choose a lens which approximates all lenses from one manufacturer best (haha!) and measure with the lens. Otherwise you have a profile for a usecase that noone uses. You always shoot with a lens. We had this discussion before, right? 
Calibrating for every lens would be something…with these lens transmission curves one could do this in software and would require a no-lens profile. Then for every lens attached you multiply (divide? well you compensate for what the lens does…) the attenuation on top of the SSF…
The spectral-lens-transmission data would be a nice addition to the lensfun-db!! 
1 Like