I’m working on a rawproc 0.6 release, but I’m still figuring out the CMS I implemented in 0.5. My question is, what image should I generate histograms from, the working profile image or the display image?
Right now, I’m using the internal working profile image, which seems to correlate to the display until I save to TIFF/prophoto/1.8 gamma, which when I open in rawproc shows a “darker” histogram than the transformed display. If I turn off CMS and open the TIFF, it is indeed dark per the histogram.
I’m debating putting a dropdown in the histogram to select the working/display image, but, would it just be more appropriate to use the display image for the histogram? It would seem the user adjusts things based on what they see, the transformed display. But somehow basing the histogram on that when all the work is being done on the internal working space image doesn’t sound right…
I guess there is the input histogram and the output one. As a user, I would want to view the output (preview) histogram that changes as I make my adjustments, but I would always want to toggle between it and the input (raw or otherwise) one to see how much the image has changed.
Personally I think that the histogram should be based on the pixel values in the working profile. This way it can guide the editing beyond what can be seen on the display.
Another useful variant could be the histogram in the output profile, if for example you edit in pro photo and you are going to save the result in sRGB. I think both would be needed.
The display profile is IMHO not useful for histogramming, because it changes from one system to another and also does not represent the pixel values that are being edited…
One is linear but not raw: it’s after white balance, highlight recovery, and exposure compensation, letting you see exactly what those do without trying to undo the gamma curve in your head.
Then I have a histogram after Filmulation. I don’t think this would be necessary in a more conventional editor.
@ggbutcher After rereading your post, I am beginning to understand your quandary. Gamma is a part of the profile but, when the working and output gammas are different from the display gamma, it can be a source of confusion for the user.
A long time ago, I spent a week or so helping this lady restore a large set of old photos only to have her be extremely disappointed with and condescending to me because I did so with a different gamma than her printer. Fortunately, I never met this terrible person again.
Maybe you could do it the way some image editors do it, where you can assign the profile and have the display emulate the gamma, such that the histogram would follow the input/output profile but the visuals would be adapted. I might not be describing it right: just launch GIMP and observe what the Image > Color Management > Assign / Convert to Color Profile options do. Try with profiles with different gamma.
Yes! That’s it. I’ve been loading raws with prophoto/1.8 gamma because that’s what all the ‘cool kids’ (those that understand color) say, and producing display and output with 2.2 gamma appropriate for sRGB. I think LittleCMS has been protecting me here, insuring the gamma is properly considered in the transforms, but the histograms confuse me.
Maybe I should be using the same gamma throughout?
Thanks @afre, I’ve already spent quality time reviewing @Elle 's articles. I use her profiles to make internal profiles for rawproc, but I’m finding I probably need to extract the actual dcraw profiles instead. They all appear to be linear gamma, which doesn’t then reflect the application of gamma with -g. Still investigating, will write a more definitive (and correct) account later.
Edit: Just finished extracting dcraw icc profiles for selected colorspace/gamma combinations, and dcraw is indeed encoding the gamma used in processing. Also, the primaries are pretty close to @elle 's LargeRGB profiles, so apparently no images were harmed in the production of rawproc 0.5… Still need to understand the histogram differences, more to come.
I now know my problem. It is I don’t know enough about ICC profiles to program them…
I’d implemented a set of internal profiles, based on @Elle’s profile set. Turns out the primaries that get stored are just not the same ones I put in, so I’m missing something. So, I spent the afternoon removing them from the program, and now all works fine just using @Elle’s files.
Nothing since my return to photography has hurt my head like color management. The ICC profiles are an extremely useful abstraction, but even then you can mess up with nary a clue to guide your recovery. My younger twin granddaughter is very into color; at the age of 4 she actually chose a set of socks because they made her dress “pop”. Geesh… she’ll probably get all this way before I do…
@ggbutcher - Hi Glenn. I read through your github rawproc page and rawproc sounds like a very nice raw processor. My name seems to have been invoked several times in this thread, so I thought I would chime in. These points might be relevant to the general questions that I think you might be asking:
I saw a reference to “dark” in the github rawproc write-up and I think also above. If the image looks darker than it should, this almost certainly indicates a color management issue that should be dealt with by assigning the correct ICC profile (if that’s the problem), or else by finding and fixing whatever “calibrate/profile/color-manage” issues there might be in the workflow and display chain.
If the monitor isn’t calibrated and profiled (both require a color measuring instrument), then the monitor isn’t showing correct colors (unless the monitor has a decent sRGB preset and hasn’t drifted over time since it left the factory, in which case you can get by with using sRGB as the monitor profile).
I’m assuming rawproc uses LCMS color management to send the image to the screen. That is, LCMS converts the image RGB values from the image ICC profile to the screen ICC profile, and then the converted RGB values are sent to the screen. So there is absolutely no reason to worry about matching ICC profile “gamma” values. LCMS takes care of this. As relevant notes:
If the monitor’s color gamut is significantly smaller than sRGB, the monitor can’t show correct colors even for sRGB images.
For images in other color spaces, the monitor can’t show correct colors for any image colors that fall outside the monitor’s color gamut (as defined by the monitor profile, which underscores the need to have a good monitor profile). “Perceptual intent” doesn’t mitigate this problem, even if your monitor profile actually does have a perceptual intent table. Matrix monitor profiles (most common case) don’t have perceptual intent tables, in which case asking for “perceptual intent” actually gets you relative colorimetric intent, which simply clips all out of gamut channel values.
For the chain from camera raw file to interpolated image file, I would recommend the following (but take my recommendations with a grain of salt, or perhaps the whole salt shaker ):
Use the equivalent of “dcraw -4 -T -o 0” to output a linear gamma, “raw” color file.
Use LCMS to assign an appropriate camera input profile. Where to get camera input matrix profiles:
Make your own camera input profile.
Or use PhotoFlow (requires the linear gamma branch?) plus ArgyllCMS extracticc to extract the camera matrix profile for your specific camera and save it to disk.
Or use the dcraw adobe_coeff table to construct the actual ICC profiles.
Or use the augmented matrix profiles available from various free/libre raw processors.
Use LCMS to convert from the assigned camera input profile to the chosen RGB working space ICC profile (over the years I’ve encountered several issues with dcraw color management). My recommendation is to use Rec.2020 or ACEScg as the large-gamut RGB working space:
Both have fewer imaginary colors than ProPhotoRGB.
Both seem to produce “nicer than ProPhotoRGB” results for chromaticity-dependent editing operations (anything involving multiplication or division by a non-gray color; anything that involves using individual channel data, including non-luminance-based conversions to black and white).
At this point you have the digital equivalent of a “flat print”. Except that it’s actually more flat than a “wet” darkroom black and white “straight print” because film and paper involve shoulder and toe rolloff, and digital does not (unless your camera settings involved shadow lifting/highlight compression/similar algorithms).
For actually editing the “flat print” to suit one’s artistic intent for the image, use two versions of the chosen RGB working space profile, with the same chromaticities but different TRCs - one version with a linear TRC and one version with a perceptually uniform TRC. Depending on the editing algorithm and the artistic intent, use LCMS to convert the layer or entire layer stack between the two profiles as desired, or else program in the two TRCs with some way to flip back and forth between the two TRCs. In my opinion, these are the only two TRCs that are technically relevant to high bit depth photographic editing:
linear gamma (gamma=1.0), for proper color mixing (most layer blend modes including Normal, blurring, down-scaling, Levels, Curves, etc).
the LAB companding curve (perceptually uniform). This second TRC is for things like most “find edges” algorithms, soft light blend mode, adding uniform RGB noise, and such.
Regarding ProPhotoRGB:
The gamma=1.8 TRC was chosen by Kodak because it’s closer to linear than gamma=2.2, thus mitigating somewhat the gamma artifacts from processing non-linearly encoded RGB images. Also ProPhotoRGB is a very large color space, which also helps mitigate gamma artifacts. It’s better to just use a linear gamma RGB working space and avoid the gamma artifacts altogether.
The ProPhotoRGB primaries are better primaries for general editing than the sRGB primaries, for various reasons covered in the Kodak documentation (http://photo-lovers.org/pdf/color/romm.pdf). But they aren’t as good as some other editing spaces that have been promulgated since ProPhotoRGB was released. http://colour-science.org/posts/about-rendering-engines-colourspaces-agnosticism/ “ranks” various color spaces. But I’m also speaking from my own testing, and I encourage other people to make their own tests.
@Elle, thank you so very much for the detailed response. First, I’d like to thank you for posting your profile set, they’re the reference that has restored my confidence in my elementary color management logic in rawproc.
Essentially, for raw files delivered by libraw, I’m assigning a profile I create with LCMS using an exif dump of each dcraw -o X’s ICC. I use another LCMS tool I wrote that finds the primaries, black/white points and TRCs, and I use those to produce a (hopefully) equivalent profile. That is what rawproc’s workflow uses as the working profile for the rest of the edit.
rawproc is what i call a ‘chain’ editor, you add tools to the processing chain in whatever order you choose, and what you eventually save to a TIFF or JPEG is the result image at the end of the chain, Each tool has a checkbox that specifies that tool to be the display image, and it is there that the working space → display transform is conducted. At output, there is also a working space → output transform. Right now, I’ve deleted all my internal profile generation code except for the dcraw profile creation, and I’m using your profiles for display and output. BTW, that straightened out my ‘dark’ thing; I was using my generated prophoto profile for TIFF saves, and they apparently are wrongly specified, all’s well using LargeRGB-elle-V2-g18.icc.
So, I still need to get the dcraw assignment straight. Yes, the more I consider it, it is best to abandon the dcraw concoctions and work with the linear raw and assigned camera profile. I’m probably going to release rawproc 0.6 with the dcraw-assigned working spaces because I need to add code to manage a separate working space.
I’m going to take some time to digest all you’ve written, both in this thread and at your website. I’ve worked a long time with various geodetic reference frames; unless my dementia prevails, i should be able to eventually grok profiles…
Oops! I wrote the wrong thing: I already extracted the ICCs . However, that format isn’t human readable.
What I wanted to ask: how did you get to the point where you could compare the profiles with @Elle’s. I know I could probably examine dcraw.c and make-elles-profiles.c, but I am not a math or code wiz. It would be great if you, @Elle or someone more capable than I could guide me through the steps. Thanks.
@afre There are quite a few ways to extract information from an ICC profile in human-readable form (even including using a hex editor). Somewhere on this forum there is a thread that lists a lot of profile-related software. I think Rawpedia has a write-up.
The simplest way to turn a profile into something readable with a text editor is to use iccToXml:
SampleIcc Version 1.6.8, ICCXML Version 0.9.8
Usage: IccToXml src_icc_profile dest_xml_file
):
.
. However, that format isn’t human readable.