Relationship between 'neutral' profile and Dcraw ? Trying to get linear TIFF

calibration
rawprocessing

(Philip B) #1

Hello,

Im hoping someone can help me understand what settings the neutral profile is calling from Dcraw. Is there any additional adjustments associated with the profile, or does it pretty much result in a direct Dcraw output?

I’m interested in making calibrated HDRI for CG lighting. This requires Linear TIFF files, with no/minimal toning introduced by the raw processor. Many use Dcraw to extract TIFF’s without introducing any toning, and thus preserving linearity. More info in the article below. se section 5.2, on pg.46. Interesting stuff.

http://blog.selfshadow.com/publications/s2016-shading-course/unity/s2016_pbs_unity_hdri_notes.pdf

Can anyone give me a sense of what the neutral profile includes?

Thanks…


(Ingo Weyrich) #2

I Colour Management tab there is a button Save Reference Image for Profiling. This could be the one you’re looking for :wink:


(Philip B) #3

Oh, that might be helpful.

Looks like a gamma 1.0 image if im not mistaken?

Now I would still ask what are the differences or similarities between Dcraw settings, Save Reference Image for Profiling, and the ‘neutral profile’.

Thanks again!


(Ilias Giarimis) #4

The pdf says that you need linear RGB with gamma=1 and sRGB primaries at D65. With RT, start with Neutral profile and then choose RT_sRGB_g10.icc as output profile… at tab color last menu down. This gives exactly what the pdf asks.

The reference image for profiling has no color correction applied (same as if you choose “no profile” at input profile menu), and (optionally) no WB. And no convertion in any output colorspace (no sRGB primaries).


(Philip B) #5

Interesting…

so, RT_sRGB_g10.icc is a lut that puts in image in sRGB space and returns the image to Gamma 1.0?

And so the ‘input profile’ (of which i often make my own with a macbeth and adobe DNG profile editor) is in a sense a transformation matrix for color? Then having no profile implies what? Im asking this question kind of independent of the HDRI creation? White Balancing neutralizes the color cast of the lighting. The custom profile transforms the color matrix to most accurately represent known values of the macbeth chart. So what does no profile give you? I hope this question makes sense.

Thanks for your time. Im learning a lot, a do appreciate your taking the time to explain it.

Also, from the above I can then assume that the neutral profile does not tone the image in any sense? It just crops dead pixels from the borders, applies gama and white balance, and thats all?

It would be really helpful to know exactly what processes are applied to the raw file when it is loaded into RT and the ‘neutral profile’ is selected.


(Ilias Giarimis) #6

Yes, but I would put it a bit differently … “and it doesn’t apply any gamma encoding”

Basically yes, … if we want to be exact the “camera standard” option is a pure matrix transformation, while the “custom” dcp or icc have more steps i.e. apply a matrix transformation and add upon this correction LUTs for cases where the matrix transformation is innefficient
.[quote=“PhilipB, post:5, topic:2842”]
Then having no profile implies what?
[/quote]
No color correction, no gamma correction, no color management … only WB is applied. It is basically same as the “reference image for profiling” … the other than profiling cases that this may be usefull are

  • cases where we have clipping in extreme lighting (mainly clipping of neon/led blues) where the color transformation fails and
  • as a workaround if color profiles are missing or wrong for a model … use “no profile” and working space set to sRGB and insert in Channel Mixer a 3X3 matrix (borrowed from DxOMark measures or from exif - this last works well for Olympus cameras … ) …

No toning … just WB, demosaic, color transforms from raw to internal working space then to output color space, gamma …

If we again speak (write) strictly … a clipping of Red/Blue values is inherent in the WB step … depending on the WB multipliers, some Red and Blue values will be clipped or else we take magenta casts at highlights …
Let’s say our range is 0-65535 (16bit) and the WB multipliers R=2.0 B=1.4 then any raw red value higher than 32767 will be clipped, any raw Blue higher than 46988 will be clipped …
RT handles the higher than 65535 values so we can recover some highlights by altering WB … but I think this is not possible after outputting to integer tiffs/jpegs


(Philip B) #7

Thank you so much for taking the time to explain…very illuminating. I really appreciate it.


(Glenn Butcher) #8

Regarding dcraw, I’ve found this tutorial quite useful:

http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm

You get a linear 16-bit image like this:

dcraw -4 -T file.NEF

To get rid of colorspace conversion and white balance correction, you 'd add the following:

dcraw -o 0 -r 1 1 1 1 -4 -T file.NEF

which will give you a basic demosaiced linear image. The only thing ‘missing’ so to speak would be -D, which would not even demosaic and just encode the raw sensel measurements, but I don’t think you want that. Interesting to do, however…


(Philip B) #9

Ohh, thats an excellent resource.

Thanks!


(Morgan Hardwood) #10

@ilias_giarimis this information should go into RawPedia. Is there any more info on the subject you would like to add?


(Iulian Drug) #11

Hi @ilias_giarimis

I tried to do exactly what you recommended @PhilipB both with dcraw and RawTherapee. Unfortunately there is a difference in between the images - even though I chose the neutral profile and then the output RT_sRGB_g10. But the one created with dcraw [-v -4 -T] looks more saturated / wb is different.

I am asking because I need to use the dcraw workflow, but it takes ages to batch 100’s of photos. I just found rawTherapee and it seems quite a tool.


#12

@Iulian_Drug Familiarize yourself with dcraw's options. I normally use dcraw -v -w -H 2 -4 -T. I have copied the dcraw usage notes for your convenience:

Raw photo decoder "dcraw" v9.27
by Dave Coffin, dcoffin a cybercom o net

Usage:  dcraw [OPTION]... [FILE]...

-v        Print verbose messages
-c        Write image data to standard output
-e        Extract embedded thumbnail image
-i        Identify files without decoding them
-i -v     Identify files and show metadata
-z        Change file dates to camera timestamp
-w        Use camera white balance, if possible
-a        Average the whole image for white balance
-A <x y w h> Average a grey box for white balance
-r <r g b g> Set custom white balance
+M/-M     Use/don't use an embedded color matrix
-C <r b>  Correct chromatic aberration
-P <file> Fix the dead pixels listed in this file
-K <file> Subtract dark frame (16-bit raw PGM)
-k <num>  Set the darkness level
-S <num>  Set the saturation level
-n <num>  Set threshold for wavelet denoising
-H [0-9]  Highlight mode (0=clip, 1=unclip, 2=blend, 3+=rebuild)
-t [0-7]  Flip image (0=none, 3=180, 5=90CCW, 6=90CW)
-o [0-6]  Output colorspace (raw,sRGB,Adobe,Wide,ProPhoto,XYZ,ACES)
-o <file> Apply output ICC profile from file
-p <file> Apply camera ICC profile from file or "embed"
-d        Document mode (no color, no interpolation)
-D        Document mode without scaling (totally raw)
-j        Don't stretch or rotate raw pixels
-W        Don't automatically brighten the image
-b <num>  Adjust brightness (default = 1.0)
-g <p ts> Set custom gamma curve (default = 2.222 4.5)
-q [0-3]  Set the interpolation quality
-h        Half-size color image (twice as fast as "-q 0")
-f        Interpolate RGGB as four colors
-m <num>  Apply a 3x3 median filter to R-G and B-G
-s [0..N-1] Select one raw image or "all" from each file
-6        Write 16-bit instead of 8-bit
-4        Linear 16-bit, same as "-6 -W -g 1 1"
-T        Write TIFF instead of PPM

#13

Thanks @Morgan_Hardwood for the edit. I wrote the reply very quickly. What did I do incorrectly?


(Morgan Hardwood) #14

@afre when using backticks the forum tries to guess the language of the code. In this case it’s not code, so I forced the language to plain-text by doing

```text

Color balance question