inverting photos of color negatives with darktable

[LaurenceLumi]

@ggbutcher

-g 1 1

This gives you the image in so-called “linear gamma”, an identity transfer function. Or, in formal terms, goezintas = comesoutas

Thanks glen.

Except it does not match the output from RT. Nor the output that I expect when I say compare it a similar result from a dedicated film scanner with a gamma of 1.

I would like to figure it out… I had a quick look at the rawtherepee code, even the matrix values in camconst.json are same as the those in dcraw. but yet the output is quite different…

I had a look through the man page etc, and nothing obvious comes up.

If you take the supplied files the OP provides, open in RT and save, then use something like imagemagick to remove the sRGB gamma, or just use RT to save the file using linear gamma sRGB profile. You will get a different result than what comes out of dcraw.

[ggbutcher (Glenn Butcher)]

I’ll have to do this when I get home, but I can do a “known baseline” with my rawproc program. If I set input.raw.libraw.rawdata=1, the opened image is the raw bayer data, devoid of any processing except what the camera itself might have done. With that, I’ll know the histogram of the baseline raw data, with which I can compare to the histogram of dcraw -W -g 1 1 -4.

[ggbutcher (Glenn Butcher)]

So, I first did:

dcraw -o 0 -4 -T DSG_3111.NEF

-o to eliminate the output colorspace, -4 is ‘Linear 16-bit, same as “-6 -W -g 1 1”’, and -T makes a tiff. Then I opened the NEF in rawproc with:

input.raw.libraw.rawdata=1

Which just loads the raw image array from the NEF. I then demosaiced and white balanced, but I had to do +2 stops of exposure compensation to make the histograms equivalent. Here are screenshots, first one of the dcraw tiff in rawproc:

Next is the NEF raw data, demosaiced, white balanced, and +2EV in rawproc:
Permission Denied

@patdavid, I was able to post my first .png, 1.4M, but it gave me the above error for the second, roughly the same sized .png.

Okay, I uploaded them to my server, here’s the NEF raw:

Note the histograms, and the processing in the NEF raw screenshot.

For a baseline, here’s the NEF raw after load, prior to any processing:

Interesting result. I thought -W in dcraw would do as it says, eliminate scaling. I’m wondering if -b (brighten, default=1.0) needs to be -b 0.0…

Edit: there is a slight difference in the histograms, it is white balance, dcraw did a better job than the camera multipliers. The general distribution of tones is essentially equivalent

[dani_l]

@LaurenceLumi

0.259,0.078,0.051 are normalised values of the film base. The red exposure is only 25.9% of the maximum so you have almost two stops to go before you clip… You can get these values from RT.

I have never really used RT before and I’m not sure how to get those using the color picker. Disabling the auto exposure helps a bit, but the values are still a lot larger (0.54, 0.31, 0.24). I’m guessing I have to change some color profile setting somewhere to get your numbers, but I’m not sure which ones.

[LaurenceLumi]

@dani_l

lot larger (0.54, 0.31, 0.24)

Your probably getting those numbers because, the linear file, has an ICC profile in it that it tells RT that it has gamma of 1. Then RT will give you values that are gamma corrected.

If I knew how to drive dcraw properly I would tell you the correct commands. A way around this if to open the .cr2 raw file in RT save it with sRGB profile that has a gamma of 1 as a 16bit tiff. The open the file tiff file in RT, and change the input profile to sRGB with normal gamma. It will then give values that match the intensities of the raw scan.

or in imagemagick just

convert 16bitsRGB.tif -set colorspace sRGB -colorspace RGB 16bitlineargammsRGB.tif

When I have some time I will have a look at this in more detail.

[anon41087856]

I have been considering adding Kodak Cineon scan system is darktable. It’s now an experimental module :

aurelienpierre/darktable

Results :

[Jade_NL (Jacques)]

Admittedly I don’t have any good quality b&w or colour negatives lying around, so this is based on a two (jpeg) downloads I could find on-line.

First impression: Very nice. Both colour and old-fashioned black and white negative.

As always one needs to get accustomed/familiar with the controls and find out where in the workflow you need to do stuff. What is the workflow in this case?

I initially didn’t do anything but turn on the module and had a look and go at the controls. Turned out that my example was somewhat underexposed which I can fix with the negative offset slider. Why is this set to 9% and should we try to correct the exposure before turning on this module (which would mean using the histogram).

Anyway: This would be a very big step compared to the current invert module…

[anon41087856]

@Jade_NL

As always one needs to get accustomed/familiar with the controls and find out where in the workflow you need to do stuff. What is the workflow in this case?

1. correct the camera or scanner colour using exposure, input profile and white balance (which, in this case, correct the light going through the film to the sensor), to get a scan properly exposed between 0 and 100%, and remove colour casts due to the sensor,
2. enable negadoctor, and correct:
   1. the Dmin, which is the colour of the orange mask or the density of the film substrate (a colour picker will be added soon),
   2. the Dmax, which is the paper density (same concept as the dynamic range but in dB/log10 unit, instead of the EV/log2)
   3. the WB coefficients (R, G, B) to correct colour casts due to the film emulsion, (colour picker planned here too),
   4. the negative inversion offset, to anchor the black level during the inversion process (9% by default comes from Kodak Cineon recommendations),
3. then adjust the printing adjustments to your taste.
4. denoise, sharpen, dehaze if needed.
5. finish with colour balance if you need more contrast or split-toning / warm paper effects.

@Jade_NL

Anyway: This would be a very big step compared to the current invert module…

The actual pixel operations performed are stupid simple, the whole code takes 5 lines. (But 540 lines of GTK UI madness).

[noahwc (Noah)]

Wow this looks awesome! I’ve recently started scanning negatives on an Epson flatbed and have struggled a bit with colour correction. This seems like a huge step up and something I’d definitely use. I’ll be sure to test it out with my next batch of scans.

[gaaned92]

trying to get something from sky with ART

[gadolf (Gustavo Adolfo)]

I couldn’t find the negadoctor module.
Steps I followed:
1 - git clone https://github.com/aurelienpierre/darktable.git
2 - cd darktable
3 - git submodule init
4 - git submodule update
5 - ./build.sh --prefix /opt/darktable --build-type Release
6 - sudo cmake --build “/home/gustavo/darktable/build” --target install – -j4
7 - renamed ~/config/darktable to backup my current database and start from fresh
8 - Started darktable as /opt/darktable/bin/darktable

[Claes]

Hei Rei!

Look for the branch negadoctor.

Have fun!
Claes in Lund, Sweden

[gadolf (Gustavo Adolfo)]

@Claes

Look for the branch negadoctor .

God morgon, Claes!

I suspected of that, but how does it translate in the building steps? I don’t know nothing about git …

[Claes]

Good info here:
https://rawpedia.rawtherapee.com/Linux#Choose_a_branch
Even if that deals with RawTherapee, the handling of git and branches works the same.

[gadolf (Gustavo Adolfo)]

Thanks, Claes, that did the job!

For further reference, the correct steps are:

1 - git clone https://github.com/aurelienpierre/darktable.git
2 - cd darktable
2.1 - git checkout negadoctor
3 - git submodule init
4 - git submodule update
5 - ./build.sh --prefix /opt/darktable --build-type Release
6 - sudo cmake --build “/home/gustavo/darktable/build” --target install – -j4
7 - renamed ~/config/darktable to backup my current database and start from fresh
8 - Started darktable as /opt/darktable/bin/darktable

[gadolf (Gustavo Adolfo)]

@anon41087856 negadoctor seems to be very promising.
I did a quick test on two negatives and it was quick and easy to get presentable results.
Color pickers will be great
Thanks for your work, perfect timing, I just started to digitize my negatives last week

[anon41087856]

The true beauty of that module is you get a fine control over the dynamic range, especially how much details you want to bring back from the shadows. I find it much better than the examples shown above and even Silver Fast, for that matter. But it’s only Kodak magic.

[gadolf (Gustavo Adolfo)]

@anon41087856

But it’s only Kodak magic

I’m curious about that.
Did Kodak shared the math behind the chemical transforms?

[anon41087856]

See references here : Improve the negative film inversion · Issue #4113 · darktable-org/darktable · GitHub

[gadolf (Gustavo Adolfo)]

Shouldn’t gain (500) be a gui parameter?
Or the film density it refers to, according to your description, is an invariant?