RawTherapee vs Darktable: invert scanned B&W negatives

Hei Claes,

sorry, doesn’t work.

If I click on the pipette the picture immediately turns blue…

Thx,

Alex

Hi Alex,

Hm… let us try to dissect this problem, one step at a time.

  1. You did manage to get a new “pipette rectangle”, right?
  2. Using darktable, please apply the enclosed dtstyle to your neg scan image.
    Hopefully you will be able to use the style as a starting point for your
    further experiments.
    negScan.dtstyle (2.4 KB)

Have fun!
Claes in Lund, Sweden

… and here are almost the same steps for RawTherapee:
fa24f9eb4277e1ee9003824cccb4d4eb0eda1db5.tif.pp3 (10.3 KB)

Have fun!
Claes in Lund, Sweden

Linear inverted tone curve, for use in inverting the colors of a scanned negative, as described in RawPedia: http://rawpedia.rawtherapee.com/Negative

linear_inverted.dcp (954 Bytes)

I tried your image and I was able to pick a rectangle on the left border which I assume only contains orange mask. Your picture is a bit tricky since it has different areas on the border which I don’t know what they are, but since you own the negative you should know which part is really the orange mask.

What I found difficult as well is that I had to tweak the exposure a lot for every of the areas. I tried all of them, but none seemed to be a good reference.

In the picture you can see (it’s a bit difficult to observe) a rectangular frame on the left side, where I used the colour picker of the invert module to select an area. Just hold your mouse button while you are “painting” the rectangle. The white balance is a bit off, but in principle it works.

image

Please, don’t do that, you will loose a lot of flexibility and VueScan’s scratch removal algorithm is, let’s call it, basic. I did a lot of tests and was able to get so much better results with G’mic (see this thread: Scanned image scratch removal with “ICE”, especially the later results).

1 Like

@Chris
I completely agree that the dust and scratch removal in VueScan is not ideal. In fact, for the strong application it does not really create a mask but instead seems to scale and subtract a thresholded IR-image from the RGB-channels. You can see this, if you subtract two image versions from the same original, one with scratches removed, the other not. In the difference image you can see all the faint structure in the IR-image, e.g. stripes along the scan direction:


For a setting of low or medium there obviously is a threshold set in the IR-channel, since the stripes are not seen there. Using a real mask does not show these structures, e.g. as SilverFast iSRD does. For Kodachrome slides the VueScan method is just not applicable since it corrects everywhere in dark areas, considerably reducing the contrast.

Thanks a lot for the discussion of the G’Mic script to remove scratches with the IR-channel. I just want to understand what inpaint exactly does. I will have to look at the manual.

Concerning removal of the orange mask:
How does one set shadow and highlight points in the histogram individually for each channel in RT?

Hermann-Josef

This may not specifically help with RT, but it illustrates what’s going on with the black-white point reconciliation. In that respect, my rawproc software may not be ‘production quality’, but I’ve found it to be a decent platform for tutorial development.

So, starting with an innocuous color negative/positive image I found on the internet:

First, you can see the color shifts in the individual channels of the histogram, but they’re not really indicative of the image proper until we crop to it:

Now, the nature of the color cast becomes evident, particularly in the intensity offset of the red channel. The black and white limits of each channel are also evident; for this particular image, all three blacks appear to be at about 40 on a 0-255 scale (internally, rawproc is working on 0.0-1.0 float, but the histogram and tone operators work on 0-255 for convenience). With rawproc, we’re going to now add three curve tools, one for each channel. First, red:

Note what I’m doing is to scooch the black and white control points along the bottom and top bounds of the curve box until the data starts to pile up at the respective side. Now, green:

And blue:

Finally, the rgb curve to invert the negative to positive:

There’s a lot of writing on the internet about knowing the film stock to do this conversion, but the compelling thing about just normalizing the black and white points is that it doesn’t depend on such consideration. Also, once you become familiar with the dynamics of the adjustment, you can also handle white balance in the same operations; however, I’ve found that just adding a white balance tool after the conversion is more intuitive.

I used curves in the above missive to illustrate, but rawproc has a separate black/white point tool with an auto-calculate option that i would use instead. With that tool, I can do color negative conversion with my img command line program like this:

$ img neg.jpg blackwhitepoint:red blackwhitepoint:green blackwhitepoint:blue curve:255,0,0,255 pos.jpg

I’ve got a box full of 35mm color negatives from my mother, pictures of people long gone whose names I don’t know, so I’ll be doing this in volume pretty soon in order to get images in front of folks who still know the names. My plan is to shoot the negatives raw with my D7000 and macro lens, and process the images with the above command.

1 Like

@ggbutcher
Hi, thanks for describing your method for negative inversion. This showed me, by the way, the answer to my question on how to set the shadow and highlight points in RT for the three channels :slight_smile: .

As far as I understand the white balance algorithms, the most common use is the grey world approach, which aims at the same “centre of gravity” for the three histograms in R, G and B. Setting the white and black point accordingly is what I usually do, just like you described.

The problem is, that you need a patch on the negative, which is not exposed, i.e. exhibiting the clean orange mask. Otherwise it is only an approximation what you get.

Hermann-Josef

PS: The method for inversion used by ColourPerfect is described in a paper by David Dunthorn.

Edit: A patch from the sprocket edge would be a correction similar to the black/white point operations (original post was flat wrong), and really wouldn’t address white balance on the residual positive image.

Edit1: So, my thinking is that white balance needs to be done on the positive image, as the negative conversion just deals with the color cast and inversion. Film stocks were formulated to capture colors appropriate to the incident light’s color temperature, so that difference needs to be handled subsequently. You can do each image with a patch, but a good starting point that’s image-agnostic might be just a full-image ‘gray world’ calculation. I’m programming a whitebalance tool for rawproc now, and I’ve been surprised at how well full-image gray world actually works.

1 Like

@Jossie
That particular inversion method has been mentioned here previously.

All it boils down to is negate with 1/x instead of 1-x (or equivalently use negative gamma). There is of course some extra information about colour balance and choosing the particular gamma values.

I haven’t scanned a lot of colour negs, mainly because I don’t have many to work with. I have been able to get decent results, however, using a relatively simple method with RawTherapee on DSLR scans. Here’s a sample from a mid 1990s 6x7 Fuji neg, cropped for the privacy of the models. Basically just inverting using the curves, getting a white balance of the orange mask, then adjusting the colour temperature and tint. In this case, the light source was electronic flash through a diffuser.

I would appreciate for documentation purposes if someone could send me a color negative, preferably with the following requirements:

  • The negative’s colors are in good condition, not like in the Pepsi-Cola slide.
  • It can have dust and scratches, that’s fine.
  • There is something white or neutral in the frame, like the three blocks in @ggbutcher’s image from the net.
  • There are colors which a human would pick up as clearly wrong if they were off, such as the skin tones in @troodon’s scan above.
  • You own the slide, so you can release it under CC0.
  • The negative is available in 16-bit format (TIFF/PNG/DNG) and/or as a raw file.

I can send you the Nikon .NEF file of the two girls. It has good skin tones but no whites that are completely neutral. the blacks and grays in the gloves might work though. The original was a Fuji pro negative film (can’t remember which one) and I scanned the negative with a Nikon D7100 using diffused electronic flash.

@troodon I don’t think I can use that without a signed release from them, so we would need a different photo, one without faces.

@Morgan_Hardwood
just recently I have made scans with SilverFast and VueScan available for the community to test and compare the software. You can find the three “raw” files here (Kodachrome, Ektachrome and Kodacolor X). I could supply other scans of negatives should the horse image be not ideal.

Hermann-Josef

@Jossie great!

Why are the horse scans from CanoScan, SilveFast and VueScan so different? Which one would you consider the typical RGB no-bells-or-whistles version?

@Morgan_Hardwood
First of all, there are hardware differences: In the folder “CanoScan” the scans were done with my flatbed scanner CanoScan9950F using VueScan. Folders “SilverFast” and “VueScan” are the scans with my slide scanner DigitDia6000 with the software specified by the folder name. Folder “CyberView” are the scans from the slide scanner obtained with the CyberView software.

The “raw”-scans also have a different format. VueScan saves 4 channels/pixel whereas SilverFast produces a 3-page TIF: full resolution RGB, reduced resolution RGB and IR scan. The scans are not gamma-corrected, thus are dark. Only the CyberView-version is gamma-corrected since this software does not produce “raw”-scans.

Looking into this I realized, that VueScan seemed to have filtered the orange mask out already. This was not my intension, on the contrary! I will check and replace the image. Thanks for pointing this out!

Let me know if you need negative scans with more colours.

Hermann-Josef

@Morgan_Hardwood

The VueScan-image has now been replaced.

Hermann-Josef

Dear all,

sorry for the late reply, had been travelling a lot…

I now understand my misstake: I selected an EXPOSED area instead of an UNEXPOSED. I think I mixed it up because in Vuescan you need to take the exposed area.

Nevertheless I was playing around with colorPerfect, what gave me quite good results in quite little time. I am trying to find a solution to find a way with a good quality-to-time-ratio and colorPerfect seems to deliver it. Even with the picture chris was working on (indeed the film seems to be misshandeld by the lab) I could minimize the frustration and dissapointment.

Anyway - Thank you very much!
Alex

Alex

1 Like