Scanned image scratch removal with “ICE”

Processing
1

I started becoming familiar with slide and negative film scanning yesterday with a mid-range scanner (unfortunately the used high-end Nikons are pricey these days, so I bought a reflecta a couple of years ago). Unfortunately, Linux support is not ready yet, so I use VueScan on Windows to obtain the image data. However, I want to do as much as possible of my workflow on Linux with FLOSS software. Therefore, I wonder if there is already some software, plugin or at least algorithm that can make use of the infrared channel data that is stored as alpha channel in the resulting tiff file to remove scratches.

Unfortunately, I did not find something on the web. I guess G’MICs inpainting algorithms could be helpful, but they are not exactly tailored to the problem or the input data format. Is there something I overlooked?

I can provide a sample scan later today if this helps.

Update: Sample file can be found here, but it’s large (150 MB).

2 Likes
2

Example images always help.

3

VueScan is an excellent piece of software and unfortunately has no free equivalent. It is one of the few non-free applications I recommend. It’ll run on Linux too!

I read as much as I could about RBGI scans (couldn’t find much) and it seems your best bet is to process it in VueScan itself.

If you can extract the infrared layer, you might be able to pull it into GIMP and use it as means to make a selection, then use resynthisizer’s wonderful Heal Selection tool.

1 Like
4

It is not that I dislike VueScan. I purchased it several years ago and what I like is that

  • it does the Job,
  • it has a lot of parameters to tune the result and
  • especially the licensing strategy is very healthy, I buy it once and I get updates for a lifetime.

However, what I dislike is that

  • it lacks documentation, especially for users that could understand what is happening technically,
  • it lacks support channels (on the support page there is a FAQ which is not really helpful and there is a support e-mail address, if the latter is responsive it is better than nothing, but a user to user help channel, e.g. a discourse forum, could help both sides; alternatively, Ed and team could answer the many partially or hardly answered questions in public places such as photography stackexchange), and
  • the UI is not what I would call intuitive (especially, better grouping of parameters would be helpful; better handling of mutually exclusive parameters or concepts would be beneficial; opportunity to switch off infrared channel for preview would speed up my workflow tremendously; etc.).

Unfortunately, I own one of the scanners (Reflecta CrystalScan 7200) for which VueScan relies on the OEM drivers to access the hardware, which are not available for Linux. Recently (last year or so), untested Sane support for the scanner was announced, but I never got it working and asking on the mailing list I got no useful reply.

That’s my backup plan.

Yes, it’s the alpha channel of the tiff file. I already thought a lot about how to work with it, but have to analyse its properties a bit more to find some useful algorithm.

5

I always got the feeling (no tangable proof) that “Ed and team” is mostly just Ed.

I’ve purchased the lifetime license like 3 times; its that good to me.

6

@chris

Assuming the alpha/infrared channel separates scratched areas well from image data (i.e. contains exclusively a scratch “mask”) I’d be surprised if G’MIC can’t do the required. It’s very simple to turn an opacity channel into an inpaint mask. There are plenty possible inpaint & blending methods. An example would really help - even just a small portion of an image would probably do.

7

I believe I still owe @Morgan_Hardwood an example RGBI file that I promised many months back, I’ll see if I can find time to provide one.

8

I just updated the original question with a link to a sample file, but it’s large (150 MB).

9

The infrared channel looks pretty good - faint ghost of the image and scratches reasonably dark. So it should be possible to do something with a threshold and inpaint.

Not certain if such a filter already exists in G’MIC but the commands to make it should be straightforward… I’ll have a quick go using command line.

2 Likes
10

@chris

It does seem promising, the machine I’m on just now is old and slow so was forced to downscale. Here’s a command line which worked reasonably:

gmic -i raw0003.tif -r 40%,40%,100%,100%,5 -sh 0,2 -sh.. 3 -lt. 84% -dilate. 3 -inpaint_diffusion.. .,,,12 -k... -channels 0,2

It’s possible patch-based inpaint would work well but frankly I don’t have all night to wait for it to complete :smiley:

Basically the above is:

Rescale to 40%
Threshold the infrared channel to 84% or less
Make the selected areas a bit bigger
Inpaint using the mask
Discard the infrared channel

1 Like
11

Great, @garagecoder. I wonder what would be better, to do scratch removal before or after editing the image. Especially, if before, for the negative or the inverted image (the latter may not be possible, e.g. when inverting is done with another software, darktable in my case).

And, I wonder if the inpainting method should be decided on the connected size and shape of the scratch: For small or thin scratches, a very simple and fast method such as linear interpolation would suffice, for medium size scratches, a structure preserving inpainting would be suitable, but for large scratches, the patch-based inpainting would be beneficial.

Edit: Oh, you have been faster. I’ll check your post, but testing may have to wait till tomorrow (real life, familiy, etc. :slight_smile:).

12

Good points/questions actually. If you inpaint before editing the artifacts from inpainting may become more noticeable than the scratches themselves. The infrared channel can easily be extracted from the original and used with the processed version as a mask.

Inpainting could indeed be decided on defect area using G’MIC -area command, again relatively simple to do just by a threshold and run each resulting mask through a different method.

Edit: I know exactly what you mean, I’m already spending time I don’t really have :smiley:

1 Like
13

Me too, but what ennobles (is it the right word?) you is that you are spending your time on my Problem :grin:. I appreciate it very much! (Luckily, It’s time you don’t have, so it’s non-existent and no problem at all ;-).)

14

Hm, I guess it’s time to learn a bit of G’MIC. I tried to use your command in the “Custom code” filter in the GIMP plugin but get a “ssignement request of shared instance from specified image (2131,1378,1,4).” error.

Anyway, I’ll try to analyse and understand the command (in my own words):

gmic -i raw0003.tif  # I skip this part because it's inside the GIMP plugin
  -r 40%,40%,100%,100%,5  # resize to 40 % with all additional
                          # dimensions and channels untouched using
                          # bicubic interpolation
  -sh 0,2  # place rgb channels on the "stack"
  -sh.. 3  # place alpha channel on the "stack"
  -lt. 84%  # boolean "less than", every value below 84 % will cause 1
            # I guess inpainting requires "white" resp. "1" pixel mask
  -dilate. 3  # dilate with large block
  -inpaint_diffusion.. .,,,12  # Inpaint on second image on the stack
                               # with first (binary) image as mask
  -k...  # Keep what? What is in [-3]?
  -channels 0,2  # Remove alpha channel

So there are still questions: Why should we keep [-3], it should contain the original image? Or does inpaint place the result behind/above the image it is applied to? Or does it cause an empty image on the stack, so [-2] is shifted to [-3]?

15

Ah OK, the custom code filter can be used for a quick test but has some issues compared to a complete filter or command line (can’t rescale, access to only one layer at a time…). Also an alpha channel in gimp can’t be removed by G’MIC only added. So you could use as a test:

-r 40%,40%,100%,100%,5  # resize for speed
-sh 0,2  # create a new shared buffer from the image using channels 0 to 2
-sh.. 3  # likewise for channel 3
-lt. 84% # less than 84% of max value becomes 1 (inpaint will fill >= 1)
-dilate. 3  # take the max value from a 3x3 square around each pixel of the mask
-inpaint_diffusion.. .,,,12  # inpaint rgb using alpha
-f. 255  # fill alpha with max value (assuming 0 - 255)
-k...  # keep only the original image, discard the two shared buffers

I probably shouldn’t have used shared buffers in an example, they can be hard to explain/use. It’s simply to save memory instead of taking an actual copy of each channel range (since it’s quite a big image).

1 Like
16

Thanks for the explanation, now I understand better. Is there a difference between outputting the RGB channels of the original image and outputting the shared buffer [-2], which is the RGB channels of the original image?

I experimented a bit with the image and these are my findings so far:

This is a crop of the original image, upper left corner (it’s a negative, so colours are inverted):

1599×1034 308 KB

Its alpha channel (the infrared channel) clearly exhibits some of the scratches in the original, especially a big blop in the upper right, some sprinkles in the centre and a long horizontal line-like scratch in the lower part, the slight scratch in the upper part seems to be not visible in the infrared image:

1599×1034 176 KB

The threshold operation leaves a lot of “noise”, maybe it is too high:

1599×1034 775 KB

The dilation adds a lot of the noisy region to the mask:

1599×1034 822 KB

This is handled very well by the inpainting algorithm, the only thing it has (little) problems with is the long horizontal scratch (only visible at 100% view):

1599×1034 268 KB

However, I thought getting rid of the “noise” would be beneficial to speed up inpainting and to prevent the whole image to be reconstructed, so I inserted an erosion step before dilation, which gets rid of too little sprinkles. The following is mask and result with -erode. 2 -dilate. 4:

1599×1034 290 KB

1599×1034 297 KB

The same with -erode. 3 -dilate. 5:

1599×1034 92.6 KB

1599×1034 302 KB

And another one with reduced threshold of 82 %, rest as before:

1599×1034 29.6 KB

1599×1034 303 KB

Seems the key parameters for a good mask are threshold, erosion and dilation size (OK, looks obvious, but at least we have reasonable starting parameters).

5 Likes
17

Nicely done! Very helpful to show what’s happening in each step :slight_smile:

It depends which form of output - if you mean a file, nope no difference so you could indeed miss out the “-channels 0,2” at the end of the command line version and just output the shared buffer. Obviously the shared buffer can only exist while the image it’s shared with exists, meaning you can’t remove the original and leave gimp with just the shared version - if you try to do so you’ll most likely get an error.

If with larger images you end up using large values with -erode and -dilate the mask can end up “blocky”. Alternatives might be -dilate_circ, -distance (with a threshold) or -blur (also with a threshold) to give rounder/smoother edges.

1 Like
18

While the concept of shared buffers is clear, I still have problems to figure out how the GIMP plugin interacts with the GIMP layer(s). Is there some documentation available (I was not able to find docs regarding the plugin, only language docs and tutorials)?

This is the size I need, because it’s the maximum my scanner can handle (approx. 18 MP), and I wonder where even bigger values would be needed. The erosion gets rid of single pixel errors (or slightly larger) which are not visible anyway, inpainted or not, and noise, and the dilation adds back the removed edges from larger areas and adds a little margin. So I might end up using dilate_circ becaus the areas to be inpainted will be slightly smaller and smoother at the corners, but I guess I will hardly go beyond a size of 5 (or erode_parameter + 2). With 3/5, the inpainted areas appear way bigger than the scratch size, if you compare directly.

Now I have to figure out how I get G’MIC into darktable, Christmas is approaching very fast ;-). To become serious again, the biggest problem I see with doing this at the end of the processing is that the crop I do in $RawProcessor will cause an alignment problem afterwards. But that’s a different story (unless you accidentally have a solution :wink:).

By the way, my gmic command did not accept “.” as parameter for the inpainting, only “[-1]”. Maybe a quoting problem.

19

If there are docs specifically about G’MIC/gimp layer interactions I haven’t seen them. Perhaps when I have more time I can write some on the wiki. It’s mostly straightforward (each layer becomes an image on the stack) but there are quirks; the non-removable alpha channel being one, another related to opacity 0 areas not being copied back to gimp - needs to be a gimp bug report really.

The dot image parameter issue could be down to version, it was only introduced recently. If not as you say could be escape character related.

20

Not sure if this will help but rather than creating layers in the GIMP I sometimes use Fotoxx. It uses them too but hidden from the user. One of the noise removal tools goes by the name of a top hat filter. Playing with the rad killed the noise. I then reduced size by 2/3. It has another feature - brightness distribution. I flattened that by a lot and use the deband option. Then used it’s tone mapping heavily to bring the contract back up from the flattening. Sounds odd but is useful. I allowed it to work on high contrast. Then the combo tool, curves, saturation and colour balance plus some work on the blue curve, lowering the dark end. Also tried spot white balance but no idea what it should look like.

Then in the GIMP. Wavelet decompose to try and get rid of the black line. Might be possible but will takes some time. I’d wonder if it was possible in this case to simply cut it out. Something like 2 layers one part transparent no line overlapping the line and then scale to make the joins match,

Anyway result on noise removal and some of the line toned down. Few spots left but they would heal easily.

3552×2296 3.58 MB

Fotoxx is a Linux package - :slight_smile: Aren’t we lucky.
John