Scanned image scratch removal with “ICE”

chris · October 25, 2016, 4:54pm

During my vacation I had little time but at least I was able to do a comparison of the inpainting methods provided by G’MIC. I did not optimize the parameters but used the default configuration, except for -inpaint_diffusion, where I used the parameter given by @garagecoder, and one other case, where no default values are available. I compared at two areas of the image, a line scratch segment and a large dot-like scratch. These places can be clearly seen on the used mask:

The command line is the same for all, but the inpainting option (and output file name) was changed of course:

gmic -i raw0003.tif -crop 0%,0%,30%,30% -sh 0,2 -sh.. 3 -lt. 82% -erode. 3 -dilate. 5 -inpaint.. [-1] -k... -channels 0,2 -o test_.tif,ushort

The results, upscaled without interpolation, clearly show a “winner” algorithm . First the results:

Numbering the upper results 1 to 3 from left to right and the lower ones 4 to 6, I think in both cases 6 is the winner and 5 on the second place, but with quite some distance. The others are almost equally bad. The main problem is IMHO that the other algos tend to shift/stretch outer values towards the mask center, which leaves a visible artefact in form of “columns” for the line and a “cone” for the dot-like scratch. I guess you already know which class the winner algos are belonging to, the used command for the 6 cases are:

-inpaint
-inpaint_diffusion
-inpaint_flow
-inpaint_morpho
-inpaint.. [-1],5,15,0.5,1,9,0
-inpaint_patchmatch

garagecoder · October 25, 2016, 6:14pm

I’m sure many will appreciate the examples you’ve shown here

The results more or less follow the complexity of the algorithm and likely processing time, but where result is most important that’s no issue. Version 5 (a patch based inpaint) can be difficult to tweak given the number of parameters.

chris · October 25, 2016, 7:46pm

I think complexity is one side, but the big difference is between patch-based and “others”. Since 6 is better than 5 out-of-the-box, I would not tweak parameters of 5 but just use 6, since it is already that good. Maybe if speed is a concern, as you are saying, one may use one of the other methods, but in most cases that will not be necessary and for me definitely not.

More important and time consuming is to tweak the mask, because a good mask can hardly be judged by an algorithm. Luckily, computation time for the operations is very low .

I still wonder what algorithms are implemented in proprietary software such as VueScan, Nikon, Canon or SilverFast. I only have VueScan, and there scratch removal is really fast, but I did not check the quality of the results. It has only one Parameter “strength” with 3 different choices.

chris · October 25, 2016, 9:50pm

Hm, I tested the same with VueScans infrared cleaning, strongest setting, and the results are extremely different:

If I try to ignore the compression artefacts (I should have saved as tiff) the problematic area is still visible, I would say better than 1 to 4, but worse than 5.

The line was hardly removed at all. At that point, I am so happy about having the opportunity of using FLOSS software !

chris · October 31, 2016, 9:17pm

I had the time to make a little filter for the GIMP G’MIC plugin from the findings above. Maybe this is useful for somebody else.

#@gimp ICE : ice, ice(0)
#@gimp : note = note("Scratch removal for scanned film images")
#@gimp : sep = separator()
#@gimp : Threshold = float(82,0,100)
#@gimp : Erosion = int(2,0,5)
#@gimp : Dilation = int(4,0,7)
#@gimp : Show preview after = choice("threshold","erosion","dilation","final image")
ice :
-sh 0,2
-sh.. 3
-lt. $1%

-if {$4>=1}
  -erode. $2
-endif

-if {$4>=2}
  -dilate. $3
-endif

-if {$4<3}
  -k...
  -channels 3
  -* 255
-else
  -inpaint_patchmatch.. [-1]
  -k...
  -channels 0,2
-endif

chris · October 31, 2016, 9:23pm

Hm, I was too fast, there’s a little problem with the preview, it seems to scale/normalize the input values depending on the image data that is actually shown. @garagecoder, any advice?

chris · January 3, 2017, 8:29pm

@David_Tschumperle, since you recently mentioned the preview possibilities in another thread, I try to address this directly to you. Given above G’MIC GIMP script (2 posts above), the preview of intermediate results is changing heavily depending on the area of the image shown. What is the reason? Is any parameter relative to the input data (I thought this is not the case)?

And another question, is there a simple possibility to make a standalone GUI program out of this? I don’t want to have to load all the images, one by one, into GIMP to find reasonable parameters. The processing is done on the command line anyway, but selecting the preview viewpoint works much better in the GIMP filter, and pulling sliders with direct feedback from the preview is much closer to the nature of the problem than iterating by changing numbers on the command line.

David_Tschumperle · January 3, 2017, 10:44pm

I’m not so surprised by the erratic behavior of the preview, as having an accurate preview for a filter is a very hard task to be honest. All depend on how your filter is coded actually, and what kind of information it uses to render its result.
If it requires only local image information, then the preview should be relatively accurate when the preview zoom factor is set to 1:1 by default. It it requires only a global view of the image, then the preview should be not so bad with a full view zoom factor. If it requires both, then it will be a mess.
The GimpPreview widget is able to give us only a small image for computing the preview either as a global view of the image, either as a local view of the image. But this is often not enough to get an accurate preview, for complex filters that requires local/global information, e.g. the min/max values of the whole image, as well as an estimate of the XY derivatives at the same time.

Concerning the stand-along GUI. Yes, this is planed.
We have started to work on an Universal plug-in done in Qt, which should be also working as a stand-alone application. This is planed for the next release 1.8.0, probably in a few months.

garagecoder · January 3, 2017, 10:59pm

I can hardly wait

chris · January 3, 2017, 11:03pm

It requires only the 1:1 view. E.g., if it shows the mask (only black and white values), it seems that the magnitude scaling (black/white point) when the threshold is computed depends on the area shown. It works correct with different areas on the command line.

I uploaded the test file again if you want to have a look: https://filebin.net/gqc36e26a63vmuin/raw0009.tif (150 MB).

Great news. I read about it here already, but did not know about standalone capabilities.

garagecoder · January 3, 2017, 11:22pm

@chris
The problem is the combination of view area and “-lt $1%”. Using a percentage means base it on min/max values in the area supplied to the filter.

You could either base it on a fixed min/max or you could make the preview full image, but neither are likely to be perfect anyway. Perhaps there’s some other way I haven’t thought of though.

chris · January 4, 2017, 6:05am

Hm, but on the command line the “-lt x%” seems independent of the actual input data range but in GIMP it seems not. What’s the difference and how can I achieve identical behaviour, such that I can reuse the parameters found in GIMP on the command line?

David_Tschumperle · January 4, 2017, 6:46am

@garagecoder is right, you have this problem here

The -lt. $1% is intended to depend on the min/max values of the whole image, which means that in the preview you’ll compute the threshold uniquely based on the min/max values of a cropped region. Not good or an accurate preview, if you zoom only on very dark or bright regions for instance. I suggest you use an absolute value instead of a percentage (the RGB input image has always values constrained to be in [0,255], so -lt. {$1*255%} should be great here.

chris · January 4, 2017, 7:21pm

Thanks for the suggestions, @garagecoder and @David_Tschumperle, it works well in GIMP G’MIC. Here’s the updated code for Version 1.8.0:

#@gui ICE : ice, ice(0)
#@gui : note = note("Scratch removal for scanned film images")
#@gui : sep = separator()
#@gui : Threshold = float(72,0,100)
#@gui : Erosion = int(2,0,5)
#@gui : Dilation = int(4,0,7)
#@gui : Show preview after = choice("threshold","erosion","dilation","final image")
ice :
-sh 0,2
-sh.. 3
-lt. {$1*255%}

-if {$4>=1}
  -erode. $2
-endif

-if {$4>=2}
  -dilate. $3
-endif

-if {$4<3}
  -k...
  -channels 3
  -* 255
-else
  -inpaint_patchmatch.. [-1]
  -k...
  -channels 0,2
-endif

Feel free to include this into G’MIC if you think it could be useful for other people. Maybe better change the name then, ICE may be trademarked in the context of scratch removal. Since in Germany ICE is a type of high speed train and G’MIC is a french product, I would suggest “TGV scratch removal” ;-).

Edit: If you decide to do so, I of course would contribute documentation on the whole process.

David_Tschumperle · January 4, 2017, 8:13pm

Hello Chris,

I’ve worked a bit on your code, to make it more clean :

I’ve added a Split preview option.
I’ve separated the preview process and the final process, so that you always get the final result when you actually apply the filter (even when the Preview as option is different).
I’ve added a loop to make your filter work with multiple input layers (and do not crash when input image is not in RGBA).
I’ve added a credit note at the end of the filter.

That’s how the code looks now, feel free to suggest any modification you would like to see:

#@gui Remove scratches : fx_remove_scratches, fx_remove_scratches_preview(0)
#@gui : note = note("<small><b>Note:</b> Scratch removal for scanned film images</small>")
#@gui : sep = separator()
#@gui : Threshold = float(72,0,100)
#@gui : Erosion = int(2,0,5)
#@gui : Dilation = int(4,0,7)
#@gui : Show preview after = choice(3,"Threshold","Erosion","Dilation","Final image")
#@gui : sep = separator(), Preview type = choice("Full","Forward horizontal","Forward vertical","Backward horizontal","Backward vertical","Duplicate top","Duplicate left","Duplicate bottom","Duplicate right")
#@gui : sep = separator(), note = note("<small>Author: <i>Chris/Pixls.us</i>.      Latest update: <i>01/04/2017</i>.</small>")
_fx_remove_scratches :
  -repeat $! -l[$>] -to_rgba
    -sh 0,2
    -sh.. 3
    -lt. {$1*255%}
    -if {$4>=1} -erode. $2 -endif
    -if {$4>=2} -dilate. $3 -endif
    -if {$4<3} -k... -channels 3 -* 255
    -else -inpaint_patchmatch.. [-1] -k... -channels 0,2
    -endif
  -endl -done

fx_remove_scratches :
  -_fx_remove_scratches ${1-3},3

fx_remove_scratches_preview :
  -gui_split_preview "-_fx_remove_scratches $*",$-1

I’ve put this filter in Testing / Chris - pixls.us. It should be really available after a filter refresh, with G’MIC plug-in 1.8.0.

chris · January 4, 2017, 9:25pm

That looks great, I already checked in GIMP. Nevertheless, it seems odd for me to claim authorship, it is mainly garagecoder’s and your work. However, I am going to take care of the filter and will start over with writing a proper tutorial the next days (or maybe some weeks, depending on other workload).

garagecoder · January 4, 2017, 10:50pm

Not odd at all, you had the motivation/idea and wrote it to your needs. All we did was a little nudge

And the best part is others can use it now too.

skerit · January 22, 2018, 12:55pm

I’ve looked for some documentation on the “Remove scratches” filter, but this thread also didn’t really tell me how to use it.

So I’ve got 2 layers in GIMP: the actual image & the infrared mask. How do I apply the “Remove scratches” filter now?

paperdigits · January 22, 2018, 6:40pm

in g’mic, look in Testing / Chris - pixls.us the g’mic filter mentioned a few posts above should be there.

skerit · January 22, 2018, 6:53pm

Oh I know where it is, I just don’t know how to use it. There’s no way to say what layer it should use for the scratches information.