This script crashes gmic-qt with a segfault on this file (saved from vuescan as 64 bit RGBI)
I may have also accidentally wound up hijacking the filebin link from the OP, which seems like a slight security risk on the part of filebin 
Hm, tested with your file and it works here:
It’s not the most recent gimp or G’MIC on my side (2.9.9/2.1.5), but maybe something important changed. Since I will not be able to test with a more recent version for a couple of days, it would be good if somebody else could do the test. However, can you give any additional information (command line/log outputs, software versions, …)?
When exactly does it crash? Could you please try the following: Before loading the image, open G’MIC and set the “Show preview after” drop-down to the first point (“Threshold”). Cancel the G’MIC window and load your file. Open G’MIC and check if it already crashes. If not, go to the second preview step and so on, and tell when it exactly crashes.
Hi, thanks for the fast response and apologies for disappearing for a couple of days!
Versions are as follows:
Gimp - 2.10.6
G’Mic - 2.3.4
Both using ubuntu 18.04 and this ppa
I ran gimp from the command line to try and source some kind of detailed output but sadly the segfault just looks like this:
*WARNING* missing babl fast path(s): "Y u16" to "Y' u8"
/usr/lib/gimp/2.0/plug-ins/gmic-gimp-qt/gmic-gimp-qt: fatal error: Segmentation fault
/usr/lib/gimp/2.0/plug-ins/gmic-gimp-qt/gmic-gimp-qt: fatal error: Segmentation fault
/usr/lib/gimp/2.0/plug-ins/gmic-gimp-qt/gmic-gimp-qt: fatal error: Segmentation fault
free(): corrupted unsorted chunks
(It crashes when creating the preview)
EDIT: Misinterpreted your testing instructions originally. So:
no preview = All is well initially, crash on hitting apply or ok
threshold = The entire preview box is white, no crash
erosion/dilation = same as threshold
final image = crashWow, well that was silly of me.
So, after wiggling the sliders around it properly created the mask and didn’t crash, so it seems the issue was as follows:
- Scanner creates noisy IR layer (Nikon LS-30 in this case)
- The filter’s default settings assume the entire image is one big scratch
- The full preview is generated and the infill (presumably) crashes because infilling an entire image is a bit silly
- Because the dialog box crashed I never managed to move the sliders!
So, problem solved! Might be worth changing the default preview mode if possible.
Hm, the initial threshold value is at 72%, and the initial preview is showing the final image, maybe it should default to showing the mask and using 0% as threshold. However, I wonder if it should be considered a bug in G’MIC that the inpaint with a malformed mask (e.g. 100% image coverage by mask) crashes instead of dropping an error message and skipping the computation, @David_Tschumperle?
@chris, @MPratley, I confirm the crash with your file.
This is clearly a bug, it shouldn’t crash of course, so thanks a lot for noticing this and allowing me to reproduce this bug easily.
I’ll try to fix it ASAP. Cheers 
!
3 Likes
Found the bug, and fixed it with commit : https://github.com/dtschump/CImg/commit/6d404f1d6a611bfe97bf92b16b800a02f8ff1178
I’ll post new pre-release packages tomorrow (v. 2.4.0_pre) with this fix included.
Thanks again for your detailed report !
2 Likes
So, after having a relatively successful run with the filter, I’ve been having trouble with a few scratched negs. The issue seems to stem from the fact that my scanner has a LOT of overlap between the IR and cyan in a negative image.
After playing around with numbers in the filter and not having much luck, I found this previous post on the forum that I assume is referring to the technique described in this paper linked from this stack exchange post. Whew, that was a lot of hyperlinks. To paraphrase the SE post:
The idea is to remove the shadow of the color information from the infrared image by using the red/cyan channel (the one with the wavelength closest to IR, depending on whether the image is inverted)
Any thoughts on perhaps using a similar technique in this filter?
Filebin | 94ru9yhnxrxow3of <-an example of a sad, scratched image with a naff IR layer.
1 Like
Are you sure that it is a leaking of the other channel? I see variations in the extend to which the scene is visible in the IR-channel, depending on the emulsion used. Most evident this is in Kodachrome, which is almost like B&W-negatives, i.e. you clearly see the scene in the IR. So I would conclude that it ist the emulsion composition, which produces the effect that you see the scene in the IR, and not any leakage. Leakage would be the same for all type of emulsions.
Having said that, I would use the fact that the defects are on a much smaller scale than the scene and remove the scene to isolate the scratches.
Hermann-Josef
That’s exactly how I observed it and what I concluded as well. The histogram of the alpha channel shows a huge but narrow spike, such as (from the example crop above)
This spike contains image information, which I would guess is not a leakage effect but simply the IR light affected by the dye of the film. The scratches are almost black, to the left of the spike. This can be shown by adapting the dynamic range:
Therefore, setting the threshold to the left of the spike in the histogram works well for me.
Simple thresholding does not always work. It depends on the emulsion type. For Kodachrome one has to use a more elaborate procedure since the scene is very prominent:
At left is the RGB image with defects marked, to the right the IR-channel of a Kodachrome slide.
Some color negative films do also show the scene in the IR-channel, albeit not as strong as with Kodachrome.
At least for my scanner (DigitDia6000) leakage cannot happen. The IR-scan is taken in a separate exposure by the G-CCD with an IR-LED which has no emission at wavelength shorter than about 820nm and the RGB-LED is dead beyond 800nm.
Hermann-Josef
Hi, I worked on this issue (scratches) in the past, then I gave up due to lack of time.
The “leakage” is implicitly meant as “effect of the emulsion on the IR channel”, as you also figured out. It’s not a “leakage” in the sense of the electronics involved as you seem to have undestood 
When “leak” (so to say) recognition is performed, the resulting image is PERFECT, I remember doing some tests. However, it required image per image tuning.
What I never got to find out is:
- which colour channel(s) is/are best for the identification and substraction of the scene from the IR image, so that the process can be automated
- how to properly perform the substraction once the best channel(s) is/are found. As I pointed out in the other thread Difference between two regression techniques different apparently correct approaches bring different correlation parameters, except for a simple test image, only for which they match as they should!
Regarding thresholding: mentioned by @chris : it cannot work, because besides scratches and dust (relatively sharp), there is something else in the IR channel… drop-shaped or smeared-shaped residues of developing/fixing solution! They cause the histogram to be much broader and with a much less certain threshold values. It won’t work automatically at all, and trust me that even manually you’ll have a hard time choosing the value.
@olma
Hello,
I had tried scaling the RGB-image and subtract it from the IR-channel without success. VueScan does seem to somehow subtract the IR-channel from the RGB-image, at least for the strong setting. I conclude this from the fact that for very critical cuts you can see the structure of stripes from the IR-channel in the reduced image. This is, however, not the case for a setting of medium or low. This works well for E6-emulsions. But whatever setting you use in VueScan for Kodachrome slides, all dark regions are affected by the scratch removal. So VS is not really usable to my opinion for Kodachrome.
What I do for Kodachrome images is to smooth the IR-channel with a median filter and subtract this from the IR-channel. For an appropriate width of the median this leaves the scratches and dust. It fails, of course, for very large defects. They are rare and can easily be eliminated manually.
The original SF images are separated into the 4 channels using imagemagick. Then a script in imageJ does all the calculations on the individual channels and finally imagemagick is again used to assemble the channels for treatment in SilverFast. This script also eliminates the bias in the scanner data which is present in the unprocessed scanner data as delivered by both, VueScan and SilverFast. Removing the bias considerably improves the contrast in dark regions of the image.
Hermann-Josef
As I cannot edit my old post (@paperdigits, @patdavid, could you maybe reenable editing on this topic such that I can update it and make the files available again?), please find here a new scan of the original image if you want to test scratch removal. For a reasonable size, it’s only a cut-out of the original image, but it shows several kind of issues to test scratch removal again, especially a “telegraphy line” (horizontal line over the whole image) and some dust spots of different size.
example.tif (9.7 MB)
(3600 dpi scan on Reflecta CrystalScan 7200 with VueScan, saved as “raw” tiff file)
If you are interested, I can make the whole file available again (approx. 150 MB), and I even have a 7200 dpi scan available now (approx. 600 MB), but it would be good if this could be a permanent solution this time (maybe directly on pixls.us, e.g. as part of raw.pixls.us?).
1 Like
I’m on mobile right now, but your post doesn’t look locked to me.
I cannot edit my old OPs either (or old posts in general). I think only mods can do that. Would be useful especially for threads that have 100+ replies.
1 Like
Hi, thanks for the feedback.
Can you link or share the ImageJ script? I’m interested in testing it.
@olma Currently the script is not working. I did some “upgrade” to it (not related to scratch removal) and there is a bug somewhere I have to find.
As I explained above its median filtering the IR and subtract it from original IR, thresholding and smoothing the difference image. That is all, using genuine imageJ commands. Then I replace the IR-image in the scan file by this manipulated IR-channel and process this with SilverFast iSRD.
In the meantime (since my last post) I tried a procedure relying on the assumption that the reduction of the signal due to dust/scratch is the same for IR and RGB:
Threshold the IR-image, normalize it and then divide the RGB-image by this (my previous assumption of subtraction was wrong). Here is a demo of what I got with this procedure:
Left: corrected, center: original, right: IR-channel
Of course this won’t work for Kodachrome since the scene is readily imaged in the IR-channel.
Problem is the slight difference in image scale between IR and RGB. Currently I have no idea how to correct this.
Hermann-Josef
I tried to check correlation between R, G, B, grayscale and IR channel, hoping to find something which could help me subtract better the image data leaked to the IR channel.
I run this script:
#load color and IR, no preview
i ${1},0,2,2
#cut out black borders
crop 20%,20%,80%,80%
div 257
+channels[0] 0
+channels[0] 1
+channels[0] 2
+rgb2srgb[0] luminance. srgb2rgb.
ap[1-5] "unroll x"
append[2] [1],y
append[3] [1],y
append[4] [1],y
append[5] [1],y
ap[2-5] "pointcloud 0,256,256"
ap[2-5] "-normalize 0,255"
o[2] {0,b}_cloud_R.tif,uchar
o[3] {0,b}_cloud_G.tif,uchar
o[4] {0,b}_cloud_B.tif,uchar
o[5] {0,b}_cloud_mono.tif,uchar"
rm
So what you get is on X axis the value of R, G,B,grayscale and on the Y axis the value of the IR channel. I have to point out that the grayscale is the “luminance”, which almost discards the blue channel, but it doesn’t matter much: the channels are not so different.
The stronger the leak, the more grouped together the values are.
Darker images have more points on the left, brighter ones more points on the right.
The results are best shown using the Windows thumbnails (each photo is followed by R, G, B, grayscale point clouds):
Overall, reddish photos work best with red channel, blue photos with blue channel and so on, so the grayscale is the best option.
I would be curious to see the same applied to Kodachrome to see how the clouds lok like!
But also the baseline is not a line, and a linear correlation would not work to remove effectively the leak. It is often good enough, but some photos have a noticeable curve.
At this point for my E6 slides I’m not sure how to proceed about subtraction of the leak.
I will try the blur as you suggested, at least to compare.
As far as I can see for E-6 there is hardly any signal from the scene in IR. So I normalized the IR to the average level, put a threshold to avoid introducing noise and then divided every channel by this modified IR-image. No smoothing applied here. That was the example above.
Smoothing was applied by my imageJ macro in preparation for the application of iSRD in SilverFast only. It was meant to slightly enlarge the defects to avoid halos around the defects if iSRD is applied with a minimum detection setting.
Hermann-Josef