Rescuing thousands of images from a blemished life

Ok, i updated on the command line and have them.

So, I’m trying commandlines out now. Thanks, I’ll have a bit more of a play. :slight_smile:

Cheers,
Jeff

Sometimes the plugin doesn’t update properly. Don’t expect too much from the filters though. They aren’t very smart and only meant to give you perspective.

1 Like

Thanks again @afre.

Just quickly, and I’ve been looking for about an hour on this one - what would I append to my gmic command to output a file with the same basename, but a png version. From what I’ve been reading, I feel it has something to do with -output png:{[image],b}, but I’m not sure how to write it…

Cheers,
Jeff

Try outputx.

You are quite right, only one [image] was missing:

o[image] …

but long is

o[image] png:{image,b}.png

If it is the last image in the list

o. {b}.png

(the extension is used for the file format)

1 Like

Ahhh, that’s linked a few coding concepts for me. Thanks. I’m going to have a play now, and write back what I’ve come up with. You guys are very helpful - I think this might work well, but have a few more things to try…

@Jeff Could you please CC license your photos in posts #6 and / or #10, so that we may test and use them as examples?

Hi @afre,

I certainly license you to use whatever I upload to this page for testing purposes and using them as examples.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

I hope the above inclusion helps.

Cheers, Jeff

Here is a light cleaning of image 502 using Clean Photo 0 (afre_cleanphoto0) with an additional user specified list of regions to mask. I haven’t committed this yet because I have to figure out how to make it presentable for the public and perhaps improve the base algorithm.

Original (zoom and toggle between the two)

Light local dust and scratch cleaning (only the more egregious artifacts have been marked)


PS1 Clean Photo 0 / 1 should work in the plugin now. (I placed it in the CLI section by mistake.)

PS2 I forgot to mention that the dust and scratches seem to have R-B aberration. Might be the scanner, photography or something else. What does everyone think about that?

PS3 Made region tiles have slightly smoother edges.

2 Likes

I can’t believe what I’m seeing. That is INCREDIBLY good!!!

I have no idea what R-B aberration means (another thing I have to google) but …

I’m picking myself up off the floor…

I’m going to test this on a few other shots, and try to make a way of automating this in digikam if I can too. I just need to get a good run of time. When I add an inpaint holes thing and auto colour balance too (which I’ll do this as a next stage) I think that’s… just… amazing… You must have cracked the hard part @afre

Wow. “Thanks” is understating things…

please…test everything on a copy of the originals! :smiley:

where the red and blue are offset from the green.

1 Like

The afre_cleanphoto CLI command is complete now. It will replace the previous two but I will keep the two for the time being. You can read more about afre_cleanphoto here:

Have fun!

1 Like

That is exceptional @afre. I love your documentation too. I was going to ask you a few questions, but you have answered most of them now. I’ll continue some testing and let you know how it goes, and add a few of my scripts to help others who might come here for a look.

GREAT stuff. :slight_smile:

1 Like

Here’s some test scripts for anyone who’s watching, and wants to use this with digikam or on the command line. And I can’t thank the devs enough for their work here…

Starting work on the commandline

Firstly, I wrote a little script that I could call from both the Linux commandline, or from digikam (or from anywhere else from that matter, so that I could run a few different variations automatically. It’s in a file called “/home/USER/gmicafre” (original, eh! :slight_smile: ) where USER is my username on that computer.

I put the following lines in it:

#!/bin/bash
gmic "$1" fx_inpaint_holes 4,12,0 afre_cleanphoto 5,10,0,1 fx_tk_colortemp 0,1 o. {f}/{b}_afreclean_holes_temp.jpg,88
gmic "$1" fx_inpaint_holes 4,12,0 afre_cleanphoto 5,10,0,1 o. {f}/{b}_afreclean_holes.jpg,88
gmic "$1" afre_cleanphoto 5,10,0,1 o. {f}/{b}_afreclean.jpg,88

Let’s go through what it does. It runs the gmic process a few times, producing different outputs each time. I’ll keep to the first gmic line, as the other gmic lines do the same kind of thing but applying fewer filters to their output.

On each of the gmic lines:

  • “$1” takes the first argument supplied with the script name - it is the photo image you want to process, and includes the full path with that photo filename.
  • fx_inpaint_holes, afre_cleanphoto and fx_tk_colortemp are each filters that are applied to the photo. The numbers that come after each filter are generally the default numbers that the programmer (@afre) recommends, though I have changed one or two of them after a few extra tests.
  • The output image is written using the “o.” command, which puts the output into the same directory (using {f}) and giving it the same base name, but appending some info about what each of the filters it has applied are.
  • Finally, the “,88” (which only works for jpg images) tells gmic to output with 88% quality, which gives me an image of about the same size as the input image. Yes, this reduces the quality a bit, but the images are still relatively pleasing to the eye. If you want to lose the least information, and you can handle a fairly large file size, using “.png” rather than “.jpg,88” might be a better idea.

After issuing

gmic update

in the commandline to make sure I had the latest filter definitions, the way I ran this script on a host of files was to issue the following command:

find /ROOT DIRECTORY OF IMAGES TO BE PROCESSED/ -iname "img*.jpg" -exec sh gmicafre "{}" \;

Of course, you’ll replace the uppercase stuff with your own directory. And this works on Linux shells only.

The script will create three images from the source image, and you can go through and see which variations suit your source images. You can delete unnecessary images or go and edit the original script so it only creates images with the filters you choose. And, of course you can use many tens of different filters as you like.

Digikam
Digikam is another program I adore. I select the images (or whole albums I want to run this on in the digikam main window and press Shift + B. That takes me to the batch queue manager and instantly lists all photos I selected in the Queue pane to the left of the window.

I then go down to the Control Panel, scroll down to the last item in the list (User Shell Script) and double-click on that.

I then move over to the Tool Settings pane, and in the Shell Script box, I type:

/home/USER/gmicafre $INPUT

Then I go to the top left of the batch queue manager window and hit the Run button.

Don’t be concerned by the error messages that come up in the Queues pane. Your images will be created. Once all images are processed, close the Batch Queue Manager window, return to Digikam’s main window, and hit F5 to refresh the screen. You’ll see your new images there and can scroll through them in preview mode to see how your various filters have been applied.

Hints

1 To make the command more universal, try to do all of the scripting inside G’MIC. E.g., try input_glob. Still need to escape though.

2 To save time, use each command only once. E.g. gmic sample tiger blur 1 +mirror[-1] x +rotate[-1] 90 name tigerb,tigerbm,tigerbmr outputx png

+command[-1] → ‘+’ means insert result at end of image list; [-1] means based on last image on image list. Together this means each step produces a new image.

name is self explanatory. You assign a name for the images in the image list. Be careful, if you have less names than images, the naming will wrap around.

outputx outputs to specified file type.


NB I often make incremental changes to my afre commands, which means their behaviour may change over time; you may have to check and adjust your parameter values accordingly. No need for alarm: usually changes don’t affect them. If they do, I would announce it in my diary.

1 Like

Again, brilliant, thanks!!

Revised afre_cleanphoto again. Parameters have changed. One less parameter too. Should be faster and yield improved results. Only tested on 502.jpeg though. Use afre_section beforehand to determine the region mask labels.

afre_cleanphoto:
        1<=size<=10,1<=_xy_sections<=50,_maskN>=1,...

      Clean dust and scratches from photos.
      Default values: 'size=3', 'xy_sections=1' and 'mask1=1'.

      - 'maskN' is a comma separated list of regions mapped by 'afre_section'.
         Use the 'afre_section' command to visualize.
1 Like

For a much later future reference, it is better to add a dash before parameter description? Is it standard to add description after default value?

Thanks again @afre.

I’ve been smashed with work the last few weeks, and haven’t had enough time to play at any length, but will certainly be getting at it soon. I’m looking forward to it.

1 Like