Picture restoration using machine learning

I came across this link on using machine learning for image restoration.

This is a photograph from the 1930s of my mother and some of her siblings. I have tried working on it over the years, to remove the scratches and other flaws, but never with great success

The instructions to install the software sound simple, but there are some things it doesn’t mention, like the fact that you need a C++ compiler, Python include files and, if you want to build the GUI, X11 include files as well. I didn’t want this so I amended the requirements.txt file to remove dlib. If you want to use the GPU, you also need include files for this as well.

Anyway, I finally got it working. The original is only about 1000x1600 but as far as the software is concerned, this is high resolution. This is the result.

It isn’t perfect, but it is in a reasonable state for some more work in something like GIMP.

5 Likes

I don’t think it did a good job. For example, the girl on the right had her lower lip ruined. IMO, better to do it by hand, even if it takes a lot of time…

1 Like

I too don’t like the AI fix. It would be interesting to see the best you have done so far. It could also make an interesting playraw. I bet the people on this forum could beat the AI.

2 Likes

With this algorithm it is apparently possible that the photo can be damaged beyond artificial repair.
Here is a real easy way to use this tech:

1 Like

It might, if I had a raw file. Unfortunately, I only have a jpg file.

That there are issues with the restored file, I would agree. One of the major ones as far as I am concerned is the fact that it reduces the resolution of an already low resolution file.

I will have a few more goes, changing it to black and white, rather than sepia, and increasing the contrast.

Jpg would be fine for restoration, tiff would be better

Examples shown in link look good. Can we download the software?

Yes, you can. The link I gave in my first post in the thread allows you to download the software and the training sets.

Be aware though, the training sets are huge. The total download is a couple of gigabytes, even when compressed.

I have tried various changes, increasing the contrast, converting to black and white. i won’t post the results, since there are only marginal improvements to the result that I already posted.

Hi Colin, so have Microsoft been selective in showing images that AI worked well on and in your experience it is not that good?

One idea that comes to my mind is if AI does some areas good and some not so good it would be possible to do layers in GIMP where one layer was AI, the bottom layer was the original. Then using a mask you could selectively keep the good areas from AI and then bring back the original areas where AI stuffed up.

I might try downing the software today and giving it a try.

Some time ago, I remember reading about a gentleman in Japan that was renwned for doing this kind of thing — what’s more, he refused to take payment for his services. The results were remarkable, from what I can remember. Sadly, I can’t remember where I read about him.

1 Like

I would say the work needs two processes:

  1. Identify pixels that need replacing.

  2. Replace them with plausible colours.

For (1), the ML has over-identified problems, eg the mouth and right sleeve of the figure on the right. So the good folds in the sleeve on the right have vanished in the “corrected” version.

For (2), most replacements are with a ghastly blur. The main exception is the window frame on the right, where an exemplar replacement seems to have been used successfully.

I did come across this, though:

1 Like

You can and do

2 Likes

@Zbyma72age
not perfect, but impressive. excellent work!

i did this in about half an hour in Gimp 2.10 - just to see what can be done. i use the Heal & Clone tools and Heal Selection filter.

2 Likes