A Neural Algorithm of Artistic Style


(Morgan Hardwood) #1

Hello

I found these:
http://imgur.com/gallery/BAJ8j
https://github.com/jcjohnson/neural-style
Wow!

Any chance of having this in G’MIC?


DreamDeeply from google
I can't open the image via https
(David Tschumperlé) #2

If we find a PhD student and about 200k$ to pay him, then maybe yes :slightly_smiling:


Sketch Simplification
(Jonas Wagner) #3

This is really cool wizardry. I was pretty mind blown when I first saw it.

But it looks like it might be quite hard to port given that it depends on relatively complex gpu based computation to make the neural networks at bearable speed + lot’s of data.


(Lyle Kroll) #4

Wow; impressive stuff. Would like to see a Windows compile of this program. :slight_smile:


(Tobias) #5

I looked into the install script and did not run it. I don’t like it that the script ask for my root password, downloads some stuff from the internet and installs it. I’ll have a closer look into it later this week. Or even better a nice and simple PPA for Ubuntu.

@David_Tschumperle
The script itself is only 500 lines of code. It uses torch7 which has a C interface. I think it should be cheaper then 200k$ to port it to G’MIC if you would add torch7 as a decency. But still a lot of work. :wink:

@lylejk
The script itself should run on Windows if you change the ‘/’ in a ‘’ in the function ‘build_filename’. But I think the hard part are the dependencies.


(Lyle Kroll) #6

I posed this at GIMPChat and Tran posted link below of an online version of this program. Requires site registration though and I’m just not that interested in doing so. :slightly_smiling:


Creating a lineart drawing from a picture
(Tobias) #7

And here I found a similar website, that uses a deep neural network too:
https://deepforger.com/


#8

Hey,
the 2D animationprogram Toonz is open source since today (Opentoonz), and on first look it seems that there should be a effect plug-in (Dwango) that can change the style of a picture with deep learning technology.
I didnt try it so maybe I am wrong.

Windows/Mac only :frowning:


#9

Another site that composites an art style is this one:
http://www.deepart.io/

When using the free option the site produces an email link to the composite image faster than it suggested.
I’ve no idea whether I received spam, the email account was ‘sacrificial’.

Also, here’s some interesting data that has been collected by Altmetric:

https://www.altmetric.com/details/4439857

Rich


(Tobias) #10

I just found this paper, where an neuronal network was used for image colorization:

https://richzhang.github.io/colorization/


(Tobias) #11

And they even try to create a new Rembrandt with a neuronal network and real paint:


(Andrea Volpato) #12

Have you seen Prisma app for Android and iOS? http://prisma-ai.com/
It seems to use the convolution neural network algorithm.
There are limitations: the final images are square 1080x1080, with the Prisma logo, you can choose only between a small set of paintings/posters and you cannot tune any parameter.
And OK, it is not open source. But the results of these algorithms can be amazing and it is nice to have the chance to easily play with them!

Here an example from the Drakensberg mountains:


#13

A Russian app called Vinci is also using neural networks. They’ve supposedly used the process described in this paper as the basis for the app which speeds up the process. I won’t pretend to understand all (or any) of the technical details but its applications seem to be growing.


(Jonas Wagner) #14

Has anyone checked whether these apps do the processing locally or on a server?


(Andrea Volpato) #15

Prisma is definitely doing the computation on its server.


(Tobias) #16

Afaik all these apps do the calculation on servers.


(Tobias) #17

I looked again into this topic.

  1. I found this project, where you cen get everything in a docker container installed. This should be much easier to install then the other projects:
    https://github.com/alexjc/neural-doodle#2-installation--setup
    They even have a nice blogpost about a concept, how to integrate the neuronal network features into GIMP:
    https://nucl.ai/blog/neural-doodles/

  2. And this is a project, that implements the algorithm on a neuronal network, that runs on Windows too. So all windows users can try this out:
    https://github.com/titu1994/Neural-Style-Transfer

There are at the moment implementations of the style-transfer with, I think, every neuronal network out there. Here is a list of different implementations:
http://www.genekogan.com/works/style-transfer.html

Conclusion:
Pro:
This is a the moment really a very hot thing. Some of the GitHup projects have more then 1000 forks, and a lot of companies (Goggle, Facebook, Microsoft, Intel, …) and researchers are working on this topic.
Con:

  • It is hard to get a good overview about all the projects out there and there is at the moment not one defacto standard project.
  • The installation of the neuronal networks are complex and the tools are not packaged for Ubuntu.

#18

I hope this technic will be available for more usual image enhancing too, like noise reduction, sharpening / deconvolution, toning, color correction etcetera.

For scaling there is Waifu2x already, it does also noise reduction.

I like to have a “simple” feature that a neural network can be trained on a original image with a (GIMP, Photoflow, Rawtherapee, Darktable) enhanced image. So creating a neural network filter I can apply on on other images.

In a beautiful German free (not FOSS) program “Fitswork” (Windows) by Jens Dierks, for astrophotography, you can already do something like this:

http://www.fitswork.de/software/softw_en.php
http://www.fitswork.de/anleitung/speziell.php (German) “Bildgröße x2 NN” and "Universeller NN-Filter"
http://www.fitswork.de/anleitung/sprachen.php (files for other languages)

But there is a lot more you can do with this software.

The entire computer program fits on an old-fashioned 1,44 MB floppy disk.

The NN-filters are simple. There is no “deep learning” possible. But it is enough interesting what you can accomplish with such a simple neural network. I like to have the simple functionality of “Fitswork”, but with more complex and bigger neural networks. I don’t know now other free or open source software with this feature that is easy to use for layman.

I can imagine that it must be possible to create something like this now (FOSS?), with all these new technic.


(Tobias) #19

Now Facebook created a version of the algorithem, that runs 100 times faster. It is now possible to do the style tranfere on a smartphone:


(David Tschumperlé) #20

Very interesting Tobias ! Thanks for the link.