linear color transfer in gmic ?

Software G'MIC
1

hello there and happy new year to all the gmic people !
when doing style transfer it’s very convenient to do “linear color transfer” operations between content and style.
The algorithm is described in Leon Gatys paper here : https://arxiv.org/pdf/1611.07865.pdf
The original code is here : NeuralImageSynthesis/ScaleControl.ipynb at master · leongatys/NeuralImageSynthesis · GitHub
i’m using this python script to perform the transfer : Neural-Tools/linear-color-transfer.py at master · ProGamerGov/Neural-Tools · GitHub

i was wondering if this operation is available or doable with gmic as it would simplify my workflow enormously.
Best
Luc

1 Like
2

I think that is exactly what transfer_histogram does.

$ gmic h transfer_histogram

  gmic: GREYC's Magic for Image Computing: command-line interface 
        (https://gmic.eu) 
        Version 2.8.2 (pre-release #200110) 
 
        Copyright (c) 2008-2020, David Tschumperle / GREYC / CNRS. 
        (https://www.greyc.fr)

    transfer_histogram:
        [reference_image],_nb_levels>0,_color_channels

      Transfer histogram of the specified reference image to selected images.
      Argument 'color channels' is the same as with command 'apply_channels'.
      
      Default value: 'nb_levels=256' and 'color_channels=all'.
      
      Example: [#1]  image.jpg 100,100,1,3,"u([256,200,100])" +transfer_histogram[0] [1]
3

thank you.I will check that.
luc

4

Reading the paper from Gatys, it appears that transfer_histogram is not a linear transfer.
But doing this linearly shouldn’t be that complicated in G’MIC I think.

5

well i tried with a few examples and it’s rather different …

6

Is this what you need ?

transfer
transfer753×395 56.7 KB

7

i think there are two parts in the process : luminance and color.i’m more interested in the color part.if you look at the code it is that function :

match_color(target_img, source_img, mode=‘pca’, eps=1e-5):
Matches the colour distribution of the target image to that of the source image
using a linear transform…

8

Ha yes, I see now.
There is nothing like this right now in G’MIC, but I’m certain this can be done.
Maybe I’ll try later, but right now, I’m too busy unfortunately.

1 Like
9

thank you David.no problem.it would be wonderful.
luc

10

OK, I have the PCA method working right now.
Two images, and I transform the first one to match the covariance matrix and the average color of the second one.
Need a bit of optimization, polishing and I’ll probably put this as a new command.
Not usable as it is right now, but almost done anyway :stuck_out_tongue:

 foo :
  v 1
  sp lena,colorful

  f[0] "begin(
       Cs = [ "${"covariance_colors[0] _avgs"}" ];
       avgs = [ "$_avgs" ];
       Es = eig(Cs);
       Ls = sqrt(Es[0,3]);
       Rs = Es[3,9];

       Cc = [ "${"covariance_colors[1] _avgc"}" ];
       avgc = [ "$_avgc" ];
       Ec = eig(Cc);
       Lc = sqrt(Ec[0,3]);
       Rc = Ec[3,9];
     );

     vec = (Rs*(I - avgs))/Ls;

     res = avgc + (transp(Rc,3)*(vec*Lc));

  "

  Ts=${"covariance_colors[0] _avgs"}
  Tc=${"covariance_colors[1] _avgc"}

  3,3,1,1,$Ts
  3,3,1,1,$Tc
  1,3,1,1,$_avgs
  1,3,1,1,$_avgc
2 Likes
11

i love your definition of later :slight_smile:

1 Like
12

Your fault, asking for interesting stuffs! :wink:

I’ll probably add this color transfer option in the G’MIC style transfer filter as well, maybe it will help to render better results!

1 Like
13

it does a lot on my side.thanks

1 Like
14

@luluxXX Your questions usually help improve G’MIC. Thanks. :+1:

1 Like
15

All this time, I never realized you could use commands within fills. Other than that, I might use this to improve transfer color [Reduced Colors].

16

So, here it is : Command transfer_pca is now available in the G’MIC stdlib available after a $ gmic update as usual (assuming you run G’MIC 2.8.0+).
Here is the command help :

 gmic -h transfer_pca

  gmic: GREYC's Magic for Image Computing: command-line interface 
        (https://gmic.eu) 
        Version 2.8.2 (pre-release #200110) 
 
        Copyright (c) 2008-2020, David Tschumperle / GREYC / CNRS. 
        (https://www.greyc.fr)

    transfer_pca:
        [reference_image],_color_channels

      Transfer mean and covariance matrix of specified vector-valued reference image to selected images.
      Argument 'color channels' is the same as with command 'apply_channels'.
      
      Default value: 'color_channels=all'.
      
      Example: [#1]  sample lena,earth +transfer_pca[0] [1]

Good things with the G’MIC version of the pca-transfer implementation:

  • It works for 3D volumetric images.
  • It works for an arbitrary number of channels.
  • It is parallelized.
  • For a color image, you can specify which color channels are considered for the transfer.

So, if I do a similar thing to what is described in the article from Gatys, linked by @luluxXX, I can do it in two steps :

$ gmic sample zelda,earth +transfer_pca[0] [1],ycbcr_y  transfer_pca[-1] [1],ycbcr_cbcr

which gives:

gmic_transfer_pca
gmic_transfer_pca938×843 125 KB

That’s all for tonight.
I’ve used some cool tricks to code this command, so I’m planning to do similar things in command transfer_histogram to make it slightly faster.
After that, I’ll add this new PCA-transfer option to the style transfer algorithm of G’MIC, to see what happens…

Until then, good night, and have a great week-end :slight_smile:

7 Likes
17

Very nice function, still some flaw (overflow or cutoff!), try

gmic sp portrait2,monalisa +transfer_pca[0] [1],ycbcr_y transfer_pca[-1] [1],ycbcr_cbcr

18

While your example gives a quite strange result (but a result that can be explained), I don’t see any overflow value in the rendered image.

[2] = '_portrait2':
  size = (800,800,1,3) [7500 Kio of floats].
  data = (128.395,129.372,133.618,136.186,136.186,133.618,133.618,133.618,136.186,133.618,134.105,126.99,(...),104.451,99.8682,99.8682,99.8682,106.592,99.8682,100.175,100.175,100.175,100.175,100.175,96.6649).
  min = 0, max = 221.713, mean = 95.2109, std = 49.4719, coords_min = (403,36,0,1), coords_max = (636,425,0,0).
19

I expected something like that during computation, typically for the discontinuities! Still an explanation for them could be interesting!

20

thank you afre.your answers usually help improve myself.