collaborators for image tiling

@garagecoder, maybe I can help saving you a little bit of time. I’ve finally managed to implement a beginning of a Style Transfer filter (https://twitter.com/gmic_ip/status/1066460833021157381).
It does not really use the idea I’ve talked with you a few days ago, so just forget about it :wink:

Wow nice stuff, that’s good news! Work became… intense this past week so frankly I had to forget about it for now anyway :smile:
But now it’s is in my head, it’s staying on the (extremely long) list of ideas!

I’ll use one of your tiles here (tell me if this is not acceptable and I’ll remove it!). Here’s how I interpret the method, please correct if I’m wrong:

You begin with one or more polgyons, defined using rectangular raster image and opacity mask - please excuse the rough cropping.
tile0

This is copied/mirrored/rotated/translated and rendered to a target output image. An example of “Fo, R-60” in G’MIC:

gmic tile0.png mirror x rotate -60 autocrop

tile1

The translations tend to be done in terms of a predefined constant (here triangle side length, height) and a factor.

Given that, I can think of 3 ways to handle it (without changing the “language”):

  1. Raster based (the current method)
  2. Vector field based (manipulate source pixel positions rather than the data itself)
  3. Vertice based (not thought this through but I think it’s possible)

Option 1 is quite simple in G’MIC, but has downsides - aliasing/loss of quality. What do you think? I suppose if you only ever clone “tile zero” then it’s not an issue.

Edit:
Here’s a quick demo of how you can simply define some custom commands in G’MIC to do this:

_tile_Translate : skip ${1=0},${2=0} # this command actually renders as well
  split_opacity. n. 0,1 j[1] ..,$1,$2,0,0,1,. rm[-2,-1]

tiling :
  v -
  t_w={w} t_h={round(1/2*sqrt(3)*$t_w)} # define constants
  {3*$t_w},{2*$t_h},1,3 # create the background image
  m "_tile_Clone : [$""1]"
  m "_tile_Rotate : rotate. $""1 autocrop."
  m "_tile_Flop : mirror. x"
  m "_tile_Flip : mirror. y"
  _tile_Clone 0 _tile_Translate 0
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[-1/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Translate {[1/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[$t_w,0]}
  _tile_Clone 0 _tile_Flip _tile_Translate {[3/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Flip _tile_Translate {[2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[5/2*$t_w,0]}
  _tile_Clone 0 _tile_Flip _tile_Translate {[0,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[-1/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Flip _tile_Translate {[1/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[$t_w,$t_h]}
  _tile_Clone 0 _tile_Translate {[3/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Translate {[2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[5/2*$t_w,$t_h]}
  k[1] v +

The trick here is to define translation as actually drawing the tile. It shouldn’t be too hard to turn this into a filter which accepts a command string, but I find string handling in G’MIC tedious and I ran out of time for now! Seams are clearly visible (I didn’t try to get a better crop) but that can be dealt with in other ways later…

gmic tile0.png tiling

2 Likes

I need a floor with a pattern like this… a much duller version. :smile:

Hello garagecoder,

thank you very much for your commitment!

The cropped proto-tile is totally fine. Attention must be drawn to the edges; if there are artifacts here they are multiplied in the resulting tile.

The stripes you mentioned are the edge artifacts that exists if non-rectangle polygon shapes were composed which is mostly the case. They are the main aesthetic problem with image tiling. The are appearing in my Matlab prototype more than in my (unfinished) prototype with PerlMagick&ImageMagick and my main work was to find post-processing methods to correct them. I was not able to find image processing methods that can correct both the heavy black edge artifacts and the more subtile ones until I started using edge masks of such a tiling. In the last days I finished a version that is using a drawn edge mask given by a set of line coordinates and the Matlab function inpaint_nans. This version delivers finally!! a satisfying quality but is only acceptable fast if the simple averaging method is used instead of the PDE-inpaint methods. I will publish in the next days a flicker album with this method to show the subtile advancements compared to the album I linked in my first post.

I am very curious how this can be solved in GMIC; probably with a boundary method because this are gap-artifacts.

Option 1 is quite simple in G’MIC, but has downsides - aliasing/loss of quality… I suppose if you only ever clone “tile zero” then it’s not an issue.

I think it is not an issue if there are no longer processing pipelines with quality disturbing operations are used. If every proto-tile (independent of the number) is directly cloned from the original list instead of reusing an already processed (rotated) proto-tile then the effects are minimized.
Mirroring and translation are non-disturbing matrix operations but rotation can cause massive disturbance dependent on the used interpolation method. “nearest” is very bad but default is mostly bicubic but this causes edge problems (From the imrotate doc: “This interpolation method can produce pixel values outside the original range.”) and I suspect that this is the most part of the edge artifacts in the tiling; the rest might be inexact autotrim because every pixel counts here. Using bilinear is causing a poorer quality of the shape texture so this is not an option for me; it is already an annoyance that there is not the opportunity for user defined interpolation in imrotate as there is in imresize where I defined Mitchel as a better option to bicubic.

Vector field based (manipulate source pixel positions rather than the data itself)

I can only speculate that this option is an analogy to meshgrid processing in Matlab. I find this very interesting!! because I have speculated about the existence a meshgrid representation of a whole! tile: every element in a [2t_h 3t_w] meshgrid points to a pixel in the proto-tile polygon image which means several elements points to the same pixel. The tile image is then generated by an interpolation of the meshgrid and the proto-tile image instead of a sequence of image processing procedures.

In my batch processing context this would be a huge advantage because such a vector/meshgrid representation must be generated only once and it is then applied to all the input images that are processed. At least in Matlab the interpolation is much faster than a longer image processing pipeline which would lead to a huge efficiency boost. Additionally I believe that the interpolation could also be a solution for the edge artifacts. But I did not came far in an implementation of a tile meshgrid representation; perhaps some day a case for a freelancer-com project.

I must keep my answer short for now, but fear not - I’m very aware the edge artifacts must be dealt with, preferably in a way easily replicated in other environments. With G’MIC we’re fortunate to have several inpainting algorithms ranging from very fast to very clever (hence not so fast…), but there are several other options. It needs thought and experiment, which I’ll have time for next week! Again, anyone else is completely free to chip in of course :slight_smile:

Edit: almost ready to push the filter (it’s very rough)

Filter added, I think only for G’MIC version 2.4.2 (and probably an hour or two before it’s available to download with filter update). Some gotchas:

Only working with one input tile.
Inpaint is rudimentary, don’t try to use it unless the background is full!
I’ve stuck with the design of using “Translate” to actually draw, the format of which is T[x_coord,y_coord].

So the equivalent command list you can paste in for the above:

C0 T0
C0 Fo R-60 T[-1/2*$t_w,0]
C0 Fo R60 T[1/2*$t_w,0]
C0 Fo R-60 Fi T[$t_w,0]
C0 Fi T[3/2*$t_w,0]
C0 Fo R60 Fi T[2*$t_w,0]
C0 Fo R-60 T[5/2*$t_w,0]
C0 Fi T[0,$t_h]
C0 Fo R-60 Fi T[-1/2*$t_w,$t_h]
C0 Fo R60 Fi T[1/2*$t_w,$t_h]
C0 Fo R-60 T[$t_w,$t_h]
C0 T[3/2*$t_w,$t_h]
C0 Fo R60 T[2*$t_w,$t_h]
C0 Fo R-60 Fi T[5/2*$t_w,$t_h]

The text formatting garagecoder used in his last post is also a template for parallel processing: every line is independent and can be processed in parallel.

In the meantime I have worked on the collection of CRMT-lists (and connected mask coordinate lists) of the non-trivial wallpapergroups (= with non-rectangle proto-tiles), so available are now:

p4m, p3m1, p6m, p31m, p3, p6.

What might be a good way to publish this and future lists?

Entirely your choice of course, but I’m wondering about adding some “presets” to the filter (where you choose one from a drop-down list).

@David_Tschumperle more questions I’m afraid!
What’s the best way to test a new version of a GUI filter? Overriding commands works fine at command line - just add to .gmic file. Doesn’t appear to work for GUI (it’s like it’s “locked”).

Is there a way to add multiline text to a GUI multiline text parameter? I can’t seem to figure that out (single line works no problem).

Ah yes, this is a bit tricky, as putting a “\n” doesn’t substitute it by the ASCII code 10.
But the following sample works for me:

#@gui Filter : filter,filter
#@gui : Text = text{1,"Really,that's\na multi-line\ntext !"}
filter :
  to "$1"
  LF={`10`}
  u "{And this can change"${LF}"with multi-lines"${LF}"too !!}"

That’s a good question, the GTK version of the plug-in was giving the priority to the commands defined in the user .gmic file, but it seems it’s not the case anymore. I’ll ask Sébastien why :wink:

1 Like

some “presets” to the filter

I think that the wallpapergroups are the first choice for such presets like in the old Terrazzo filter because they are basic for all periodic tilings and they are using only one proto-tile. I attach the other CRMT-lists and the masks for further usage (the commands relay on the specific mask shapes and orientations); be aware that they are much less tested than p3m1.

CRMT_p3.txt (174 Bytes)
CRMT_p4m.txt (164 Bytes)
CRMT_p6.txt (375 Bytes)
CRMT_p6m.txt (772 Bytes)
CRMT_p31m.txt (505 Bytes)
p3_mask.txt (122 Bytes)
p4m_mask.txt (59 Bytes)
p6_mask.txt (122 Bytes)
p6m_mask.txt (67 Bytes)
p6_mask.txt (122 Bytes)

1 Like

Should be able to include those, probably compress quite well.

I was thinking about vector maps, it turns out very simple for a single tile (edge aliasing issues aside). In G’MIC all you need is a vector valued tile as the input (each vector at x,y holds it’s own position) - very trivial to produce:

gmic tile0.png f ":i3>0?[x,y,z,i3]:[0,0,0,0]" o vectortile.tiff,float

Then when you create a completed tiled image as usual, what you have is a vector map. Now you can take any tile of the same dimensions and do:

gmic vectormap.tiff tile0.png +warp. ..,0

warptile

The top part is the “warp map”

For more than one input tile it becomes more tricky, you’d probably need to save an extra channel in the map specifying the source tile (amongst other things).

Edit: what I mean about using a warp map is you don’t even need the opacity mask, only that the whole tile dimensions match. Example:

tiledcats
tiledwall

1 Like

Been thinking about the aliasing issue. If you are going by the upside down triangle, only 2 edges are aliased. Couldn’t you simply make the edges fit pixel perfectly?

1 Like

Hmm I suppose the problem is lack of information. If you know it’s a triangle, you can use that fact to rotate by vertices and match them exactly. The problem is the filter is only given a raster image, so there’s no concept of shape (without getting into transforms). When you begin using rotations less than 90 degrees, edge mismatch is probably inevitable with a raster image. I might look at testing worst cases to see what reduces that error though :slight_smile:

Hmm, I’m interested into the filter as I am a fan of tileable hex-art. That’s all I have to say here.

Ooh, I have a idea, each triangle can go into a separate layer, right? So that I can adjust these triangle individually?

vector maps

VERY INTERESTING!!! That is probably near to the idea of a meshgrid representation of tilings in Matlab I wrote about on Nov. 25 and something that is useful in my batch processing context.

What about the expected run time of such a representation compared to an original CRMT-pipeline? The run time of a CRMT-pipeline is proportional to the number of proto-tiles that are distributed over the background (14 with p3m1, 28 with p6m, …) and the background area but a grid representation is independent of the number of proto-tiles and only dependent on the number of output pixels (and the interpolation method).
If I am comparing the interpolation of conformal maps to a p3m1-CRMT-pipeline with a similar image area I estimate a 2-3x faster image generation. But perhaps the expected problems in the majority of cases with more than one type of proto-tile (C1, C2,…) could make this efficiency advantage disappear.

Interesting to see that in your first image example some edge artifacts vanishes, some were reduced or transformed but some remain unchanged. So the vector method have active influence on the edge artifacts which could lead to the hypothesis that there exists parameters of the warp/interpolation function (like interpolation method) that could also solve the edge problems (two birds with one stone).

If you know the tile type (p3m1 here) you know automatically the shape of the proto-tile and its orientation bec. the same shape with another orientation and the CRMT-command list is not working properly.

You have even more control if the proto-tile is explicit generated with a mask from an input image so you have all the edge information from the mask coordinate list.

I spy a potential polished kaleidoscope filter.

1 Like

That’s exactly right, it can also be very parallel - therefore it will be faster to warp from a pre-made vector map than to build one. I’m trying not to get too far ahead of myself though, there are very many options :slight_smile:

Exactly what I’ve been thinking!

For now the plan is

  1. Allow multiple input tiles (might take a bit of rewriting)
  2. Add presets
  3. Easy output of vector maps

Time is once again running short, so this may happen in drips :slight_smile:

That’s one advantage of using a vector map - it eliminates some glitches caused by imperfect input tiles (but still depends on the opacity map/shape being precise)

The current Kaleidoscope sort of mixes color where one doesn’t want them. I can certain welcome a new Kaleidoscope filter with more options that doesn’t have that problem.