collaborators for image tiling

Software G'MIC
,
1

Symmetry and symmetry breaking is a central topic of my artistic work. I am fascinated by image tiling as a source of symmetry building but until recently the situation was not different than 25+ years ago when my interest started with the Photoshop plugin Terrazzo from Xaos Tool: There are thousands of known euclidean tilings (Tiling Database http://www.tilingsearch.org/) but only the 17 wallpaper groups (Wallpaper group - Wikipedia) were used for image tiling (which has its reasons but nevertheless).

I am working on a general approach to change this: Every image tiling or image pattern in general can be made by one or more proto-tiles (rectangle or masked polygon shaped images with transparency) and a list of clone-, rotate-, mirror- (flip,flop), and translate-commands collected in a CRMT-command list. A CRMT-interpreter would take such a list and a set of proto-tiles and generate an image tiling, ornament or pattern by step-by-step processing the commands.

For example the following CRMT-command list is coding the 14 processing steps to generate a p3m1 tile from a given equilateral triangle proto-tile image (see my p3m1-examples using this CRMT-approach on ET_p3m1_2018-10-23_01 CRMT-approach | Flickr):

C0, x0, y0, C0, Fo, R-60, x-1/2t_w, y0, C0, Fo, R60, x1/2t_w, y0, C0, Fo, R-60, Fi, xt_w, y0, C0, Fi, x3/2t_w, y0, C0, Fo, R60, Fi, x2t_w, y0, C0, Fo, R-60, x5/2t_w, y0, C0, Fi, x0, yt_h, C0, Fo, R-60, Fi, x-1/2t_w, yt_h, C0, Fo, R60, Fi, x1/2t_w, yt_h, C0, Fo, R-60, xt_w, yt_h, C0, x3/2t_w, yt_h, C0, Fo, R60, x2t_w, yt_h, C0, Fo, R-60, Fi, x5/2t_w, yt_h

The operation sequence for one proto-tile processing like C0, Fo, R-60, x-1/2t_w, y0 is interpreted as: clone the first (starting with 0) element in the proto-tile list, flop it, rotate it 60 degrees counterclockwise (+ trim), make a translation in x-direction with floor(-1/2t_w) where t_w is the width of the equilateral triangle, make a translation in y-direction with 0 and then compose the proto-tile over the tile background which is in the p3m1 case a black image with an (2t_h, 3t_w) area with t_h = 1/2*sqrt(3)*t_w.

Additionally the CRMT-interpreter also needs a list with coordinates for one or more masks that must be draw because the masked proto-tiles must come somewhere; in the p3m1 case the x and y coordinates of the triangle are: x_coord = [0 t_w t_w/2]; y_coord = [0 0 t_h];

Such an approach is not fast but universal and because of the no-overlap condition of tilings the processing steps for one proto-tile are independent and therefore the 14 steps in the p3m1 case could be made in parallel. And there is always the option to optimize some specific tiling by using some knowledge about its structure. In the p3m1 case run-time can be saved by the knowledge that the lower half is a flipped version of the upper half.

To further develop the CRMT-approach I am searching for

1. Programmer implementing CRMT interpreter in different environments

– environments that are free available like G’MIC or making a CLI program with Python or Perl in combination with a free image lib like ImageMagick (http://www.imagemagick.org), GraphicsMagick (http://www.graphicsmagick.org/ might be faster than IM) or GeGL (http://gegl.org/ basis for GIMP)

– environments that are widely used by students and professionals like the three big M’s: Mathematica, Matlab, Maple. Mathematica has by far the larger audience in academia but Maple has the better tool to translate code in the target languages C, C#, Fortran, Java, Matlab, Perl, Python, and VisualBasic so that one Maple implementation could serve as a first prototype in those languages.

Programming a CRMT-interpreter seems neither a difficult nor a too costly task because the core is some string processing combined with calling some image processing functions. And if the used image processing system has RGBA abilities and boundary methods like “-virtual-pixel mirror” it makes everything much easier because composing polygon shaped images results in artifacts at the edges if non-90 degree polygon-masks were used.

I have programmed a (not as reliable as wished) batch processing prototype in Matlab but because Matlab lacks both RGBA abilities and boundary methods it was unnecessary complex and a costly post-processing step was needed to deal with the edge artifacts. I will share the code with interested developers for inspection but for sure in the end they will come up with more effective and efficient implementations.

2. People writing their own CRMT lists

Every high school student with basic trigonometry knowledge can determine the angles, lengths and distances in a given tiling image from the Tiling Database to write a CRMT-list without any knowledge of a specific programming language or image processing, so there is potential a huge community. It is only a question of persistence to code even the most complex periodic tilings like Islamic patterns (https://patterninislamicart.com/drawings-diagrams-analyses/1/elements-art-arabe).

Math teachers are often looking for something motivational. Real world applications and aesthetics in math (see Bridges conferences http://bridgesmathart.org/) are mostly the areas they come up with and I think that image tiling is an excellent example for the later. Writing a CRMT command list for a tiling would be a nice assignment if intermediate states could immediately be checked for feedback with a freely available CRMT interpreter.

Future perspectives

  • Using trigonometry to extract information from given tilings, ornaments and patterns will restrict the audience mostly to academia and some math enthusiasts. A much wider audience would be reached with a GUI based system where humans can drag&drop proto-tiles to generate pattern. Combined with an interaction log that records the relevant actions and a CRMT-optimizer that combines all the actions related to one proto-tile to one command set a CRMT-list for a pattern should be reconstructible. I am thinking about modifying the freely available Mathematica demonstration http://demonstrations.wolfram.com/TilingConstructorTileDraggingVariant/ and the Giri App (https://twitter.com/GirihApp, Girih App | Stefan Hintz) is also worth a look how a GUI type system could be made. If in a long-term perspective AI-agents with a learned sense for symmetry and aesthetic can play such a system an endless stream of interesting pattern descriptions would be accessible that can directly be used in every environment with a CRMT-interpreter.

  • The CRMT approach is also extendible in many directions like simply adding a new command type “S” for scaling which makes image types like euclidean Fractal Tiling (Compendium of Fractal Tilings), Iterated Function Systems and orbit trapping (Fractal Explorer Pixel Bender Plugin • subblue) accessible.

2 Likes
2

G’MIC is certainly able to help with what you describe.

There is already a command called “montage” that accepts a “processing command”. Seems like a similar command could be written to process a CMRT list.

G’MIC has a powerful expression evaluation engine that would be able to handle the math. And it should not be too hard to work out how to use G’MIC’s string processing to parse a CMRT list. There is even support for parallel processing.

Is this list needed if the proto-tiles have transparency?

And would you please post a more detailed example? Maybe include some sample proto-tiles and the intermediate results of some of the steps of processing the list?

3

Is this list needed if the proto-tiles have transparency?

if you have already RGBA with transparency you could use this. My workflow relies on RGB so the polygon shaped proto-tiles must be first generated with those masks which I choose to do within the CRMT-interpreter

intermediate results of some of the steps of processing the list

I will prepare a sequence of the 14 p3m1 steps today and post a flickr-link
here it is for two kind of sequences for the same tile: https://www.flickr.com/photos/gbachelier/albums/72157703949214445

4

Is “CRMT” defined somewhere - are there no looping constructs or decisions? I guess that has to be periodic tiling only (i.e. penrose would require an increasingly long list)?

I hope you don’t mind me saying… this is possibly more interesting than you make it appear! The instructions are so short/concise as to be a bit scary. Perhaps mnemonics or even full words for the language would be less intimidating (you can always reduce later). Or is the intent for that never to be used directly and only as a backend?

5

are there no looping constructs or decisions

until now the “language” of the CRMT (CloneRotateMirrorTranslate) approach has 6 elements. Perhaps this could some day grow to a “Pattern Description Language” that includes the usual control structures.

I guess that has to be periodic tiling only (i.e. penrose would require an increasingly long list)?

It makes no difference if a command list describes 60 proto-tiles of a periodic or non-periodic tile. The restriction is RAM because image proto-tiles should have an interesting internal texture. I am using as standard t_w = 1000 so if there is a complex tiling type like islamic patterns then moving 60 proto-tiles over a background would have serious RAM requirements.

The economic idea of periodic tiling is to make a rectangle tile from possibly non-rectangle proto-tiles and then make an arbitrary large pattern by making a n x m array of the tile (translation vector (1 1)). Or to make a non-rectangle basic cell from the proto-tiles like a hexagon in the p3m1 case and then use a more complex translation vector to distribute such hexagons. Such strategies are not working with non-periodic image tilings where any pattern detail must always be constructed by moving the proto-tiles.

this is possibly more interesting than you make it appear!

I do not know how this could possibly be make more interesting than by linking to the incredible aesthetic result pictures as motivation for developers and list writers. And the writers have also the motivation of sustainability because command lists can generate images even decades later independent of any actual interpreter implementation.

Or is the intent for that never to be used directly and only as a backend?

yes a user don’t have to get in contact with the content of the command list and the coordinate list. In my prototype this two files were selected by giving their paths and the program is doing the string and image processing.

1 Like
6

Since there are so few, could we have a clear definition for each? I propose something like:

Clone <element> | C<element> :
    Duplicate tile number <element> to current tile.

Rotate <angle> | R<angle> :
    Rotate  the current tile by <angle> degrees

… and so on. Obviously however you choose, but if there’s a nice clear definition of the “language” (even with a visual example for each) the task will perhaps be easier to follow.

7

I don’t get along with the formatting of < and > you used in your example so I using "

Clone “element” | C"element":
Duplicate proto-tile number “element” from the prototile_list/cell and make it the current proto-tile for the following processing steps

Rotate “angle” | R"angle":
Rotate the current proto-tile by [degrees] (plus additional trim = auto-crop of borders that are possibly generated by rotation Cutting and Bordering -- IM v6 Examples)

Flop | Fo :
Flop the current proto-tile = left-right mirroring

Flip | Fi :
Flip the current proto-tile = top-bottom mirroring

x-translation “length” | x"length" :
Translation of current proto-tile in x direction by “length” [pixels]

y-translation “length” | x"length" :
Translation of current proto-tile in y direction by “length” [pixels]

8

Thankyou, that’s quite clear enough :slight_smile:
I’ll have a look at it now, but if I run out of time will be another week. That of course doesn’t preclude others from investigating :wink:

1 Like
9

@garagecoder, maybe I can help saving you a little bit of time. I’ve finally managed to implement a beginning of a Style Transfer filter (https://twitter.com/gmic_ip/status/1066460833021157381).
It does not really use the idea I’ve talked with you a few days ago, so just forget about it :wink:

10

Wow nice stuff, that’s good news! Work became… intense this past week so frankly I had to forget about it for now anyway :smile:
But now it’s is in my head, it’s staying on the (extremely long) list of ideas!

11

I’ll use one of your tiles here (tell me if this is not acceptable and I’ll remove it!). Here’s how I interpret the method, please correct if I’m wrong:

You begin with one or more polgyons, defined using rectangular raster image and opacity mask - please excuse the rough cropping.
tile0

This is copied/mirrored/rotated/translated and rendered to a target output image. An example of “Fo, R-60” in G’MIC:

gmic tile0.png mirror x rotate -60 autocrop

tile1

The translations tend to be done in terms of a predefined constant (here triangle side length, height) and a factor.

Given that, I can think of 3 ways to handle it (without changing the “language”):

  1. Raster based (the current method)
  2. Vector field based (manipulate source pixel positions rather than the data itself)
  3. Vertice based (not thought this through but I think it’s possible)

Option 1 is quite simple in G’MIC, but has downsides - aliasing/loss of quality. What do you think? I suppose if you only ever clone “tile zero” then it’s not an issue.

Edit:
Here’s a quick demo of how you can simply define some custom commands in G’MIC to do this:

_tile_Translate : skip ${1=0},${2=0} # this command actually renders as well
  split_opacity. n. 0,1 j[1] ..,$1,$2,0,0,1,. rm[-2,-1]

tiling :
  v -
  t_w={w} t_h={round(1/2*sqrt(3)*$t_w)} # define constants
  {3*$t_w},{2*$t_h},1,3 # create the background image
  m "_tile_Clone : [$""1]"
  m "_tile_Rotate : rotate. $""1 autocrop."
  m "_tile_Flop : mirror. x"
  m "_tile_Flip : mirror. y"
  _tile_Clone 0 _tile_Translate 0
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[-1/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Translate {[1/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[$t_w,0]}
  _tile_Clone 0 _tile_Flip _tile_Translate {[3/2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Flip _tile_Translate {[2*$t_w,0]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[5/2*$t_w,0]}
  _tile_Clone 0 _tile_Flip _tile_Translate {[0,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[-1/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Flip _tile_Translate {[1/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Translate {[$t_w,$t_h]}
  _tile_Clone 0 _tile_Translate {[3/2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate 60 _tile_Translate {[2*$t_w,$t_h]}
  _tile_Clone 0 _tile_Flop _tile_Rotate -60 _tile_Flip _tile_Translate {[5/2*$t_w,$t_h]}
  k[1] v +

The trick here is to define translation as actually drawing the tile. It shouldn’t be too hard to turn this into a filter which accepts a command string, but I find string handling in G’MIC tedious and I ran out of time for now! Seams are clearly visible (I didn’t try to get a better crop) but that can be dealt with in other ways later…

gmic tile0.png tiling

testtile
testtile2046Ă—1182 2.13 MB

2 Likes
12

I need a floor with a pattern like this… a much duller version. :smile:

13

Hello garagecoder,

thank you very much for your commitment!

The cropped proto-tile is totally fine. Attention must be drawn to the edges; if there are artifacts here they are multiplied in the resulting tile.

The stripes you mentioned are the edge artifacts that exists if non-rectangle polygon shapes were composed which is mostly the case. They are the main aesthetic problem with image tiling. The are appearing in my Matlab prototype more than in my (unfinished) prototype with PerlMagick&ImageMagick and my main work was to find post-processing methods to correct them. I was not able to find image processing methods that can correct both the heavy black edge artifacts and the more subtile ones until I started using edge masks of such a tiling. In the last days I finished a version that is using a drawn edge mask given by a set of line coordinates and the Matlab function inpaint_nans. This version delivers finally!! a satisfying quality but is only acceptable fast if the simple averaging method is used instead of the PDE-inpaint methods. I will publish in the next days a flicker album with this method to show the subtile advancements compared to the album I linked in my first post.

I am very curious how this can be solved in GMIC; probably with a boundary method because this are gap-artifacts.

Option 1 is quite simple in G’MIC, but has downsides - aliasing/loss of quality… I suppose if you only ever clone “tile zero” then it’s not an issue.

I think it is not an issue if there are no longer processing pipelines with quality disturbing operations are used. If every proto-tile (independent of the number) is directly cloned from the original list instead of reusing an already processed (rotated) proto-tile then the effects are minimized.
Mirroring and translation are non-disturbing matrix operations but rotation can cause massive disturbance dependent on the used interpolation method. “nearest” is very bad but default is mostly bicubic but this causes edge problems (From the imrotate doc: “This interpolation method can produce pixel values outside the original range.”) and I suspect that this is the most part of the edge artifacts in the tiling; the rest might be inexact autotrim because every pixel counts here. Using bilinear is causing a poorer quality of the shape texture so this is not an option for me; it is already an annoyance that there is not the opportunity for user defined interpolation in imrotate as there is in imresize where I defined Mitchel as a better option to bicubic.

Vector field based (manipulate source pixel positions rather than the data itself)

I can only speculate that this option is an analogy to meshgrid processing in Matlab. I find this very interesting!! because I have speculated about the existence a meshgrid representation of a whole! tile: every element in a [2t_h 3t_w] meshgrid points to a pixel in the proto-tile polygon image which means several elements points to the same pixel. The tile image is then generated by an interpolation of the meshgrid and the proto-tile image instead of a sequence of image processing procedures.

In my batch processing context this would be a huge advantage because such a vector/meshgrid representation must be generated only once and it is then applied to all the input images that are processed. At least in Matlab the interpolation is much faster than a longer image processing pipeline which would lead to a huge efficiency boost. Additionally I believe that the interpolation could also be a solution for the edge artifacts. But I did not came far in an implementation of a tile meshgrid representation; perhaps some day a case for a freelancer-com project.

14

I must keep my answer short for now, but fear not - I’m very aware the edge artifacts must be dealt with, preferably in a way easily replicated in other environments. With G’MIC we’re fortunate to have several inpainting algorithms ranging from very fast to very clever (hence not so fast…), but there are several other options. It needs thought and experiment, which I’ll have time for next week! Again, anyone else is completely free to chip in of course :slight_smile:

Edit: almost ready to push the filter (it’s very rough)

CRMT_alpha
CRMT_alpha1803Ă—1084 722 KB

15

Filter added, I think only for G’MIC version 2.4.2 (and probably an hour or two before it’s available to download with filter update). Some gotchas:

Only working with one input tile.
Inpaint is rudimentary, don’t try to use it unless the background is full!
I’ve stuck with the design of using “Translate” to actually draw, the format of which is T[x_coord,y_coord].

So the equivalent command list you can paste in for the above:

C0 T0
C0 Fo R-60 T[-1/2*$t_w,0]
C0 Fo R60 T[1/2*$t_w,0]
C0 Fo R-60 Fi T[$t_w,0]
C0 Fi T[3/2*$t_w,0]
C0 Fo R60 Fi T[2*$t_w,0]
C0 Fo R-60 T[5/2*$t_w,0]
C0 Fi T[0,$t_h]
C0 Fo R-60 Fi T[-1/2*$t_w,$t_h]
C0 Fo R60 Fi T[1/2*$t_w,$t_h]
C0 Fo R-60 T[$t_w,$t_h]
C0 T[3/2*$t_w,$t_h]
C0 Fo R60 T[2*$t_w,$t_h]
C0 Fo R-60 Fi T[5/2*$t_w,$t_h]
16

The text formatting garagecoder used in his last post is also a template for parallel processing: every line is independent and can be processed in parallel.

In the meantime I have worked on the collection of CRMT-lists (and connected mask coordinate lists) of the non-trivial wallpapergroups (= with non-rectangle proto-tiles), so available are now:

p4m, p3m1, p6m, p31m, p3, p6.

What might be a good way to publish this and future lists?

17

Entirely your choice of course, but I’m wondering about adding some “presets” to the filter (where you choose one from a drop-down list).

@David_Tschumperle more questions I’m afraid!
What’s the best way to test a new version of a GUI filter? Overriding commands works fine at command line - just add to .gmic file. Doesn’t appear to work for GUI (it’s like it’s “locked”).

Is there a way to add multiline text to a GUI multiline text parameter? I can’t seem to figure that out (single line works no problem).

18

Ah yes, this is a bit tricky, as putting a “\n” doesn’t substitute it by the ASCII code 10.
But the following sample works for me:

#@gui Filter : filter,filter
#@gui : Text = text{1,"Really,that's\na multi-line\ntext !"}
filter :
  to "$1"
  LF={`10`}
  u "{And this can change"${LF}"with multi-lines"${LF}"too !!}"

That’s a good question, the GTK version of the plug-in was giving the priority to the commands defined in the user .gmic file, but it seems it’s not the case anymore. I’ll ask Sébastien why :wink:

1 Like
19

some “presets” to the filter

I think that the wallpapergroups are the first choice for such presets like in the old Terrazzo filter because they are basic for all periodic tilings and they are using only one proto-tile. I attach the other CRMT-lists and the masks for further usage (the commands relay on the specific mask shapes and orientations); be aware that they are much less tested than p3m1.

CRMT_p3.txt (174 Bytes)
CRMT_p4m.txt (164 Bytes)
CRMT_p6.txt (375 Bytes)
CRMT_p6m.txt (772 Bytes)
CRMT_p31m.txt (505 Bytes)
p3_mask.txt (122 Bytes)
p4m_mask.txt (59 Bytes)
p6_mask.txt (122 Bytes)
p6m_mask.txt (67 Bytes)
p6_mask.txt (122 Bytes)

1 Like
20

Should be able to include those, probably compress quite well.

I was thinking about vector maps, it turns out very simple for a single tile (edge aliasing issues aside). In G’MIC all you need is a vector valued tile as the input (each vector at x,y holds it’s own position) - very trivial to produce:

gmic tile0.png f ":i3>0?[x,y,z,i3]:[0,0,0,0]" o vectortile.tiff,float

Then when you create a completed tiled image as usual, what you have is a vector map. Now you can take any tile of the same dimensions and do:

gmic vectormap.tiff tile0.png +warp. ..,0

warptile

The top part is the “warp map”

For more than one input tile it becomes more tricky, you’d probably need to save an extra channel in the map specifying the source tile (amongst other things).

Edit: what I mean about using a warp map is you don’t even need the opacity mask, only that the whole tile dimensions match. Example:

tiledcats
tiledwall

1 Like