G'MIC and LLMs: What's the Connection?

Hello, everyone,

At a time when the use of LLMs is polarizing society (there are those who use them often, those who are fiercely against them, and ultimately very few people in between :slight_smile: ), I thought it would be interesting and honest to share my humble view on the subject, as a maintainer of modest open-source software (CImg and G’MIC), and how LLMs are influencing/have influenced my development activities.

Above all, I’d like to share my personal experience on the subject, and, most importantly, I don’t want to preach! (everyone is entitled to their own opinion, and we should let others have theirs as well).

As you may know, I am the founder of the G’MIC project (in 2008), and I am currently its lead developer and maintainer. G’MIC’s codebase currently consists of about 200,000 lines of code, including all the code for the various user interfaces (CLI, G’MIC-Qt, etc.), the standard library functions, and contributions from the gmic-community.
Let’s just say it right away: that’s a lot of lines of code for a single person to maintain in his spare time.

So, to make a long story short: yes, I’ve already used LLMs to help me develop G’MIC.

I’ll explain in detail below how I used them and for which parts of the project.

My use of LLMs for G’MIC is obviously fairly recent (I’d say about one year or so), since that’s roughly when I’ve started to find them useful.
Since the G’MIC project began in 2008 (a time when LLMs didn’t even exist), it’s clear that the vast majority of the project was not vibe-coded by LLMs (just in case you had any doubts about that! :wink: ).

1. Project Core

Here, what I call the project core consists of the C++ code for the G’MIC language interpreter, the implementation of all its operators (the G’MIC Standard Library stdlib, which is all written in G’MIC language), and the implementation of the underlying C++ image processing library CImg.
This is, of course, the most critical and complex part of the project (and the oldest as well), so I already had a quite solid codebase for this part, even one year ago.

So here, I mainly used LLMs for code review: I let the LLMs (primarily Claude, ChatGPT, Mistral, and Gemini) analyze the tens of thousands of lines of code that make up this part and provide me with suggestions for improvements or bug fixes.

Note that I didn’t use an AI agent to do the fixes automatically, I preferred to review each suggestion myself and decide whether or not the proposal is worth (and write the fix by hand when necessary). LLMs are quite good in analyzing correctness of C++ code, so I have to say that, I’ve indeed been able to fix quite a few bugs between G’MIC versions 3 and 4. It was a long and tedious process (reviewing code suggestions done by LLMs for 10 to 12 hours a day for one entire week, I can attest that it’s exhausting!).

But I can also attest to the results: in practice, these suggestions from LLMs (sometimes far-fetched, sometimes brilliant) allowed me to make the G’MIC core codebase more robust and optimized, and to do so much faster than if I’d had to do this reviewing job entirely “by hand” (I wouldn’t have had the courage to do that in practice with such a large number of lines of code). That’s a fact.
But clearly, it wasn’t at all a “hands-off” job!

2. G’MIC filters

I’ve also tried programming new filters for the G’MIC-Qt (so written directly them in the G’MIC language) using LLMs, though I have to say I’ve had very little success doing so until this week.

Although the G’MIC language has a fairly comprehensive reference documentation, it remains a niche language with very few code examples available online, so it makes sense that LLMs aren’t very effective at writing G’MIC code. What’s more, the language has evolved quite a bit over the past 18 years, so the language syntax from 10 years ago isn’t supported anymore. And all the LLMs I was testing still tended to mix the syntaxes of different versions of G’MIC, which obviously didn’t work.

In short, I’ve never really managed to extract functional and useful G’MIC code (even with some reworking) from an LLM…

…That is, until this week, when I ran a funny experiment: I generated the entire G’MIC reference documentation as an ASCII file named gmic_reference.txt, and attached it to an initial prompt for Gemini (this taking like 100K tokens in the prompt), asking it to program a G’MIC filter while forgetting everything it already knew about G’MIC and relying solely on the language description in that attached reference documentation. And lo and behold, this significantly improved the quality of the generated G’MIC code.

That’s actually how I implemented the filter Rendering / Chladni Figures, which has been just added in the G’MIC-Qt plug-in a few days ago.

To be clear, it didn’t work right away, but it did after just a few iterations (2 or 3 additional prompts).

This filter is simple enough that I could implement it myself and, more importantly, I could review the code generated by Gemini to understand what it did and improve it if necessary (and it definitely needed some work!). In the end, I rewrote about half of the filter’s code to make it cleaner, but this is the first time an LLM has been able to generate a filter from scratch (at least for me) that was functional. I’d say, it is an interesting step!

Then I tried to keep working in that direction for a few days, and with the help of Gemini and Claude, I managed to develop a second filter, with significant assistance from LLMs, namely the Rendering / Cosmic Nebula filter.

Once again, I was pretty helpful as a human coder in cleaning up, improving, and debugging the code, but the LLMs helped me establish a solid foundation for developing this new filter (and clearly saved me time).

To sum up, developing these two filters wasn’t exactly a walk in the park: it took quite a few rounds of back-and-forth with the LLMs just to get something functional.

So I naturally thought that this back-and-forth could be handled by an AI agent, and I installed one on my PC yesterday (kilo) For the LLM, it uses an open-source DeepSeek-4 model, so it’s clearly less capable in terms of reasoning than what Gemini, Claude, or ChatGPT offer, but it allowed me to see how it works and to start creating an agent specialization dedicated to implementing G’MIC functions.

So far, nothing concrete has come of it, but I’ll continue my testing, and I’m hopeful that this agent can help me in some way to quickly prototype new filters. Just as with I’ve done with G’MIC-core, I want to have the final say on the code anyway, to make it as clean, robust, and efficient as possible. I definitely don’t want to put just anything generated by the LLM into the G’MIC stdlib. I haven’t worked on it nonstop for more than 18 years just to ruin it overnight with pure AI slop without verifying the quality of the generated code!

3. All the extras

I’ve also made quite a bit of use of LLMs for all the “extras” that are part of developing a project like G’MIC: setting up build systems (Makefile and CMake), writing Bash scripts (for the creation of the binary packages, for example), configuring the various virtual machines I use to compile binary packages, writing code for HTML pages, and so on. I have to admit that all of this involves technologies and languages I’m not really interested in and thus I don’t fully master (seriously, who wants to master the syntax of Makefiles or CMakeList.txt files? ), and here, LLMs have also been very helpful to me, improving my configuration files with syntax magic I couldn’t even think of!

4. Final thoughts

To sum up, I’d say that in my situation, as a solo developer working on a relatively large project and striving to make it as easy to install as possible, LLMs have been a huge help in many aspects! Most importantly, they’ve helped me optimize the time I spend on development tasks I don’t necessarily enjoy (“extras”, code reviews), leaving me more time for the development work that really interests me (image processing operators).

My impression is that these technologies will soon make it easier to create truly functional, high-quality G’MIC filters by potentially anyone, developer or not. And in a way, that’s a good thing: one of the initial G’MIC’s goals was to serve as a software platform for developers to create/share/distribute image filters. Now that anyone can potentially become a G’MIC filter developer with some LLM help, why not taking advantage of the good ideas these people might have to enrich the framework with new interesting filters (provided they are of sufficiently high quality, of course).

Other than that, yes, I’m aware of the concerning environmental impact of this technology. For my part, I use LLMs to perform tasks that I consider important enough for developing my open-source projects, tasks that save me a lot of time on aspects that can be a bit tedious (and that I couldn’t do in a reasonnable time without the help LLMs provide, let’s be clear). But I think the key is to strike a balance between the environmental cost and the value/utility of the output generated by these LLMs. But it’s very subjective, indeed (is it important to be able to generate images of kittens in underwear yodeling? I’m not sure about this).

So, that was a recap of my experience, which I hope will help you understand why and how LLMs are used in the development of G’MIC!

18 Likes

Bookmarked for future reading. Given your tendency of late to write long form posts, I wonder whether any of them could be on PIXLS.US - Articles. Food for thought.

1 Like

Personally, i don’t find LLMs useful for G’MIC coding asides for pointing out where’s the bug. But, the level I’m doing my thing is much higher than what LLMs can do. I’m glad it can enable beginners to at least make something out of it.

If the LLM can be used to create an Opus Musivum or Opus Vermiculatum mosaic out of an input reference image simply by telling the LLM to do so with simple words, then that would be impressive. lol

:slight_smile:

Helping solo developers find bugs and write code, allowing more people to create software (even good software), adding new features more quickly…these are all things that LLMs can do.

I don’t think any of the benefits are worth the costs in the long run. I think people who think we can balance the benefits against the costs are dead wrong. The more humanity becomes dependent and addicted to LLMs, the more energy and land will be used to meet the demand.

We’re all going to see a ton of examples where LLMs can work, especially in the hands of experienced developers and on mature projects. But I think it’s obvious that the environmental costs are too much, but then again everyone has a different definition of what is acceptable and people will do what they’ll do. People who benefit a lot will typically justify their reasoning after the fact and that practicality will overrule any other sort of reasoning.

I have to use software as part of my job, and I know I won’t always be able to use software that’s free from LLM influence. It’s just the way of things. But since the introduction of AI I’ve lost any admiration for software projects that use LLMs and now if I have to use such software, I just consider it a practical necessity, when previously it was a source of some pride to be able to use something open source.

4 Likes

@JasonTheBirder

I agree that LLMs shouldn’t be used whenever possible. Regardless, I do think LLM usage can be justified, but it has to be a tool of last resort.

For example, I wanted a iterative solution to finding the next() for permutations which contains all elements from 0-M in N sized vector. I took in months of attempts, ideas, coding up a solution, getting close, but it has never worked. Then I resorted to asking StackExchange. They gave me recursive solution involving resizing array and getting the next. That was not what I needed. Finally, tried Claude, and it has worked out.

But, thing is, this is a small portion of my own code. Less than 1%, actually, less than .1%. I would never use LLM otherwise. For my G’MIC code, my policy is to never ever use LLM, unless it is of last resort. Thankfully, the amount I have to use LLM as last resort is so small, I can literally careless about having LLMs.

I think that’s wrong.

Or maybe it’s just as obvious that all modern technology is clearly harmful to the environment, starting with the very existence of the computer. Still, it is useful in certain situations (but obviously not for watching TikTok videos about makeup or weight training…).

As I was saying, there’s clearly a balance to be struck between usefulness and consumption. What is obvious is that this balance is very difficult to define, and we should’nt take too black-and-white a stance on the issue.

(same issues arise with regard to modes of transportation).

Hello David,

Disclaimer:
I use the AI at work (e.g. for SQL coding…)

Sorry, up front, for asking such a weird question…

Since way back, probably (???) the most requested feature for G’mic is the option to taking advantage of the GPU in order to speed up the filters.
Some of them are slow (e.g. Inpaint patch-based) even with a very powerful CPU. At least compared to the softwares (e.g. commercials) where the GPU is used…

It would be possible to use the AI for coding such a great improvement?

Yeah. I am 100% aware is NOT that simple (hence my question…).
For instance:

  • It would require a complete overhaul of the G’MIC infrastructure;
  • It would need a gigantic amount of testing from your side;
  • GPU drivers are a problematic (often buggy): with darktable, for instance, the OpenCL code is constanstly fixed and updated because of this.

As regards your topic about the AI…
Personally, these last days, I was eagerly waiting to test the new stable version of PostgreSQL (19)
September has always been the month where the annual stable version of this open source database is released.
This year is different because (thanks to…) to the AI :slight_smile:
In short, after testing (debugging) the code through the AI a lot of bugs have been discovered. Consequently this year the developers have decided to postpone the release
Here is the link:

Half a year ago, I experimented with using AI to write a GIMP plugin, but it only worked after many iterations. I used ChatGPT and Gemini, and they sometimes removed comments, added new ones and changed variable or method names for no reason. The most annoying thing was that they switched from the GIMP 3 to the GIMP 2 API.
I concluded that it is hard for AI to write code for a nice tool that has recently changed its API. Perhaps I should test it again.

Your idea of adding a reference text file sounds smart. Is there any reason why you did not create a G’MIC MCP server? As far as I know, MCP servers are the usual way to give the AI more context.

Yes but the key is the direction. We should change directions, not keep going. Your analogy is like saying an overweight man should start eating at McDonalds because he already eats donuts and pie. Fact is, just because we started somethin

Maybe this one needs a “spike offset” to create shooting stars?

I was tempted to feed my old code to Gemini but i’m afraid i’d be the first person to make an AI laugh. :sweat_smile:

Anyway, yesterday i wanted to create a "Cushioned glass window/door’ effect like in the photo i took of my children above. Did one by myself by just playing around with Gmic-qt first to test different effects, then writing a command with that.
I lazily used split to cut the image as a grid and apply effects to each resulting image… but i couldn’t find out how to reconstruct the image afterwards, as all images would not be added in the right order with append_tiles. So, seeing your post, i asked Gemini and it showed me 3 methods. 2 were wrong, like, use interleave, but that doesn’t seem to exist, or sort_list y which isn’t working unless you add + or - as first argument, and doesn’t work anyway in this case.
But the 3rd one was the simplest : reverse the split order :

Method 3: Reverse the split order

If you split by y first and then by x, the default list order already matches append_tiles

And that worked fine for me. That’s all i asked him. I got this in the end :

But i knew using split wasn’t the right thing to do. I knew i ad to use warp or some other stuff. So i fed it the image above and asked it :

How would you create this image effect with Gmic? Anyway to create the same effect with warp?

And it gave me a one liner with renderings… that actually worked. Even though i suspected that there was more than G’mic involved in it since i (really) wasn’t getting the same results when i tried the command in my pc :

And flash forward, after a few more questions to guide it, i finally got this from my pc :

And rearrange it’s one liner to this because it was using fixed values:

pr_glass_door:


  sp portrait8,1500
  n 0,255

  fx_heightfield_warp 1,"0.25*cos(3*pi*x)*sin(3*pi*y)",119,10,1,2.352,47.8207,90.411,50,20,20,40,30,20,1,"50,50",50,50,74.3462,51.0585

  SIZE=$1
  DISP=$2
  WHAT:=$1/2
  OLAP:=$WHAT+$4
   
  +fill "X=(x%$SIZE)-$WHAT; Y=(y%$SIZE)-$WHAT; M=max(abs(X),abs(Y)); M<$OLAP ? 1-(M/$OLAP)^4*0.45 : 0.8"
  mul
  100%,100%,1,2 
  fill[1] "X=(x%$SIZE)-$WHAT; Y=(y%$SIZE)-$WHAT; R=(X^4+Y^4)^0.25; A=R<$OLAP?sin(R/$OLAP*pi)*$DISP:0; [R>0?X/R*A:0, R>0?Y/R*A:0]"
  warp[0] [1],1,2,1 
  rm[1]

And no, i can’t write anything like this fill command above by myself.
For comparison, here is my original mess, which is not even a script :

pr_glass_door:

  sp portrait8,1500
  n 0,255
  fx_heightfield_warp 1,"0.25*cos(3*pi*x)*sin(3*pi*y)",119,10,1,2.352,47.8207,90.411,50,20,20,40,30,20,1,"50,50",50,50,74.3462,51.0585
  +fx_imagegrid {0,w*$1%},{0,h*$1%} 
  b.. 8 b. 6
  blend value,1 
  split y,-$1%
  split x,-$1%
  ap " fx_distort_lens 0.3,0,0,46.8244,46.6999,1 +cl_reliefFrame 3,20.246,30,127,0,11.9,0,0,0,1,192,192,192,0,0,2 blend average,.7 " 
  append_tiles ,

Still, i think both renderings can be improved, they still look a bit flat to me.

Haha, never seen that one…

Not surprising, I’ve just added it after a few iterations with a LLM.
Still WIP, as I’m currently playing with Claude to add some cool parameters for more diverse rendering.

Yes, i think there should be refraction on the grid too, not just pure black.
That’s why i asked Gemini how to overlap the cubes a little. But i’m no expert,ionly have my eyes and a glass door with molded cubes. :sweat_smile:

Version 2:

2 Likes

This one is super cool. Needed quite a lot of interactions with the LLM, but the result is worth!

1 Like

Can you add a 2 layers option so that one can add a second image for the other side?

2 Likes

I second that. :slight_smile:

We are just now in a ramp up to a bursting bubble. The financial costs are immense and backed only by investment, not bey revenue. A lot of the planned data centers will never be built and people will stop using AI so much when they have to foot the bill.
This stuff will stay but not at that immense scale.
Except if the computing needs go down, the need for power, the price and the environmental damage.