G'MIC stability under OSX

Software G'MIC
21

Hi David!

Here is the log file produced with the suggested printouts and with the RL deconvolution filter, which was previously crashing with the stack trace reported here: G'MIC stability under OSX - #11 by David_Tschumperle

Unfortunately, the small modification to the gmic.cpp code completely changes the symptoms of the crash, not surprising if we are facing some memory corruption.

On the there hand, I think that the working and non-working filters definitely show a pattern: all filters that are “native” (anisotropic smoothing, diffusion, median, patch-based, guided, …) work reliably, while those that involve a certain fraction of scripting lead to crashes.
In my opinion this means that we should search the issue inside the script interpreter (probably in the string handling) and not in the filters.

Would it be possible for you to provide few “dummy” scripts of increasing complexity, which I can include in my commands definition file and test for crashes?
Meanwhile I’ll try to install valgrind on my system, maybe it will help as well…

22

Ha that’s interesting. Thank you
There are many lines at the end of your log file, like:

data = (char*)0x112bec6d0 ..0x112bec6d2 (non-shared) = [ '45a r118g 30= ]

which means that there is clearly something wrong in the way the text is output (the last string between [...] should be a list of float values).
Could it be that the printf() calls (and derivative snprintf(...) and so on…) are just not re-entrant neither (just as scanf we fixed some days ago) ? Maybe this means we could just try to fix it with the same kind of macros we did for CImg ?

If only that could be as simple, it would be really nice :slight_smile: (and of course, I can’t test it).

24

I’ve committed a code modification to make CImg use mutex-protected versions of sprintf() and snprintf() when doing it on Mac.
Could you please get it and test if that makes things better ? Thanks !

26

Ok, I will test it asap!

27

Hi David!

An update on the OSX issues: I have tested the new code with your reentrant sprintf implementation but the crashes are still there.

However, I have managed to write a minimal example of failing code. It repeats a given gmic instruction several times, either directly from the main thread or asynchronously from a separate “producer” thread.

The gmic instruction is very simple: I create a 128x128 pixels image, then I create a mic object and I use it to run the “total variation smoothing” filter on the 128x128 image. Then I delete the mic instance first and the allocated image next.

This simple code unit is first called 100 times from the main thread, and then 10 times by the producer thread. However, the code crashes as soon as gmic is called by a thread different from the main one.

This is consistent with what I’ve been observing in photoflow: at least in my case, the problem does not seem to come from the parallel execution of gmic code (I only have 1 producer thread in my example), but from the mere fact that gmic is not called from the main thread.

I could not go further than that for the moment, however you can already have a look at my minimal example here. Maybe you’ll simply find some obvious mistake I’m doing…

28

Thanks. Your code looks correct to me…
How frustrating it is :frowning:

29

Hi!

I’ve now compiled GIMP and the G’MIC plugin myself under OSX, however I doesn’t work properly for the moment… I’ve copied the plugin under $HOME//Library/Application\ Support/GIMP/2.8/plug-ins/ and it appears in the “Filters” menu, but when I try to apply any of the gmic filters, nothing happens.

If I enable console debugging, this is what I get:

[gmic_gimp]./preview/ -debug -gimp_gaussian_blur_preview 10,0,0,1,0,0,0
[gmic]-1./ Start G'MIC interpreter (in debug mode).
<gmic>-1./ Initial command line: '-debug -gimp_gaussian_blur_preview 10,0,0,1,0,0,0'.
<gmic>./ Decompose command line into 3 items: 
<gmic>./   item[0] = '-debug'
<gmic>./   item[1] = '-gimp_gaussian_blur_preview'
<gmic>./   item[2] = '10,0,0,1,0,0,0'

<gmic>-1./ Enter scope './'.
<gmic>-1./ Item '-debug', selection [0].
[gmic]-1./ Abort G'MIC interpreter.

<gmic>-1./ Exit scope './'.

Any suggestions?
Thanks!

30

Yes, unfortunately, there is a bug in the gmic_gimp.cpp file in latest 1.6.5.2 version of G’MIC.
Please get the latest git version, where this bug is corrected.
On MacOSX, any G’MIC pipeline is aborted by the plug-in, which means you can do actually nothing :frowning:

31

That’s a bit too drastic isn’t it? :open_mouth: I still hope we will finally fix the OSX bugs! :wink:

32

That was not an intended behavior :smile:
I hope so, this is really annoying to have so much problems on Mac.

33

It works! Now I can start testing parallelization issues in the gimp plug-in!

1 Like
34

Do you have any example of filters that crash when used in the gimp plug-in?

35

I don’t know. Maybe Karsten could help. Probably all parallelized filters (like the Smoothing filters in ‘Repair/’) have problems.

36

Hi, I have actually the problem that I don’t get a crash of the plugin! So normally switching Parallel on produces them.

Still I am supiscious that parallelization is not enabled although I have set the flag.My activity dont show it!

This is a build from git (1653_pre) compiled with the native compiler (10.10.5)!

The cli still crashes always with -x_hough (bus error 10) and that happens with Custom code “-x_hough” in the plugin too. Maybe by other reasons! I have display enabled too in the plugin!

The smoothing filters produce timeouts, if that could be switched off crashes probably appear again.

I have now tried “-demo 1” via the custom command in Gimp. Demo works in parallel but my mac does not take more than 110% cpu for the plugin! Several animations are quite slow (2fps). That is not normal

Karsten

37

Crashes are indeed really random… in my case, -x_hough seems to work fine in the cli if I do not pass any external image (in which case an internally generated image is used), while it crashes with an external one. In other words,

./gmic -x_hough

is ok while

./gmic image.jpg -x_hough

fails.

By the way, what do you exactly mean when you say “switch parallelisation on”?
Is it -Dgmic_is_parallel or -Dcimg_use_openmp or both? I’m only setting the first flag for the moment.

My personal feeling is that the troubles start when gmic calls itself (i.e. nested calls to gmic::_run()). I have started to play with the freaky details filter, and again

./gmic picture.jpg -_gimp_freaky_details 2,10,1, -output out.jpg

works fine while

./gmic picture.jpg -gimp_freaky_details 2,10,1,1,2,1 -output out.jpg

fails with a segfault. If I’m not wrong, the second call uses a single thread, but goes through the -gimp_parallel_overlap function, isn’t it?

Would it be possible to “simplify” the freaky details filter (for example removing the -repeat commands)? Just to see at which point things become unstable…

38

Hi, I don’t use openmp, since the apple g++ don’t support it. I have tried it with the macports clang 3.7 but that was no improvement. Since my public build of the plugin should not need extra libraries beside GIMP I stay with the apple compiler. For the cli everything is possible, still I have tried there beside MacPorts the compilers from High Performance Computing for Mac OS X http://hpc.sourceforge.net/ and from François-Xavier Coudert http://coudert.name/ without any improvement.

As you say, it is in a certain way random. At least crashes are stable!
Karsten

39

Ok, I’m also using the default clang from Xcode, not using macports or homebrew but compiling everything myself with jhbuild. So our environments should be quite similar.

However, in this case in order to force the parallel computing with the CLI one has to go through -gimp_parallel_overlap, right? Except maybe for the demos, which I don’t know very well…

40

On my machine both calls work flawless!

41

I am using the “-demo 1” and "-parallel “cmd”,“cmd1"” or on several images “-apply_parallel “cmd””.

42

Hi all!

I’m beginning to believe that at least part of the issues are related to the -local command.

I’ve been playing with the freaky details filter, trying to make it more and more simple (although at some point not really meaningful, but for testing that’s OK).

Here is what I have at the moment:

_gimp_freaky_details2 :
      [-1] ---[-1] 255 -*[-1] -1
      -bilateral[-1] $2,{1.5*$2}
      #-blend[-2,-1] vividlight 
      #-blend overlay

As long as I do not include the blending steps, all seems to work fine. However, as soon as I apply one of the blending (and therefore the -local command gets used internally) the code crashes if it gets executed in a thread different from the main one.

In other words, the crashes I see are not directly connected with parallel processing, but to - local commands being executed in a spawned thread.

As soon as I manage I’ll look into the gmic code for the local command, but maybe this piece of info will also ring some bells? :wink: