Well, it’s just the behavior of the white, difference in quality is a bit unsettling for me:
This white is on the layer above the black one, and I’m just shifting between RGB (linear) and RGB (perceptual). But it always acts like this (bit more indexed) when it’s in contact with black on the same layer, except when the brush is switched to Legacy… to sum up previous posts : ).
Hi @gungan - I looked at your example3.xcf.
Then I made a new XCF file - 800px by 800px with a black layer and a transparent layer, and made three white brushmarks on the transparent layer, using default/auto for the brush and layers, making each mark one after the other all in the same spot, using guides and “snap to” to make sure each successive brush mark was in the exact same place. I made sure to use your exact brush parameters. This is a screenshot at 200% zoom so the edge posterization can be seen clearly:
The result certainly is posterized at the edges, “steppy looking”. It’s less strong though still obvious, if you use Force=50.0 instead of Force=100.0. 50.0 is the default Force setting. But even at Force=50.0, when zoomed in the edges aren’t nice to look at.
@gungan - is this what you’ve been describing as “indexed looking”? Or do you mean something else?
I don’t know what is the source of this degree of posterized edges - maybe it’s just a normal result of painting with linear RGB. Normally when I paint I use a largish canvas and don’t normally zoom in more than to 50%. Plus usually I paint using a very reduced opacity and the brush is always moving and set to “incremental” rather than being left in one spot. So I’ve never noticed this sort of “steppy edges” result before.
I want to check this and that to eliminate various possibilities regarding the amount of edge posterization, starting with checking Krita and also checking an older version of high bit depth GIMP that I modified in various ways.
@anon11264400 - do you see anything like this “steppiness” at the edges when using Blender, using as similar as possible method of putting down three successive marks in the same place? - assuming of course that Blender can be used to paint dabs as described, no transforms to make prettier results - everything has to be linear!
Setting Krita up to make a similar brushmark in the linear gamma sRGB color space, the same posterized edges appear. Using my modified version of an older version of GIMP, the same posterized edges appear though perhaps not as extreme - brush settings are very different in the two programs so I just made changes to the default Krita brush tip settings until it had a similar fall-off from center to edge, compared to the brush we’re using in GIMP.
Here’s a screenshot of the Krita brush marks at 200% zoom:
The weird thing is that there are no posterized edges in GIMP when drawing a radial gradient from white to black with the same radius as the brush stroke radius, even if the layer is subsequently modified with Curves or Levels to look more like the brush mark. But as soon as the radial gradient layer is turned into a mask and added to the layer, the posterized edges appear.
I’m working at 32-bit floating point in both GIMP and Krita. So I’m wondering if possibly the mask and/or the brush stroke and/or the alpha channel is somehow at 8-bit precision instead of 32-bit floating point precision in one or both of the programs. I’ll try asking around to find out, unless maybe someone reading this thread already knows? @Boudewijn_Rempt ? @Jehan ? @Americo ?
Can you check the values? Are there tween values that are being quantised or are they solid blocks of quantisation? It sure looks like quantisation errors.
The other issue could be that the brush itself isn’t painting in float. That is, it could be upsampled to float, but the posterised edges are present in the brush itself, as opposed to a smooth gradient.
Bingo. That sure sounds like the culprit. Check the tween values.
That was exactly what I was talking about, as you say it doesn’t happen with black in GIMP. Though I didn’t notice that in Krita with a normal layer, I guess I don’t know hot to reproduce it…
So I didn’t know if this could be considered a form of bug or something, cause I didn’t understood the “technology” behind it and wanted to check first with you.
Putting multiple sample points in each apparent block of quantized pixels, indeed the values are all the same:
This seems to rule out another possible culprit, which is that maybe the colors in the image weren’t actually quantized but rather merely looked quantized as displayed on the screen.
Hmm, well, I didn’t actually mention the difference between black brush mark on white and white brush mark on black. There are areas of quantization in both cases, it’s jut not as obvious with black marks on white. But the dividing lines between the various quantized areas appear to be in the same place for black on white as for white on black - zoom way in and look closely, and you’ll see it.
There really are quantized-appearing areas even for the radial gradient, but the “lines of quantization” are much closer together and I’m guessing this might be a case of screen quantization. There’s always quantization on the screen as the screen only displays integer increments (speaking of normal screens, not $10,000 state of the art screens used in the movie industry). And if one wants to be super nit-picky even floating point numbers are quantized at the bit/byte level, though the size of the increment is presumably hugely smaller than the human eye can discern.
I don’t know how to export a Krita brush, but here’s a screenshot of the settings I used. I don’t have my tablet hooked up at the moment, so this is just a basic “mouse only” adjustment:
It sort of looks like a bug to me. But maybe it’s a deliberate compromise/trade-off between speed and accuracy, in which case it still could be considered a bug depending on one’s priorities when painting.
If it is brush bit depth, it sure is a bug. One would expect brushes to be aligned to the reference space, both colour space and bit depth I would think.
Perhaps not a bug that can be repaired immediately, but should be on the TODO at the very least.
i note that the transparent layer isn’t needed to reproduce this and that brush hardness affects the position of the quantization considerably. On a quick inspection the brush preview is 8bit but i’m not sure that’s used for painting.
However, paint/gimp_brush_core_subsample_mask() has a suspicious-looking 127 in it.
So maybe the brushes are 8bit?
@gungan - if you would like to file an official bug report, that would be nice - maybe reference this thread and of course post your example images and/or any of the screenshots I uploaded to this thread.
This quantization really does look like a bug. If perchance you haven’t ever made an official GIMP bug report before, this link has the “how to” information: GIMP - Bugs
I suspect @barefootliam and @anon11264400 are right, that the brushes are still 8-bit brushes, and that might be the problem. But either way, those quantized edges aren’t pretty!
Ok, I will do that, probably until tomorrow, hope I sum it up correctly. But I will probably reference this thread as well.
If that’s the case, it is a double whammy. First it is quantised at eight bits and upsampled, so there are canyons between values. Worse, there are collisions in all of the lower values potentially as they collapse into a many-to-one situation, hence the huge posterisation gaps.
Should I report the issue, or maybe some of you guys? I’m thinking you have a better insight in what exactly is going on in the back end, so you’d explain it better? Just wanted to check with you.
The actual bug report is just that there’s very visible posterization along the edges when making certain kinds of brush marks. Anything beyond that is just guessing.
If you make the bug report, then as soon as I see it I’ll post a follow-up comment confirming your findings - I check GIMP bug reports every morning, or you could post the link here. But if you prefer, I don’t mind making the bug report - your sharp eyes have found an issue that does need attention, and having seen that edge posterization I think I’ll have trouble “unseeing” it 
.
Well, here you go:
https://gitlab.gnome.org/GNOME/gimp/issues/2372
I tried to simplify it as best as possible, and to follow the bug template. I hope it’s alright. Do keep an eye on it a bit, cause I’m all over the place 
.
Thanks much! for posting the bug report - very clear! I’ll add some comments/confirmation/etc tomorrow morning first thing.
In krita, brush masks are indeed 8 bit.
The gbr and gih brushes are 8-bit brushes - if you open one of these brushes as an image, change the precision to 32f and then export again as a new brush, and reopen the exported brush, it’s at 8-bit precision.
I thought there was an open bug report about GIMP gih and gbr brushes only being 8-bit precision but I can’t find it - the gitlab bug reports are not easy to search, or maybe I just haven’t figured out how to use the search interface. Apparently the vbr brushes also are 8-bit brushes.
Thanks! for the information. Are the Krita layer masks also 8-bit? Or maybe just the histogram binning when using Levels/Curves on a layer mask? I remember being puzzled about this the last time I used Krita for painting.
If you were painting an image that’s in the regular sRGB color space in Krita, you wouldn’t notice edge artifacts because the regular sRGB color space has an approximately uniform Tone Response Curve.
GIMP is a bit different from Krita in that for Krita, to paint using linear sRGB, the image has to be converted to a linear gamma sRGB ICC profile, which of course should only be done with a high bit depth image. But for GIMP the RGB channel values are automatically converted to linear RGB for most operations, unless you do something like choose Legacy or change the composite mode.
This should depend on the quantise step.
The more I think about it, the more something feels “more wrong” in GIMP. An 8 bit brush, if handled properly, should result in the exact same quantised steps in a linearised reference space, and appear identical to its 8 bit counterpart in a nonlinear reference.
Is it possible the brush isn’t being managed and the 8 bit values are being interpreted as linear values?
