This is a topic that I’ve become very interested in, and lead to my involvement in the MyPaint community (a cousin of GIMP) and (trying) to learn programming and math in general. It all started when Anna Timm posted a patch to allow Blue+Yellow to blend into a Green by shifting the Hues more or less manually. I followed the rabbit and eventually found Scott Burns’ work on synthentic and plausible spectral reflectance curves for any RGB color space (He provides detailed information on specifically sRGB).
Armed with this I’ve been working on a patch to MyPaint that does spectral mixing in addition to a bunch of other things (some pretty silly).
My patch largely only affects one particular brush setting; smudge. I haven’t done anything to layer modes, etc. What I do is take the sRGB colors and do a quick lookup in a massive 3DLUT that maps all of sRGB space to unique spectral curves (36 lights). Then the colors are mixed in this 36-light linear cauldron and transformed back to sRGB. The performance is quite good despite the 2GB memory overhead, and the results are incredibly satisfying despite often subtle in appearance. The colors mix in a more intuitive way and makes the “feel” of the paint much more realistic and predictable. What I mean specifically, is that when you blend just a tiny bit of white into a color, it reacts according to your desired blend ratio based on linear input (assuming pressure here). Whereas with non-linear, the blends swing wildly and it is hard to achieve smooth transitions. I suppose when you look at a gamma curve this all makes sense. The same thing happens, I think, with the smudge trail. In non-linear blending the “smear” is very short because the linear distance does not align with the non-linear blending. Distance is linear, right? So now with the smudge/blend ratio changing in-step with the distance the brush traveled, it all works very nicely. At least, that’s what I think is going on.
The caveats are that for best results you need a 100% opaque brush and disable some settings such as anti-aliasing, etc (anything that uses non-linear blending). Despite this, the brushes you can make are extremely useful since the “smudge” setting can effectively stand-in for opacity in a lot of ways. So, mapping smudge to pressure is a pretty good option. I have a lot of images and videos on my site demonstrate these.
Although Scott’s work talks a lot about Subtractive mixing, it finally clicked with me that this would work for normal blending too (or additive, etc); it’s exactly the same concept as RGB but a lot more lights. So, my MyPaint patch can be a cool learning tool since you can slide the sliders to switch from RGB to Spectral simultaneously while switching from Normal to Subtractive mode. By preventing absolute 0 there isn’t much issue with Subtractive’s multiply method.
There are a few other settings that aren’t as exciting; I allow tweaking the blend result chroma and luma via HCY space depending on the hue angle difference of the two colors. This was an attempt to increase the saturation cost when blending dissimilar hues, but since I can reverse that process it becomes more interesting (create surprising flares of pure saturated colors, for instance).
Another cool feature extends the usefulness of a patch by Anna TImm that provides us with brush offset settings. Anna’s patch allows creating wide flat brushes since the dabs can be offset by some pattern to the either side of our path. I’ve added a settings called “smudge buckets” that adds 10 additional smudge states. So when you offset dabs into, say, 11 “bristles”, via a stepped-waveform graph on a random number input source , you can apply that same exact pattern to control which smudge bucket is used for those offset dabs (correlating particular dabs to a particular smudge bucket/state). The result? Each “bristle” picks up its own unique color from the canvas and smears that along its own path. This is in contrast to the default singular smudge state that averages everything into one bucket, muddies the colors and causes major problems when blending near the edges of objects (merely nicking the edge of a colored object could instantly affect the color of the entire brush instead of that one particular bristle).
Another project I’m working on (with some coaching by Troy S is getting OpenColorIO integration within MyPaint, and allowing scene-referred painting as well as the obvious benefit of display color management. Initial patch is here, which does the basics of transforming the drawing canvas and just about all the UI from sRGB TRC to your particular display via your provided 3DLUT. The next step is to get the internal reference a numpy32 float array and allow selecting you color space. Also, switching cairo to use RGB30 for the display GUI should help prevent issues with wider gamuts.
I would love any feedback or support of these endeavors, fork my code, fix the stuff, etc
you may want to use science to avoid this . there’s quite a bit of work on this topic, these may be entry points:
D. L. MacAdam. Maximum visual efficiency of colored materials.
Journal of the Optical Society of America 25(11):361–367, 1935.
Brian Smits. An RGB-to-spectrum conversion for reflectances.
Journal of Graphics Tools, 4(4):11–22, 1999.
Johannes Meng et al. Physically meaningful rendering using tristimulus colours. Computer Graphics Forum (Proceedings of Eurographics Symposium on Rendering), 34(4):31–40, 2015.
Thanks!! I was able to access Brian Smits and Joahnnes Meng’s papers. I don’t really understand math that well (closer to “not at all”), but it looks like both of their methods use interpolation to avoid the expensive fitting of the “smoothed curve” that natural surfaces reflect. So, using a precalculated LUT might still be a better option for performance and quality? Memory is cheap enough :-). Actually, I think I’m still doing interpolation, since my table is 8 bit (256^3) but mypaint is internally using 15 (16?) bit floats for color mixing. I’m just rounding to the nearest integer and converting that from base256 to base10 to get the index for my big LUT. I’m sure this is very naive and might have issues with not being perceptually uniform. This needs to be as fast as possible even if it means more memory. My thinking is that, if the human eye can only discern less than 256^3 colors, then it doesn’t matter a lot if we round up/down to a slightly different color- the result will be indiscernible despite being “off”.
To get an idea, it took matlab(octave) at least 24 hours to generate this table of smoothed curves, using all 4 cores on my desktop (intel i7 4790K). One question would be, could this method support higher color gamuts using a similar memory footprint? That is, can I generate a new table using Adobe RGB colorants (even change the white point) and still get satisfactory results? ~2GB seems to be a sweet spot where it’s not really that much memory in the scheme of things. Adding just one more bit would put that up to ~20GB. Suddenly eek.
My hope is “yes” and we’ll be able to just swap in a new table whenever we want to switch color spaces/colorants
At the beginning of “Real-Time Comparison Additive vs Subtractive Pigment” (https://www.youtube.com/watch?v=RUa8IHNtGb4), there is a paint sample (bright yellow) being modified using three “modes of blending”: “non-linear RGB, linear pigment, and linear RGB”. Also “Blending Modes” (https://www.youtube.com/watch?v=Qa6V6k6kMIw) shows linear pigment blending.
Can you, would you explain “linear pigment” mode of blending a bit more at length, mathematically speaking and also in normal, everyday language? Is it the same as, or different from using multiply blend mode in linear RGB? Well, it’s not the same as it seems to use spectral something or other, but I’d like to know a bit more about the “something or other”.
Applying cyan and yellow to make green using linear multiply also simultaneuosly darkens the blended colors as the colors applied. This “darkening while mixing the colors” can be worked around using various blend modes. But doing so results in a situation where color and tonality are applied separately, which sometimes is nice, and sometimes gets a bit frustrating. Maybe “linear pigment” allows to avoid the “getting darker while mixing”?
Also, is there a link to your modified mypaint code? Are all the modifications - floating point OCIO, linear pigment, the color choosing/modifying dialogs, etc - in one branch that can be cloned and compiled?
Scott Burns does a pretty good job explaining it, but the gist of it is, instead of multiplying RGB[3]*RGB[3] you multiple S[36]^n * S[36]^(1-n) where S is actually 36 colorant wavelengths and n is the mix ratio (such as .25 to be 1/4 to 3/4 ratio). Then all of that gets converted to XYZ and directly back to RGB.
I just say “linear pigment” because, as you know, the sRGB TRC has to be reversed in order to have accurate results. The “pigment part” I suppose is really just describing the weighted geometric mean multiply mode.
This is exactly what it does, in addition to those more natural “curved” mixing lines that allows blue and yellow to curve around and avoid the middle achromatic area. It really is wonderful and I can’t imagine painting without this now that I’ve tried it
I haven’t combined the spectral w/ the OCIO stuff yet. The OCIO Floating point is very rough at the moment.
If you want to try the Burns “spectral stuff” please read this:
Basically clone my Burns branch, compile it, install it. Then you can use MyPaint master branch and build that and it should work fine. That is, the brush engine is separated from the GUI, in case you were not aware Oh, and don’t forget to generate or download the rgb.txt data file.
Of course you’ll need a brush that actually uses these settings, so you can download my brush pack to get jumpstarted. These should all used “spectral subtractive smudge” out of the box. To be clear, you’ll only get “spectral blending” when smudging colors around on the canvas.
I’ve been changing the names of the settings around after I realized “spectral” has nothing to do with subtractive color. So, you can change the brush setting smudge mode from normal–> subtractive independently with another setting for RGB–>Spectral.
Some progress on OCIO/Float/Linear stuff. This is super cool, you can flip the Alpha (the alpha slider here is just a “toggle”) so that Opacity is controlling JUST the alpha channel. You can also erase the alpha (occlusion) from things already on canvas
and continuation showing “erasing occlusion” and different backgrounds:
Edit
I’ve changed the setting to be called “Occlusion”
Here is a much much better demo:
is getting 
. there’s quite a bit of work on this topic, these may be entry points:
