There is merit in the idea of plugins and there are several good examples where user-maintained plugins work seamlessly with the core software. Take Blender with add-ons, video NLE’s like PremierePro and DaVinci Resolve with e.g. Boris FX plugins and indeed the whole professional music industry with VST samplers, synthesizers, effects and libraries.
I can imagine @OK1’s idea quite well, where you would perhaps load a custom DLL, and then another module shows up at a designated spot in the pipeline of darktable. The biggest downside of such a model is of course compatibility.
2 Likes
I didn’t say it was a bad idea, rather that we basically have a plugin system already.
2 Likes
“plugin architecture” goes to the foundations of a particular software, rather hard to implement it retroactively.
That said, doing such to my hack raw processor may be the next thing I take on. It’s already organized to use such a mechanism (arbitrary workflow), and a plugin interface would let others do what they will, without by-your-leave from me. Every tool a plugin…
1 Like
I’m guessing the majority of OpenFX plugins will happily operate if you feed them only a single video frame.
Or: There’s already a fairly well established image processing plugin standard used by both FOSS and proprietary software.
That would handle most potential compatibility issues.
That’s a very personal wish.
I respect that, still, a very personal one.
Trouble to whom?
Again, a very personal opinion, clearly not shared by developers and those from the community closer to the development process.
With all due respect, I understand that newbies (in dt, in the community) may have these thoughts at start. What I don’t understand is why they insist and invest so much time on this discussion, even when all evidences show them it’s a void discussion. Why not spending that time learning more the tool, even sharing what they learn by producing tutorials, for example?
5 Likes
Just want remind everyone that this thread isn’t the place to discuss everything about darktable. Try to stay on topic with feedback on the sigmoid module, especially true for everyone trying to defend darktable. Just ignore or remind off topic comments about the topic of the thread rather than bringing in all the armor and weapons. I have no problem with lengthy discussions on the nitty picky details on the current sigmoid module but we have to stay on topic for this thread to be usable.
Examples that are off topic: plug in architecture, general darktable ease of use, any comparison to other softwares that isn’t focusing on their tone curve, Foss comments etc.
Rule of thumb: will this comment help Jakob improve his code such that the module gets good enough for a merge? Does this comment make Jakob smile such that he will spend more time on developing? Criticism about the module?
Yes? Please post here!
Uhh or no? You should probably post in a different thread or not at all.
Thank you 
19 Likes
Your ignoring the pipeline and the role in DT correctly processing images…adding plug-ins not written for DT is not as easy as just making them available and sigmoid is not a plug-in
An example with only sigmoid + tone-eq (which is what I now use instead of shadows and highlights module, to remain true to the scene referred workflow) + sharpening(input sharpening with raw denoise, output sharpening with sharpen and contrast eq) + local contrast. + exposure + white balance adjustments.
With really well exposed images, sigmoid delivers a good result, with its characteristic added “shimmer” in the highlights(of all the comparable modules in dt), works well for images like this taken in bright sunlight.
Ignore the funky green blocks on the left and top of the image in darkroom, which I think is something with how this dev version displays the edge of images. This only shows up in the editing window, not in the final exported image.
.jpg.
1 Like
If I may add, explicitly. For “poorly exposed” images, typically images taken in environments with very bright highlights, such as backlit images, where the shadows are dark in the raw image, and one would ordinarily have used the shadows and highlights module, a key challenge with scene referred was learning to apply the tone eq module to achieve a similar role, to lift exposure only in the shadows.
This approach has been key to solving some of my key challenges with sigmoid, and filmic.
I do hope there is a scene referred version of the shadows and highlight module, in the works. Cos while it is possible to achieve something suitable with the tone-eq module, having a dedicated module for this similar to the current shadows and highlight module, with fewer sliders, would be more effective.
I tend to view the tone-eq as a module for localised broad contrast shaping, but not a tool to be used for the broader kind of xposure changes, across the entire image, which the shadows and highlight module (scene referred version when it arrives), would be more apt for.
Relevance to this thread. I can imagine many may struggle at first with filmic and sigmoid, if they continue to use the current shadows and highlights module, as I once did, with less successful results. Th authors of sigmoid and filmic may wish to include this tip in the documentation, and in their demonstration videos/ comments on forums, as I have found this use of the tone-eq, to be a lifesaver, in the interim period, where we do not have a suitable scene referred compatible shadows and highlights module.
In an ideal world, we would have had all the scene referred versions of the modules we have been familiar with, delivered with filmic/sigmoid, to avoid us breaking the scene referred new rules.
Some would think that exposure alone would do the trick, but it does not, cos it applies to the whole image, when as stated above, what you really want is only the shadows lifted. Sure some clever workarounds like using a mask to lift xposure in only shadows is thinkable, but a scene-referred shadows and highlights module - highly anticipated if I may add, is key to the efficient use of filmic and sigmoid (hopefully sometime soon, this will not be a issue any longer)
We already talk about using Tone Eq instead of Shadows and Highlights here: https://www.darktable.org/usermanual/en/overview/workflow/edit-scene-referred/
I don’t think we’ll get a scene referred Shadows and Highlights module because, as you mention, Tone Eq can do what S&H did and a lot more.
You might also want to look at the rgb color balance module. You can modify shadows and highlights there both luma and color and you can now define the shadow, highlight and midtone areas of images and display the mask of each tonal range…
The “recent” modules, sigmoid and filmic, achieve something beyond simply transforming from scene to display referred. Once well understood, they encourage me to get out there and take photos, with the knowledge that I have a fairer opportunity of coming back to the computer, with much more control than was possible with the base curve.
I am uncomfortable with comparing sigmoid and filmic, but its difficult to describe one without referring to the other, like if everything were blue, it would be difficult to describe. White only makes sense if there is something else to compare it with.
Because these two modules have different controls, its also impossible to actually compare them, cos if you had all the time in the world, maybe you could come up with settings that would make an image almost indistinguishable from the other. So any statements of a trend, are based on how each module “pushes” me to use them, and make adjustments with them, and the result thereof. Your mileage may vary, from mine. These opinions are not based on the default settings, but on how I have tended to make changes from the defaults.
From recent images, my opinions, from trends that have developed in my use of them are :
-
sigmoid tends to deliver images which pushes aspects of the image more into the extremes - dark and bright, leaving less information in the mid-range. So you get more contrast broadly speaking.
-
sigmoid also tends to deliver images where there is less separation between colours, pushing shades of yellow and orange nearer to each other, and in greens you get less of a separation between the various greens. It is the same with other colours, but in the examples I have posted here, the predominant colours are the yellows and greens.
If there was one wish for sigmoid, it would be one more control (ideally three or four), so one could control the contrast globally, with another set of three controls to manage this contrast with more control, in the shadows, midtones and highlights. Maybe we are now stepping on the toes of other modules in dt. But I did check and to the best of my knowledge there is no other module which gives you control over contrast in shadows, midtones and highlights distinctly. I make the layman’s assumption, as I am only an end user in this case, not a physics professional, that the chroma observation, which I mentioned in 2 above, will also be adjusted by these controls, so you can have more or less separation of colours, globally or in shadows, midtones, and highlights when the luma contrast is adjusted.
Then there will be introduced another challenge, adding controls to adjust the crossover between the shadows and midtones, and between midtones and highlights.
Then one more wish, 3 controls more controls to adjust the luma (darker or brighter) in the shadows, midtones and highlights. Note, I have not asked for a global control of luma, as this is already included in the exposure module, and ideally should not be duplicated, elsewhere.
To the casual observer, unless you had the images side-by-side, you may not be able to tell any difference. Broadly speaking this gives the current sigmoid images a bolder, more vivid look, more binary, monochromatic, which lends itself well to certain kinds of images, where the colours look more like the result of reducing luminance via the “color balance rgb” module. Greens are darker, yellows are darker. Somewhat like the effect of having used the local contrast module.
Sigmoid–
Filmic–
I thought it good to express this via curves, cos some of this could be simulated via curves also.
If you had two fixed points for the transition between shadows and mids, and between mid tones and highlights, its possible to then use three other points on the curve to modify the dynamic range of each of the three regions. I’ve used curves only to hopefully make it easier for you to understand what I had written earlier. I do not have the mathematics skills to fully describe the additional controls which you may wish to consider. Curves may make it easier to convey what I mean.
Curve example. compressing dynamic range in each region.
Curve example, lifting the shadows only. all other regions have no change.
Curve example lifting only the mid-tones.
Of course in an ideal implementation these mid points will not remain on the linear line, but will have some “latitude” to move up or down, with changes made in each region, leading to two other possible controls. So that’s 8 controls in total described below.
This introduces the potential for what I call a super highlights region a 4th. Typically the brightest whites or light/bright colours
- to set the transition point between shadows and mid tones.
- To set the transition point between mid-tones and highlights.
- To set the transition point between highlights and super highlights.
- Adjust gradient up or down, in shadows (i.e lift shadows, or crush blacks more)
- Adjust gradient up or down in midtones ( less contrast - brighter, or more contrast - )
- Adjust gradient up or down in highlights.
- Adjust gradient up or down in superhighlights up or down.
- Adjust gradient globally, to make it more or less contrasty.
- Adjust latitude of transition points, i.e how far up or down they are allowed to deviate from the set position, when local adjustments of gradient in the 4 regions are modified.
I can imaging this sound really like a lot. But I have come to accept - this is the mindset of darktable and the darktable developers - more control not less. So rather than me try to simplify things, why not fully explore it to create something that is quite flexible, even though novices may find it difficult to comprehend, at first.
I thought it good to re-express some of my earlier comments, via curves, cos some of this could be manually achieved via curves also.
Assume there was no change in exposure, If you had two fixed points on your sigmoid curve, representing the transition points between shadows and mids, and between mid tones and highlights, its possible to then use three other points on the curve to modify the dynamic range of each of the three regions. I’ve used curves only to hopefully make it easier for you to understand what I had written earlier. I do not have the mathematics skills to fully describe the additional controls which you may wish to consider. Curves may make it easier to convey what I mean.
Curve example. compressing dynamic range in each region.
Curve example, lifting the shadows only. all other regions have no change.
Curve example lifting only the mid-tones.
Of course in an ideal implementation these mid points will not remain on the linear line, but will have some “latitude” to move up or down, with changes made in each region, leading to two other possible controls.
This introduces the potential for what I call a super highlights region a 4th. Typically the brightest whites or light/bright colours.
Which leads to a smorgasbord wish list of additional controls.
- to set the transition point between shadows and mid tones.
- To set the transition point between mid-tones and highlights.
- To set the transition point between highlights and super highlights.
- Adjust gradient up or down, in shadows (i.e lift shadows, or crush blacks more)
- Adjust gradient up or down in midtones ( less contrast - brighter, or more contrast - )
- Adjust gradient up or down in highlights.
- Adjust gradient up or down in superhighlights up or down.
- Adjust gradient globally, to make it more or less contrasty.
- Adjust longitude of transition points, i.e how far up or down they are allowed to deviate from the set position, when local adjustments of gradient in the 4 regions are modified.
- Now I am becoming Oliver twist - lets add a crossover variable for the shadows to mid tone transition, to define how smooth or abrupt this transition is. (really this is just a parameter that describes the smoothness of the curve transition, so for example changes in the shadows may affect the mid-tones to a more or less degree depending on the value of this parameter.
- Similar to 10, but for transition between mid-tones and highlights
- Similar to 10, but for transition between highlights and superhighlights.
On second thoughts 9 may be a way of implementing 10, 11 and 12 i.e similar idea.
I can imagine this sounds like a lot. But I have come to accept - this is the mindset of darktable and the darktable developers - more control not less. So rather than me try to simplify things, why not fully explore it to create something that is quite flexible, even though novices may find it difficult to comprehend, at first.
Whatever the failings of my approach, I am not a photo or optics expert, whenever I struggle with achieving the desired contrast in a specific aspect of the image, I just use curves. Am not asking that you implement a curve module, but you may wish to add a similar display, so it simulates some of what is taking place under the hood. kinda like how filmic creates a display(but I must be frank, I never bother to look at that display, cos its quite involved), but simpler.
Aren’t you reinventing the tone equaliser?
2 Likes
Yes he is, or suggesting something similar to the parametric curve available in Lightroom and similarly in RawTherapee
1 Like
Not exactly, most tone tools allow you to move parts of the histogram up and down, but the idea here is rather to spread or contract parts of the tonal range, to add contrast in a tonal range. You can do that with curves but it’s very finicky because you don’t have individual controls for the slope and the offset of the points.
I was thinking of trying to do a prototype, it would look a bit like the current parametric curve but with another parametrisation and double the amount of sliders (one for offset, one for slope).
I would certainly hope not.
I would have loved to use the tone eq for what I have described, but I find it an excellent tool for more granular changes, and at the same time not granular enough, for certain levels of change. See below
Please as I keep noting, I am not an optics or physics expert, just an end user, which hopefully also counts, as a member of the ecosystem.
Sigmoid, filmic, base curve or any similar tools, all attempt to define in various ways, the curvature of the transform.
If we look at this issue as the size of a brush, one would have different sizes of brushes for painting.
-
Global - changes like in exposure module, where I would expect a simple multiplier is used mathematically when you change exposure, so all values in the entire image are adjusted. Broad strokes. or rather very broad strokes. Many of the global gradient parameters are already defined in sigmoid, addressing the two ends of the spectrum black and white, and some control of what happens in the middle, which is great. Then I saw the new color balance module, and had an aha moment, cos this is the kind of thing I had been thinking of, somewhat, but in my case, I’ve typically used curves, to adjust brightness, and gradient, in each region, manually, in the absence of other suitable modules.
-
Regional - Next level would be the 4 regions, I described (or 3 for those who prefer), the superhighlight region is my own creative largesse. Shadows, Mid tones, Highlights, Superhighlights. The color balance rgb module, hints at using these regions to adjust the brightness (i.e the luminosity) in each of the regions, but its primary purpose is colour and it does not yet have anything for controlling the gradient within each region. My proposal for sigmoid, fundamentally speaks only to gradient, i.e contrast, either globally or regionally.
-
Granularly - Like Tone EQ. It is a great module, but for broader changes, the crossover points between the regions are a bit “sticky”, and its not easy to pick the transition points, you want, exactly. The best you can do it pick the nearest region. So its close enough but not close enough. Using it to achieve what I had described needs compromises. Its also somewhat impossible like with curves, to avoid longitudinal movement of the transition points, i.e if I boost brightness in mid-tones, but I do not want any change in shadows, its hard if not impossible to achieve in Tone EQ. Tone EQ is a wonderful tool, but a bit too granular, for what I had described. As an aside, not wishing to derail this thread, but a good place to mention it here, I would wish for someday, a Tone EQ that has options for 16 levels, like the Ansel Adams thing. And of course to still have the option to only use 8 regions as we have today. I do not thing Sigmoid should be as granular as Tone-EQ, in its adjustment of gradient, cos that would be a bit too much, and be like a plane that is difficult to fly - way too delicate controls, and too many of them.
-
Super granular - the most flexible tool we have now is the rgb curve where you can do whatever you like with gradients, but the flexibility comes at a price, not a very efficient tool - I have memories of spending days hand-creating my own curves with base curve, when that was all we had, back in the day. Filmic and Sigmoid, do a much more efficient job than anything one could create manually in rgb curve.
So if sigmoid could provide the regional gradient control, similar to what I have described, in addition to the global gradient control which it already provides, that would be christmas come early.
I get the feeling that we’re drifting from the aims listed here:
2 Likes
Thank you.
May I respectfully and cautiously extend the proposal to add.
Note, I do not know the intrinsics of scene referred and display referred, and how these are implemented, so I may be speaking utter nonsense, because I am ignorant of the implementation.
But thinking with an open mind, assuming it is possible.
Could any modules such as Sigmoid, which may implement such regional contrast adjustments controls, also have the possibility of turning off their global transformation
Why?
Today we have different modules, for adjusting aspects of luminosity, globally and all the way down to very granular detail, manually with curves. So you pick and choose what suits your intention best.
With the scene to display transform, should it not also be possible to have multiple modules perform aspects of this transform, so choose each for what you prefer.
e.g.
- Module 1 - Sigmoid - for Regional contrast.
- Module 2 - Filmic - for Global transform
So you can pick and choose a transform path that has the features of each which you like best for the image in question.
Options to achieve this.
-
If the global and regional changes are implemented in the same module, this could have the option to turn off or on, either of these changes.
-
A module like the current Sigmoid may have a companion module which addresses only regional aspects, and you use the companion module. Reminds me of rockets with primary and secondary stages. No need to have everything in one module.
So if I wanted to test regional gradient adjustments, independently of global transforms, I would have two sigmoid instances, and turn on only global transforms in one, and only regional transforms in the other.
There are lots of details I have left out, otherwise this becomes exhaustively long, such as in what order these would be sequenced, in the workflow, in dt. But these are implementation details. Best to partition requirements from design and implementation. I’ll stick to requirements, and let others worry about the best way to design and implement.
In simple terms, it would be great to be able to also turn on and off regional gradient adjustments, per region, or turn off all regional gradient adjustments., as an independent set of changes from the global scene to display transform.
Using rgb curves to achieve this not efficient - or needs multiple instances, and you cannot synchonize the cross over points between regions, across multiple instances of curves - not elegant. Best achieved in sigmoid or filmic.
1 Like