I think what would help darktable is to be more opinionated and introduce more modules to imply a specific workflow. Having too many tools that all do something similar is good when you first encounter the app and try to find out for yourself what works for you, but beyond that initial period it all becomes just noise. Also having the do-it-all modules like color calibration, color balance rgb introduces even more noise, even if it makes sense mathematically that, for example, a color mixer is built into the same module as where the white balance is set. Navigating through the noise is what makes darktable difficult to use. There are tools to reduce the noise, but you can’t rid of the do-it-all modules where for 99% of your edits you only ever need 2 sliders out of the 15 it offers across 3 tabs. Additionally these modules introduce “preset noise” where you need to manage and navigate through multiple instances of these do-it-all modules because you needed to apply two atomic presets (for example, one color balance rgb preset to introduce a warmer white balance, and another color balance rgb preset to recalibrate the saturation and make highlights more saturated).
None of this is inherently bad. None of this is inherently unlearnable and “too difficult”. It’s just noise which results in cognitive load that is difficult to explain. When people try to explain it they’re inclined to provide examples, then come other people for whom this noise is more tolerable or this part of the app causes no noise, and say that no, this is fine and we should keep it this way.
This would be my frustration with darktable, and there’s no real solution to it without someone taking the entire user workflow into a tight grip and completely reworking it to be much more opinionated (and by extension, worse for some people, better for others)
Having followed (and participated in) this kind of discussions over time, I realize as a plain user:
It takes some time for us who have never been involved in a FOSS project before, to understand what the dynamics are that drives changes. How the bottlenecks seldom are clever ideas for improvement, as us newcomers are good at providing … , but resources in the form of people who have the necessary qualifications, spare time, interest and willingness to perform work. And this applies in force to any suggestion to changes outside the scope of the design of one module, like the broad proposal here to invert the whole pixelpipe system to be mask based, (which obviously have great potential to wreak havoc with a long-standing and core policy of not doing anything that break backwards compatibility). It won’t happen in darktable.
Yes, there are other projects that can go such a way, but then it is exactly this, “other projects” like vkdt that has been created as an alternative to darktable’s framework.
It may also be a challenge to understand how best to move proposals forward to entice any development work that in the end will be accepted for inclusion. Focusing on specific issues, and keeping an open mind that others may think differently because they (which may be a majority compared to our own thoughts of one), have other needs and experiences – to identify obstacles and (well-founded) inertia. Focusing on building consensus that there really exist a need and relative benefit of doing anything different than the current seems to be central.
If some people have spent lots of time and resources to create the tool that solve their needs and wishes, like darktable to be a well-functioning powerful and speedy tool for people in the know to work on the type of images they photograph, they are not necessarily very keen on doing a lot more work to make it work for somebody else’s (like beginners’) needs and wishes – in particular if those wishes are because other users have been primed to think/act in particular ways from previous use of other tools.
(That said, it’s anyhow impressing how darktable has evolved over the last years, including becoming a lot more friendly for beginners. And the more reason to hail @Masterpiga now doing a very promising work on major improvement of the UI and UX of darktable’s masking system, Aurélien Pierre who was the driving force behind the move to scene-referred editing, (irrespective of what happened afterwards), and @Pascal_Obry and all the other heroes who, unknown to us plain users, work behind the curtain to maintain and develop the system, like the current long-time project of migrating to a new version of the underlying Gtk4 UI framework).
As for the tone equalizer in particular, I believe there is some confusion as to what tool it really is created as.
It was launched as a tool for “dodge and burn while preserving local contrast”, as both the dt and ansel manual still describes it. I remember that when somebody in the forum showed how one could set up the curve in TE to work as a global contrast tone curve (shadows & highlights) controller, Aurélien (TE’s creator), remarked that it is possible to use it in a global role, preferably for images with a narrower dynamic range, (ref TE’s 8 EV bandwith), but that TE really was created for local work like:
That new users may think of TE to fulfill a (literally) broader role, is very understandable, though, as it later has come to be equipped with some default presets for a more general tone curve/S&H control, and as dt’s manual also describes TE as “replaces the need for other tone-mapping modules”. But I think that Aurélien and others have been clear that the point is not really seeking to create a mask that covers the whole DR of an image, but, to reduce the risk for artifacts, rather use several instances of TE with one mask covering the highlight end and another for the shadows.
TE has been discussed a lot in this forum. Some major threads like
and
may shed some light on this.
(After this it ended up in the two mask sliders being moved from the Masking tab to Advanced.)
Thank you for this background, but can you tell me than, which module in darktable should be used for dealing with shadows & highlights in a scene referred pipeline ?
As proof that there are at least as many opinions on darktable as its users, I utterly disagree with this (although I second everything else you say in the same comment): the fact that there are so many overlapping modules is one of the main causes of frustration and bewilderment for new users.
When I start using a new application, I don’t want to be exposed to all its complexity and I don’t want to have to choose between 3 or 4 modules, each with its own technicalities and learning curve, to achieve my result. It doesn’t help that some of these modules (purposefully?) only have an “expert” mode (I’m looking at you, Diffuse or sharpen) instead of offering an intuitive, basic interface and, in case you need, a more powerful, advanced set of controls.
It doesn’t help that old modules are deprecated but never removed in order not to break the workflow of long-time users. Yes, it’s important to minimise the disruption to the users’ established workflow, but every software with a long history naturally evolves and changes. And humans are a flexible species, they can learn new tricks even when they are old. The important thing is that changes do not remove features or flexibility while improving usability or adding new features.
Yes, I know, “darktable requires users to be patient and willing to learn some theory and it will repay with much more power than other tools”, it’s a same mantra which I have repeated a few times here or on reddit.
It’s partially true. Or is it? Are we sure that it has to be this way? Sometimes it seems as if a sort of elitism, or gatekeeping, is actually at the bottom of this resistance.
It annoys me too. On the other hand, I often manually make manual adjustments with those sliders. It annoys me that I have to click just to get TE started: why isn’t enabling the module enough?
But none of these annoyances annoy me very much. None of them are likely to be deal breakers in whether or not someone uses the software.
Bottom line on TE, for me is… Why so much fuss over a couple of clicks? Its trivial, unimportant.
Why not put the energy into something like Diffuse and Sharpen, which is, seriously, a very, very difficult module? And, even as I type that, I remind myself that the useful energy if expended by those who do, rather than those (including me) who complain
The sliders have been put on the same tab as the histogram. This is a huge improvement. Be happy.
No! You may not yet have found it useful: I find it invaluable!
I don’t even think AP’s explanation is correct. In my opinion, the RGB multipliers are there to compensate for the excess green caused by the Bayer matrix, as well as for the varying sensitivity of the filters.
In an ideal world—for example, with a hyperspectral camera—we wouldn’t need the RGB multipliers. “CAT” and “Relightning” (as described in Anders Torger’s article https://torger.se/anders/dcamprof.html , on the other hand, are necessary.
The distinction between CAT and Relightning, in particular, makes many things much clearer.
I found Anders Torger’s article far more helpful than AP’s.
Much of AP’s article strikes me as sloppy. For instance, color temperature is not a color (as AP writes) but rather refers to a spectrum. However, it refers to a specific type of spectrum that can be described mathematically by a single parameter. Sunlight follows this pattern perfectly good; LEDs, on the other hand, do not.
Furthermore, an object has no color. Strictly speaking, there is no such thing as a “white” sheet of paper. Color is created by lighting. It is true that the eye and brain perform color adaptation (CAT), so that, for example, a sheet of paper that appears white under D65 light will also appear white after a short time when illuminated with D50 light. However, this is because the brain recognizes the paper as paper and has certain expectations. If we look at only a tiny section (with a magnifying glass) of the paper, the eye and brain do not recognize it as paper, and we would certainly see the difference if we illuminated it with D65 or D50 light.
Throw rocks at me, but I have, after reading through so many threads about darktable being difficult, unapproachable, unattractive to newcomers, etc. come to a simple conclusion:
Never mind these “new users,” whom some people claim to represent, who want change it. Especially never mind those who take one look and go away. The latter are not the target audience: the former are only new users for a few days or weeks, after which they can produce decent results.
To put in development time for those people is a complete waste of the limited and valuable resources available. Our maintainers/developers have better things to do. The priority is not to get people in the door. And the people who do come in the door are, or should be, willing to experience some differences, some things that may not be “intuitive” to them. And that some things will take years to understand!
It does change, it will change. User interface issues might be given more importance now than they were in the past. If some things do, thus, become easier to newcomers, that’s nice.
It should also be remembered that for every new user whose interface wish is granted, there are a hundred existing users who have to accept the change.
The problem with modifying only specific ranges such as the highlights region or the shadows, is that it’s easy to introduce artefacts, because the global tone curve becomes less smooth, if not disjointed. This happens to me with tone equalizer too, even though it tries very hard to keep the curve regular.
Highlights and shadows are often contiguous to mid-tones or even to each other.
In addition to several instances of TE, there is the swiss army knife of darktable, Color Balance RGB (which was more or less developed by Aurélian to work together with TE and Filmic RGB). It has three internal masks, i.a. one for shadows and one for highlights, that can be adjusted in the Mask tab. They can be used i.a. in the two corresponding local sliders under Perceptual brilliance grading to adjust luminance. (Previously there was a limitation in the mathematics as to the relative size of amendments one ought to make with the Brilliance sliders, but that was eliminated a couple of versions ago.)
And that’s why dt default offers just one visible and simple tone mapper, (sigmoid), hides the deprecated modules and offers the most relevant (and not so very overlapping) modules sorted within a limited number of tabs in the module groups header.
In addition there is the Quick Access Panel, where all controllers of major importance has been extracted/copied and gathered in one place. (Cropping not included for technical reasons.)
What are you really complaining about – that there is a learning curve at all?
Yes, darktable is a monster, and one cannot expect to master it without any instruction and training. But for my own part as a total beginner in image processing with darktable, I must say that I did not perceive there to be much “noise” if I stayed with the instructions given in the manual and the built-in help system. (That the manual, being a reference manual, is not the best tool for introducing new beginners to dt, is a quite different thing.)
What “resistance” are you actually referring to? That some kind people who have already spent more than enough of their time and their loved one’s patience on a project that you are offered the result of for free, don’t jump to activity for all the proposals that are launched by us free riders in this or other fora before we really have bothered to peruse existing info material to find out what is really there or understand how much effort is really needed to achieve an improvement?
Nothing is hindering us to create/design some well thought-out (within in the community, not only in our own head) improvements, except our own inabilities and limitations.
Are you using the masking options… different numbers of filter passes, quantitative adjustment and the aligo used for luminance estimation for the mask have a massive impact on the results…At one time I rarely used them but I do now.
It’s worth experimenting…I often find for me I like using the rgbsum estimator…
Nope. I was specifically replying about the overabundance of modules which I see as an unnecessary (note the emphasis) source of confusion (for beginners) and not as a boon.
If I were a developer (and by the way, I also do software development) I’d say that having to maintain an ever growing and evolving code while being unable to decommission old parts is a burden and eventually a nightmare. Often it’s necessary for external constraints but I have never met anyone who is happy to do it.
Incidentally, when I started using darktable I started using AgX (not sigmoid) because that module was hailed as a state-of-the-art, most powerful and flexible tone mapper. Ideally there should be one tone mapper, capable of achieving what the predecessors could do and more, and better.
As per the manual, it’s not suitable for beginners. If I managed to go past the initial difficulties, it is thanks to third-party tutorials and introductory manuals and videos on YouTube.
Assuming a good faith misunderstanding of what I have said, I’ll explain myself better: first of all, most of the opposition is not coming from long-time developers but from users. Secondly, there’s a distinction between expressing a feedback (and suggestions or just sharing pain points) on one hand and demanding certain features from Darktable’s developers.
But there’s a pattern, which I’ve observed time and time again, where the same arguments are used to dismiss the feedback, and this pattern includes one or all of the following:
a sort of victimisation of the new user: “you have to take time to learn how to use darktable”, even when they have actually learnt how to do it and are just saying: “you know, this is unnecessary complicated”
“darktable repays the time and the steep initial learning curve with greater power”. This is crucial. Generally unproved. It could be true but I’d like to see examples.
be thankful and not criticise the work of people who dedicate their precious spare time for something that you use free
I have used all of them. And now I think that most of the time these replies are ungenerous and a kind of knee-jerk reaction.
Occasionally a deeply technical motivation is used, which still doesn’t explain why that technical complexity is not exposed to the user by other applications (commercial or not) which nonetheless have a large user base made of amateur and professional photographers, who certainly have advanced needs and are served by solutions which are less daunting (point 2 above).
masking… yes, and also trying to make the separation between masked and non-masked parts smoother and more “feathery”.
The other features, I have to admit that I have to explore them. But in general I accept and embrace the fact that some photos with a large range have harsh transitions and that I have to choose between preserving details in the highlights or in the shadows.
There is not a lot of maintenance that needs to be done on modules, this has been discussed before. They are mostly isolated algorithms and at most you need to maintain the GUI which uses shared components anyway.
I used to believe the “too many modules” argument too, but I no longer do. Davinci Resolve, Blender, etc, all have multiple ways to achieve the same results. Davinci in particular has a ton of different effects you can add in the node editor. darktable doesn’t really have that many overlapping algorithms and most of the stuff that’s added nowadays is either an improvement, replacement (for old display referred), or genuinely new.
Ya I have rarely used the TE now for some time with a mask histogram centered and spanning the full scale… I will target usually one end or the other of the DNR and one other favourite is the relight preset…often rather than pushing exposure too far in the exposure module I will use it and then I will slide the exposure comp in the TE left or right to tweak that initial midtone bias curve. For shadows I will also try the older guided filter as it can sometimes perform better… Lots of tweaks if you have the time and inclination to play around…
I never wanted it to be a thread in this form, if it was up to me I’d probably have closed it at around 20 replies I think that maybe a few handful of messages after that are truly adding anything.
