At the top of this thread…
First I’d like to be sure I’m using the right version of sigmoid, cos I followed the process linked to below, adapted for Windows, to build what shows up as this darktable version
…
…
With a sigmoid module that looks like this :
…
I ask cos I do not see a “log-logistic” or “weilbull” option which was discussed or referred to here in the opening post of this thread. Have these options been deleted from the user interface? :
Will do in due course. If your request is still open… I have to ensure I have all the collateral, to support any opinions, so please give me a few more days. Discovered this thread only a short while ago.
Nice progress on building darktable!
That is indeed the correct version, the module has changed over the spring and will change some more soon. And another reminder, this module WILL brake a couple of more times as I still change the UI parameters from time to time.
The module now only supports log-logistic with skew as that generalizes better with less clutter. Looking forward to hear your feedback!
1 Like
This picture might be interesting. People struggled to get a nice rendition of the highlights in darktable. Sorry I haven’t yet tried installing a sigmoid build…
I hope the maths supports some way to translate this into an option in sigmoid (drop down or check box) to toggle this. So anyone who wants it can have it. i.e like being able to turn emulate lens coating on/off digitally?
CarCav, link please, happy to try it out, if you kindly share link to the code, so I can build this and experiment with it.
Image raw here
All attachments are CC BY SA, and xmp’s are provided below each image.
DSC01728.ARW (15.9 MB)
This 1st image uses filmic
DSC01728.ARW.xmp (37.3 KB)
This 2nd image uses sigmoid
DSC01728_01.ARW.xmp (38.6 KB)
Th 3rd image uses sigmod + a bit of shadow lifting via the similarly named module.
DSC01728_02.ARW.xmp (6.6 KB)
This 4th image has no equivalent in dt, its the in-camera jpg as generated by Sony’s Imaging Edge app, which I find is pretty faithful to the embedded jpg (which you can xtract from the raw file - which dt displays when you first open up the raw file, prior to any edits - depending on your config - this is typically the default image you see). At the time I took this pic, I was not generating a discrete jpg, with the in-camera raw.
Why include this in the consideration. In my exhaustive tome below, I discuss the expectations of the non professional photographer, who are the majority by numbers. At the time, I was “developing” the image in dt, with filmic and sigmoid, as above, I had no reference to the in-camera jpg, I simply wanted something that looked alright, to my eyes. We all are conditioned by what we have seen, even if we do not use these as a reference. My including the “in-camera” jpg here was an afterthought, to bring proper context, into the real world. Anyone who wants an in-camera jpg, can get one via the camera or via the manufacturers image editing app, which usually stays faithful or slightly better than the image generated in camera. Why is this relevant in the real world?
The default settings for filmic and sigmoid, which I have not bothered to include here as xmp’s or images thereof, cos we can all see this in dt, and I do not have to add these here, are pretty dark, when applied.
Without deliberately attempting to so do, my edits with filmic and sigmoid, look a bit closer to the in-camera jpg, so these kinds of edits should be included as presets in filmic and sigmoid, to reduce the shocks that some may experience with the defaults, or the defaults in filmic and sigmoid be modified to something similar, that reduces the shock of any deviation that some may feel when they see the current defaults in sigmoid and filmic.
In all cases, except the "in-camera jpg and the default from sigmoid below, I tweaked the default sigmoid settings.
The default sigmoid generated image below, uses the same exposure settings as all the images produced above by dt. (included only so that those who may not have sigmoid can appreciate the difference). Its a great image, if only a bit darker, than I might have anticipated.
In general, I find not just from my own pictures, but from recollection of how filmic and sigmoid have been used in images/development published by others, one has to state the obvious.
-
While there is creative freedom to “bend light” (my own term for digital raw development), these two scene to display(s-t-d) transform modules in darktable, unless you really make an effort otherwise, deliver an image that is closely correlated with the raw file. When you take a photo that already looks almost real, from looking at the raw file in darktable, before any s-t-d is applied, they provide pretty realistic results, subject to possible tweaking of the default module settings.
-
I think end users should view these modules as the dark room, not the dodge and burn room, so the intention is not to deviate from “reality” even though some of that is possible, but possibly have a workflow that leaves the extreme transforms and editing to other modules. So with a few tweaks, you nudge the image in the intended direction, and finish the creative alteration elsewhere.
-
With that mindset, I think both filmic and sigmoid provide a more flexible s-t-d than the base curve, allowing more latitude in image development, without going crazy, unless you deliberately do so.
-
In West Africa, there is a saying, in a culture where almost all cooking at home is done by women, that until you have tasted the cooking of another elderly lady, you think your own mother’s cooking is the best. We are spoiled by choice, and that in itself is a problem, too much bickering over small things. Refugees are only too happy to have something to eat and do not care about niceties like organically grown or avoiding genetically modified produce. If sigmoid had been available before filmic, it would have been a pretty hard sell to justify the need for filmic with more complexity in the controls.
-
Sigmoid produces an image that is usable, and has the advantage that there are fewer controls. To the uninitiated, having the middle grey luminance in filmic, modify the white relative exposure and black relative exposure, can at first be quite confusing, cos so many are expecting this orthogonal - each slider does its own thing behaviour. What’s key is each end user learns each tool, and understands how best to use them. There is a need with these new tools (it’s no more than 24 months since filmic itself has been available IIRC) for there to be a lot of user education, and all the education with filmic has helped. I would hope that there will be a similar focus on creating the right documentation and handholding to assist users with becoming effective with each new tool. Probably far more than any other development in darktable, filmic has had exceptional disclosure and support by the user to educate. And if anything, please think about this aspect, cos no matter how good sigmoid is, it may never be intuitive, to end users who are not knowledgeable about the underpinnings.
-
We must take learnings from tools like Adobe Lightroom, to study what has made it a success, and there is nothing wrong with copying a successful strategy. The one thing I would beg all darktable developers is to invent user friendly names for all these engineering variables. End users think in terms of light and dark, and bright and colour - more this, less that. If I was using a tool like AutoCAD, sure throw all the technobabble at me. But I want to edit a photo, should not need me to figure out what the heck is Target Black and Target White - means nothing from the view of an end user. Anyone who knows what these things mean, would have written their own software to do their own processing. So how can we simplify all these names that are engineering oriented. Skew has no meaning in image terms, what is Crosstalk - means nothing. The same apples to filmic, you first have to get through the jargon of the slider names, none of which mean anything, maybe except things like “Contrast” which everyone who has ever used any photo or imaging tool understands.
Far ahead of the maths behind the scenes, must be the user interface, cos that’s what will make the most difference. That’s what will get more people to use the tools.
My opinions will be unlike any other comment hitherto on this thread, cos from what I read, the others have primary concerns about the math, and the correctness. No harm in discussing the engineering behind the scenes. My wife does not need to know that the pedals of her Mercedes (when I ever have the money to buy her one) are drive by wire, and she could not care less, if it was diesel petrol, or electric or hybrid or hydrogen powered, and in her case, once upon a time when she test drove a merc, which we ended up not buying, she would complain therefter about the plethora of controls.
A painter is not too bothered about how their paints are made, i.e exposing the terminology or variables behind the paint. It’s good enough to know its watercolor or oil paint, anything more is superfluous.
So kindly consider a name change for the controls, into simple human understandable terms that will mean something in the real world. One tool uses the word Latitude for a slider, and everytime, I have to wait for the tool tip to be displayed, to recollect, what exactly does this do again?!!
-
My attached image does not have significant contrast, nevertheless, the one thing I found a bit difficult was the lack of a control like the middle grey luminance in filmic, to brighten things up a bit. My attempt to use shadows and highlights was not especially successful. One may have to explore using an HSL like module to lighten the dark tones. In comparison, using the middle grey luminance, filmic has become more usable since I discovered it and created a default preset which unhides this control. If I had no access to filmic, I would not know that I needed this. Choice in this case is our enemy. I could use color zones to lighten the greens or change the colour to a lighter green.
-
Which s-t-d (sounds similar to an acronym for something terrible in medicine), is more realistic to the scene as shot. I could not tell you, s-t-d’s are like life, you get used to them and their look becomes normal the more you use them. You adjust.
-
No transform algorithm is going to be perfect and please everyone. But what is important is far more than the tech, the user experience. Permit me to share one from my experience. I was once a software developer, further to education in Computer Science. A long forgotten company - Borland had superior compilers to those of Microsoft, but over time, cos Microsoft had other advantages, superior education and phenomenal documentation which is one of Microsoft’s strengths for the developer and their cutting edge at the time certifications, contributed to their dominance, and today, I can’t recollect that anyone else creates mainstream compilers for the Windows ecosystem.
I would encourage that those who like sigmoid, and want it to succeed, of which I am one, focus on
a. User Interface/User Experience
b. Documentation (including Tool Tips)
c. User guides - how to succeed with sigmoid, Videos…etc…
d. Presets. I’m happy to share mine with you and you over time as I test sigmoid on more images, to arrive at set of defaults that fit most images well enough as a starting point. On that note, the default presets of most s-t-d’s like filmic, base curve, and sigmoid, only work well with “professionally” shot images, where the utmost attention has been paid to exposure, lighting, proper white balance at capture - to prevent single channel saturation. In the real world, these kind of images are in the minority(by number of human beings involved in the picture taking). Most people who take photos with digital cameras, do not have the best lighting, best subjects, composition, etc. So features and defaults that will assist such mere mortals to still feel good about their images, is something to take into consideration. Sure the purists do not need any extra help, but most photographers do.
In the good old days, the photo shop used a lot of their own intuition to add this value, rather than hand you a bunch of unusable printed pics, cos otherwise they’d lose repeat customers.
I am not asking for sigmoid to do any auto adjustment, but rather to include a few default presets that help images that need a bit of help - overexposed, seriously underexposed, or low contrast images. But who knows, sometime in a later version of sigmoid, there is nothing in the world that stops you from thinking about this opportunity to analyze the image and tune the default settings. A bonus, for those who may need it.
So the starting point in using sigmoid, would be - start with a default - which can be improved upon over time based on feedback. Defaults can change as versions change, and as long as prior defaults are included as new presets, all is well, for those who want the legacy. If not happy, try a few presets, find the one you like the most, then tweak from there. For the professionals, they can skip the “try a few presets” step, their egos may not permit such condescending habits, all their presets have to be custom made by them.
- Strategy.
There is every possibility that sigmoid may become included in darktable’s official distro, which will need inclusion in their documentation, etc. Positive. But this also limits any further improvements and ideas and creativity, cos it has to pass through the eye of a needle for any change to be included in the next version.
I propose you consider the following future for sigmoid.
- A branch of sigmoid that is aimed at the least resistance from the darktable authorities, and will have the most opportunity to get included sometime in the future. Its features will be conservative to please everyone in the committee of decision makers.
There is also the possibility that sigmoid, may take a long time, or never ever be included in the mainstream darktable distro. You must prepare yourself for that eventuality, to avoid your enthusiasm from being deflated.
- Even if you have sigmoid accepted in official darktable, you will most likely need to keep alive the the non official branch, to give you an opportunity to xplore new directions, as precursors to any further revision of the official version. This way you get the best of both worlds, albeit at the cost of maintaining two branches. If sigmoid never makes it into darktable, based on what I recently achieved with the help of others, I’d like to propose that the unofficial branch of sigmoid, become one of those “treats” to be included in an unofficial distro of dt, along with any other goodies that have not yet made it into official dt.
The keyword missing, most likely for very good reasons, which I may not understand, is “plugin”. and maybe one day, that will be resolved. If dt had a plugin architecture, where you could share your enterprise independently, we’d not be having this discussion. Anyone who wants it, could have downloaded it, installed it, job done. Their business if they like it or not, and there would be none of the acrimony or contention over which s-t-d is ideal for darktable.
So until dt has a plugin architecture, we have this other approach, making unofficial components available, and built if need be, for those on Windows and Mac (Linux geeks do not need such help). I’d take a guess that the huge majority of the target market for dt and image processing tools, are users on Windows and Mac. A coalition of developers like you can work together, with those like me, to make such dt+ distributions with goodies, that did not make it into the official distro, available periodically (every 6 months to start of with) to anyone who wants it. And we can define a more encouraging set of protocols for deciding what makes it into this dt+ distribution, and which official version of dt is the dt+ build based on.
Your enterprise deserves support, and I hope you find a way out of the current impasse, via the suggestions I have proposed. I’d like to always have sigmoid in my toolset.
Furthermore sigmoid deserves an ecosystem of “supporters” for that day when you may become too occupied with other life demands and may wish to hand off further development of sigmoid. Let’s think big picture, and think and plan long term, and not get bogged down with the challenges of today. It’s an initiative that deserves propagation and wide use, at the very least the opportunity for end users to choose, with caveats obviously such as the need to run in a separate instance, to avoid corrupting, their primary dt config/databases.
Congratulations on what you have achieved.
2 Likes
You are correct for many painters. Especially school students, hobbyiests, the inexperienced, the ignorant or the lazy (I say those terms not as criticisms, that’s fine if people are that way). But this is a gross assumption. Many artists care what their paints are made from. Quality and type of pigment has a huge impact on presentation, and longevity. The same analogy mostly holds for image editing. If you want to be a pro, you have to know your tools as best as possible, and learn as much as possible. You can use a housing analogy as well. Do you want your house built by a pro, or some guy who knows how to use a hammer but doesn’t know when to use long or short nails? Darktable tries to cater to the pro, those who are willing to learn, research their pigments and know the different nail sizes. A lot of commercial software, by contrasts, targets mass consumption, mostly of people who dont care about the pigments and want results fast.
The question of slider names has been raised before. I agree some things are unknown at first, but that’s why we learn. Darktable cares about the difference between chroma and saturation, for instance, whereas many other software will falsely call chroma saturation because it’s a familiar term.darktable wants things accurately named in a scientific way because it actually gives greater clarity when you understand the meaning of those things. I’m not disagreeing that some things are hard to understand, and if I was the dev id name a few things differently too, but ultimately I do not think you will get them to change their current philosophy. They rather things be correct than easy. And I for one like that. Not every software has to cater for the same crowd.
14 Likes
Sigh. @OK1, do you realize that you (perhaps inadvertently) keep writing posts that suggest you are the supremely knowledgeable person about how things should be done? Most likely you will rub any developer the wrong way with this attitude. The extreme length of your posts does not help. Please reconsider what you want to achieve and how you approach that goal. Some modesty about the relevance of your particular wishes would be nice, instead of generalizing to the world of common photographers. Try to understand the approach of the equally dedicated and visionary developers, instead of keeping sketching your utopia.
12 Likes
Apologies for my oversimplification, to make a point.
Sure - any accomplished painter will care a lot about their paints, and will most likely know exactly how they are made.
I second this. These threads are for conversations, and conversations can only really happen when posts are kept short and to the point. Overly long posts simply will not be read.
4 Likes
Noted. Thanks for the feedback. I’ll post any further long form thoughts, on my own blog, to avoid ruffling feathers on pixls, and also limit my outreach, to the developers, to direct comms with them via private messaging.
Don’t mistake my criticism for an appeal to stop posting. If you want others to contribute to your thoughts and ideas about development, it’s important to keep it in the public.
8 Likes
The above seems about your central concern, yes?
There are plenty of “easy” programs out there that hide their minutiae behind “easy” sliders. Darktable does not. That’s what’s great about it. I agree, to an extent, that that’s also what’s annoying about it, but for better or worse, Darktable is not the place for “easy” controls.
Instead, Darktable is where we can think in terms of “Latitude” and “Hue Preservation” and all the associated technical jargon. If you what Lightroom, use Lightroom.
Indeed, I didn’t read @OK1’s super verbose posts because of their length. It is a good policy in any conversation to keep it in digestible chunks. Otherwise, you would risk alienating people and giving the impression that you are the most important person in the room.
If you must write a wall of text, be sure to summarize it at the top or hide it with this handy code:
[details=hidden title] hidden text [/details]
It would look like this:
hidden title
hidden text
Carry on.
4 Likes
Why? whats the benefit using different names for well defined terms? It’s already hard enough to translate terms into different languages without losing precision.
But feel free to provide a simplified_terms.po file to check if using different terms is really a benefit 
Thanks for trying it out OK1!
For your input, I will see if I can break it down into something replayable, it’s a lot to chew at once.
Lens coating thing, no it should never be part of any global tone mapper or as you wrote std. Why? It is a local effect that is easy to simulate in scene-referred space without any other dependencies.
Not sure what you want to say with the example images. I have yet to figure out any set of reasonable presets but I feel quite good about the default as that is close to the ACES RRT+ODT (See earlier posts on the subject if you are unfamiliar). As for SOOC jpgs, I think these should be possible to recreate with ease in the raw developer as they are a valid edit, but I also see how unreasonable it is to deliver perfect matches as presets. To many brands, cameras, styles, and options to every make that feasible. Would you say that the sigmoid makes it easier or harder for you to recreate the jpg style? That would be interesting feedback!
-
darktable is very much a dodge and burn tool, especially with the new scene-referred workflow. You are losing ALOT of potential if you do not want to do that in darktable. Pretty much all images I have posted in this thread are based on simple dodge and burn using masked exposure/tone equalizer before the sigmoid module.
-
I think it is unavoidable to become technical when this much control is exposed, the reason Lightroom can use friendlier words is because they have fewer things to turn on. I’m open to suggestions though! I think the most important thing is to stay consistent within the software, that is why I used the same naming for display max and min luminance as in filmic f.ex. I’m actually considering renaming contrast to compression as it changes both contrast and saturation…
I know plenty of very technical women, no need to use that sort of generalization here. The module is still under development, so the topic of discussion is of course focused on the math and science behind it. There is no point in merging the module if the math doesn’t work out.
- Middle grey control in filmic is the same as an exposure change. Just use the exposure module to get the same effect. Do not use the shadows and highlights or color zones module modules for this as they don’t do proper scene-referred math.
a. Mock ups and suggestions are welcome, this is my best effort so far.
b. Of course, but later, not now.
c. Kind of what I’m using this thread for, more should follow as things stabilize.
d. Really no use to figure out presets yet as they will just break a few weeks from now because I made a change in the code. We have to figure out the math first.
- No, absolutely not, I have no interest in doing anything like that. I think you are vastly underestimating how much work it is to maintain software like darktable. And, I have no problem with the sigmoid not begin merged. I do all of this because it has been an awesome platform to learn more about digital image editing, not to start a new competing darktable version.
Thanks a lot!
Hope to see more images and testing coming from you (but maybe in smaller chunks as that makes it easier to answer properly )
5 Likes
Played around with it a bit and could at least reproduce the level of the filmic edits. I think the main problem in this image for darktable-users is that the highlight reconstruction is quite poor which leaves most of the sky wonky in this picture. There is information in there to make a much better highlight reconstruction than just clipping all channels over 1 though. This would indeed be a interesting topic to tackle at some point but nothing I can do atm. So the sigmoid module won’t be the hero in this case, it would rather be a proper scene-referred linear-RGB update of the highlights reconstruction module! (This is partly what I think is exciting about darktable development, there are still so many lose threads in the new scene-referred workflow that have to be figured out).
2 Likes
I haven’t had time to fully explore I was messing around with a bunch of filmic parameters including setting the latitude as low as it would go when I decided to try blending filmic in lightness blend mode. Not much changed in the couple of examples I tried but the sky for some reason is less desaturated maybe but I actually liked the look. I will experiment more and summarize…
EDIT: I tried it as I often do this if I use dehaze and it shift the color too much so I just thought what would this look like with filmic…
Most professional photographers are using Lightroom and they’re getting great results very fast in Lightroom. I think we should acknowledge this fact.
Thank you for highlighting (
) this. Good highlight reconstruction is a must in any raw converter for me. Sadly, it’s the only thing holding me back from using darktable. I’m all RawTherapee 5.8 (personal projects) and Lightroom (work) at the moment. I’m not a developer, but having something like this in dt would be amazing. BTW, I’ve been experimenting with Sigmoid builds of darktable though and I was very pleasantly surprised. Generally I’m getting better results than with Filmic RGB with less time consumed. Win-win. Great work @jandren!
Do you say that as a developer, or a user (or both)? 
3 Likes
True, and my reply could be seen as a generalization itself, although I did preface that by saying “mostly.” But whether pro photographers use it or not, it’s still true a lot of commercial software, including lightroom, targets mass consumption, and simplifies some things to achieve their aim. Not all pro photographers are pro editors or have the time to be, so they rely on this simplification.
1 Like
Thats done - If Lightroom is satisfying the demand of a professional photographer then there’s no need to change the tools. There’s no ambition for darktable to take over as much users from Lightroom etc. as possible.
darktable scope is to provide the options Lightroom doesn’t give to you.
If you’re demanding just a free Lightroom clone then darktable is simply the wrong place to search for it.
It’s like the green rectangle on your camera - there‘re plenty of user for this but at least ambitious users prefers to customize their tools. darktable is focused on the M mode
6 Likes