I was doing some further testing/review/use in practice of sigmoid over the last day or so, and have a few more observations. I am following the “protocols” on this forum, so the details of each topic are “hidden”. Each forum has its own “culture”.
1. The Darkness. (not a horror movie!)
The impression I had formed and discussed earlier, about the tendency to “darkness”, which led me to try to find some way of resolving this.
If you turn on the clipping indicator (ideally set to clipping preview mode = Full Gamut), the images with this darkness, are those where settings in sigmoid have caused the dark regions to dip below the black point. So even aspects of the image, which are not 100% black still have some of the darkness applied.
All attempts to use other tools to resolve this (the new colorbalance rgb, rgb curves, etc) met with mixed success. Improved but not resolved.
When the clipping indicator hinted at what the cause of the issue may be, next step I took was to use the black point correction slider in exposure module, set to a negative value, which brings the black point back to where it should be and there are no more warnings in the clip indicator. But this black point adjustment is a broad one affecting more of the image than desired. so the entire image goes a bit dull. Not the most ideal solution.
The optimal solution, which I was satisfied with was to leave the exposure module as is, no adjustment of black point in the 1st instance, and use a 2nd instance of exposure module, which processes the image, after the 1st instance, and a parametric filter set to a grey value range that restricts any change to only the darkest parts of the image. In this 2nd instance of the exposure module, the darker parts of the image can be modified using the exposure slider in the exposure module. Positive EV, to brighten things up in the dark areas, and Negative EV, for the opposite.
Its my workaround, so I can us sigmoid with a lot more “latitude” i.e much more than I could using only the features in sigmoid alone. in some ways, this is not unusual, every tool will have its limits, or its own quirks, which need other tools to complement it.
I am not sure if this issue is something that needs to b fixed in sigmoid, or something that the users will accept and find workarounds for, as I have done, and possibly others may find my workaround something which helps them also.
I do not know what could be done in sigmoid, but if I were to express myself in layman’s terms, I would almost wish it was possible to move the target black indicator in sigmoid, to a value below its current minimum. Whether that is correct maths, or valid in image processing, I cannot say, as this not my area of expertise. Am only an end user, in this case.
Good enough, if it is fixed in sigmoid, or via a workaround as I have discovered, what’s important is there is a solution, and for me the concern I had with the dark tendency, is no longer a showstopper.
2. The Contrast
To a certain extent, yes sigmoid enables low contrast images, but the range of values on the contrast slider, within which one can lower contrast without turning the image to something undesirable, is a bit small, so one has to be quite delicate with the mouse or use numbers or other value adjustment feature with right click - can’t recall the name of this user interface feature, but I’m sure you know what I mean.
Somewhere around the midpoint of the contrast slider, in the current version of sigmoid, if one could have finer control, or maybe a 2nd slider that allows the use of the entire width of the tool area to make fine adjustments to the value of the contrast slider.
Ultimately as the custodian of sigmoid, at this time, its up to you to decide when and how, a more formal structure will be established to coordinate the progress of sigmoid. At this time, its just you - one person. On one thread in pixls.us.
At some point in time, to obtain the maximum from the community, it would be helpful, to have the following, if it were possible.
a. A Roadmap. What’s the direction of the “project”. Without a very specific agenda, it becomes difficult for others to know exactly how to help, cos the direction is undefined, or loosely defined, or not yet communicated.
b. Repository. While sigmoid has a code repository, I hope you will agree that all the other items mentioned in this topic, also need to be stored in this central sigmoid repository, ideally also on github, as the current thread, may not be the ideal place to enable others to read, learn, and understand what sigmoid is about. The current protocol requires any discoverer to thumb through an entire thread, which is the only place where information on sigmoid can be found. Not very elegant. Or easy to do.
c. User docs. Sure if and when this sigmoid is formally included in dt, I am sure there will need to include documentation in the manual. There might be value in developing this now.
d. Project Team. Would sigmoid benefit from having a formal team of contributors, which can vary in composition as the need arises, who work with you a bit more closely, than this is possible via this forum thread alone, to augment your efforts? At such a time as you may consider you need this team, please establish.
e. Requirements. Each version, feature set/build will need a set of its own requirements, otherwise testing will not have the utmost efficiency, cos there is nothing to check against.
f. Testing. At this time, we may consider sigmoid as a “Proof of Concept”, so may not need rigorous tests, but at some point in time, there will be the need to have formal tests, and these tests are best achieved only when conducted against a clear set of deliverables, defined prior in the Requirements for each build/version. Would there be the need to collate a set of test raw images, against which all testing will be conducted. These images being a representative test of all kinds, overexposed, underexposed, saturated, out of gamma, etc, etc…
g. The build environment/Strategy. I would expect that most of those interested in sigmoid, would be running at least three environments of dt. In my case there is the version 220.127.116.11 (stable), then the latest dev version which I built about 2 weeks ago, and then there is the build derived from your github repository. Until sigmoid is integrated into dt, would it be possible to have a monthly refresh of your custom build, which is synced up with the latest dt dev code.
The sigmoid build, includes a version of dt dev which is several months behind the current code version. If your sigmoid build, could be synced monthly and refreshed with the latest dt code, then for many of us, we can keep only two instances of dt running, one with sigmoid which is no more than one month out of sync with the latest dev build, and our usual stable dt release. This allow us to test and experience the most recent developments in dt(no more than one month out of sync), with your sigmoid code, in the same build.
This text will be hidden
4. The DT pipeline may have some issues, which affect perception
I sincerely always wish it were possible to discuss sigmoid, independently of darktable’s other features, but this is not so, as much as we or you may wish it. Why? at this time, sigmoid relies on the DT infrastructure, to provide other services, on which sigmoid must rely.
DT provides demosaicing of the raw files, and allows us to display the image, etc, etc. export the image, and whatever we view or export is heavily reliant on the quality of the services provided by other DT features/modules.
While testing sigmoid, it is only sensible that I would compare its results with other methods of achieving the same thing, in DT or in other tools.
Whatever I say next, is a function of many things, for any other who reads this to be able to appreciate. In testing DT from different environments, we do not have the same displays, the same in-room lighting, and neither do we all have the same quality of eyesight(vision and colour appreciation), and for the kind of evaluation I have done, which is not based on any scientific tools, using only eyesight, any comments I make here, are impossible for me to verify with any measurement tools. I do not have the skill or experience to compare images scientifically, and can only describe what my eyes see. Others may have a different way of seeing these images.
When comparing images across different raw processors, I tried to exclude any variables, with the raw processor being the only variable. But this is impossible, except when using filmic, which shares the same pipeline as sigmoid, in DT. When using other non DT raw processors, which have their own sharpening and noise reduction, and colour science, I tried as much as possible to arrive at an equivalent way of comparing results with sigmoid.
It revealed something most unexpected. I either do not have the skills to get the best out of DT’s noise removal and sharpening modules, or DT’s noise removal and sharpening modules deliver results that also “colour” the final outcome, in their own unique way - a way which I was not especially pleased with.
In my comparative tests, I noted that the sharpening/noise tools in Lightzone (I’m using version 4.1.1 which I found to be the most stable and resource efficient on Windows) was IMHO producing images which looked quite visibly better than the results from DT.
To prove this conclusively, I exported an image from DT, processed via sigmoid, with:
- All sharpening and noise management turned off
- Exported as a 16 bit Tiff, using LinearProphoto RGB as colour space, uncompressed
- Imported into LIghtzone
- Applied Lightzone’s sharpening and noise reduction.
- Exported from Lightzone as tiff 16 bit keeping the Linear ProPhoto colour space
- Imported tiff from Lightzone back into DT.
The Linear Prophoto colour space may not be significant. I have not attempted using sRGB as the colour space during the transfer from darktable to Lightzone and back, but I do not think this should be of significance.
I was rewarded with the most impressive view, of what sigmoid is capable of, when the image has had an alternative processing to what I could achieve with the current sharpening/noise removal tools in DT.
I need not provide a photo, or image, cos this is science - hopefully, I have outlined the steps I followed and anyone else can check my assertion, by themselves.
For whatever the reason, my own skills or the excellence of Lightzone, processing sharpness with Lightzone, was visually more appealing, than with DT. I could have vouched that I had had a camera sensor upgrade.
I mention all this especially for anyone testing raw processors, as the raw processor is just one factor in the “environment”, and many other factors outside the control of the raw processor provide an objective or subjective difference to the end result.
If there was one word to describe the difference - it is resolution. With ligtzone’s sharpening, I was seeing so much more, almost as if the image had several times more megapixels, or my display had more megapixels, or my lens had developed a macro ability of microscopic vision. The level of detail I could see in the image was so much more… And its something everyone can try out for themselves. The image from DT modules alone is great, until you see an unsharpened image from DT, sharpened in lightzone, you do not want to go back.
I would recommend that while sigmoid is unavoidably tied to DT, anyone testing sigmoid, should adopt a process similar to that described above, so they can get a more revealing image of the end result of sigmoid, and can see so much more of what it does.
Mine is not to explain why, I have no clue of the sharpening code in Lightzone or DT, can only describe what my eyes observed. A pretty shocking alternative, and relevant to anyone testing any modules in DT.
5. The Colours
From values above 2.7 or thereabouts, the contrast slider, creates images which are definitely out-of-this-world, more like an effect, which some may value, but not a natural rendition. More like you turned on some artificial lighting on the image, and colours become vivid. Do keep this feature.