Place holder for a discussion, to be continued, follow up to some chat on this other thread
and more direct development of this aspect 
2 Likes
- Why attempt to make a photo look like the JPG, at least for educational purposes.
a) I think it is a useful learning exercise, and for those who have never processed RAW files, like me for 12 years I had taken jpg photos like everyone else and knew no better, even though all this while I was aware of RAW, but never pursued this, cos it requires tooling - like Lightroom, a further expense. Unless you are pretty serious about the photography or curious, no one bothers to add the hurdle of processing in RAW.
b) The ooc jpg, typically taken using All Auto settings on the camera, cos seriously thatās what most snappers at this level do - leave almost everything to the cameras choices. So its a decent benchmark, to work against. in developing RAW files from scratch. Case in point RAW Therapee, which I did not get on well with, attempts to create a ābase curveā from information gleaned in the embedded jpg stored in the RAW file, so there must be some value in this comparison.
I will have to read up on terms like scene referred or display referred - so cannot comment on these cos I have no clue what these are about.
What I find interesting is that each of these approaches while I do not use them together typically, complemented my learning, about the other.
- I STARTED WITH CUSTOM BASE CURVES
I spent a while frustrated by using custom base curves and was not really pleased with the results, in spite of all the effort.
- THEN LOTS OF LUTS
Then I discovered LUTS via the rawtherapee dump of LUTS, and other LUTS resources on the web.
And in my study of video I discovered that in the film world, using LUTS is pretty standard, everybody uses them - i.e. pretty much every single major or minor video we have ever seen in the last 5 years has LUTS on them. So I opined, there must be some value in looking at this approach for photos.
In simple terms, the typical film uses a custom enhanced form of the gamma curve which is used to preserve as much detail in the image, captured in the RAW file typically called a log (i.e logarithmic) so they have things like Canonās Clog, CLog2, etc. and Sony has their own, in fact everyone making serious cinema cameras has their own logs.
They then provide LUTS to enable you transform an image captured in their log transformation, into an acceptable starting point, for your further tweaking of the image. This type of LUT is known as a Technical LUT. (there are other types of LUTS which one can call Creative LUTS whose purpose is more to enhance rather than transform from RAW).
So having played around with the LUTS in RAW Therapee, I found much greater success using the Technical LUTS from say Canon, which are free to download.
Typically it would be using a Canon LUT to convert from some Canon log format to Rec 709, of course one is welcome to see the effect of the other LUTS provided in the Canon LUT download, its art at the end of the day, underpinned by science(from inks in the days of hand paintings, to chemical formulas for films substrates, and now digital equivalents).
The advantage of this is it gives one a consistent starting point, from RAW to a pleasing image, that looks more like real life, rather than the contrast reduced RAW file.
This was the start, and over time you discover which technical LUT conversions you prefer the most, from the LUTS directly downloadable from the cinema camera manufacturers.
Because your original RAW file is not created with as much of a logarithmic transformation as the expected input of these technical LUTS, the result of applying a Technical LUT may tend to be oversaturated, so some adjustment is needed to compensate, such as
a) reducing Saturation using the Contrast, Brightness, saturation darktable module.
b) Altering Contrast using same, or using the Exposure module sliders.
Generally those in the Film industry have very different approaches to photography, cos their particular focus is on mid tones, so in my experience using Technical LUTS, the dark areas get crushed when applied to non cinema log raw inputs. Cos the Technical LUT was expecting an input file with a very different kind of curve. Typically some lifting of shadows is needed, and some brightening.
I read your concern about LUTS having an issue with interpolation. My response is - if this is good enough for the film industry who have millions of dollars to spend on the stuff, it should be good enough for photography. But there is also the issue of LUT quality, which is based on the size of teh cube, so what I have done from prior experimentation is use the largest size version of the LUT(cube) where possible, and this certainly has a positive effect on the image, however slight an improvement. Typically 33 x 33 x 33 should be good enough. Larger is obviously better.
Next step in the journey was discovering some web sites which generate Technical LUTs, using their own algorithms, and offer opportunities to tweak these LUTS. One of these has an off line app also - google LUTCalc. albeit I do not use these LUTS cos I have not figured out a way to fix their excesses - ok if you want that bright commercial look.
Another source is at the link below :
So the idea is - I generated lots of LUT conversions from various CInema Log input gamma formats to Rec 709. Hint - important hint - set Gamut of input and output to āpassthroughā
Experiment with āauto adjust to optimise dynamic rangeā - enable this.
So using these approaches I generated sets of Technical LUTS, from each gamma log of interest such as Canon Log 2 - to every other target output gamma and also made sure i included one for the conversion to Linear - which has a sub option to select Rec 709 y1.90 -( exp 2.22).
Then went through a whole comparison process, to deduce what each of these transforms were doing - LUTCalcās site helped me better in this regard, looking at the curves. But it is an involved process, experimenting and deducing which log formats were stronger and less strong and why.
At the end of the day most of the time I just use a Canon Log 2 to Rec 709 y1.90 (exp 2.22) Technical conversion LUT generated via one of these sites. an I can use other modules in darktable to tweak further.
The important aspect of using LUTs was it accelerated my workflow, to the point that I also created darktable styles to apply a whole set of modules to each image using my most preferred default LUTS - pretty much like Light room presets. The only thing I find is that unlike Lightroom presets, the styles in darktable are cumulative, if you apply a 2nd style it adds, not replaces the 1st style. Further more over time I stopped using styles, cos using LUTS already accelerated the workflow so much that it did not need any further acceleration to combine modules automatically, I ended up preferring not to use my darktable styles which I created.
- ALL YOU CAN EAT BUFFET - LUTS or Base Curves - the more the merrier
Then one fine day, thank to darktableās awesome duplicate feature, I was comparing the LUT approach with some of my custom base curves - which I had abandoned, and in many cases I ended up preferring the base curves, so now, depending on my mood, and what the image needs, I have a choice - to try some Technical LUT conversions, which I generated from these aforementioned sites, or some of my custom hand made base curves
So much easier to flip through a bunch of presets to get you as close to where you want to be, and this varies with each image, and reduce the amount of further tweaking you need to do.
And it is good to have a range of starting points to energise creativity and keep things fresh, one makes surprising discoveries by trying out alternatives that would not ordinarily make any sense.
The discussion of LUTS above is predominantly about using them in a Technical conversion, any other use of LUTS can be considered as using them in a Creative way. to further augment a base conversion, including instances where I have used a base curve instead of a technical LUT, and can then augment with a Creative LUT for artistic enhancement.
- Filmic
In the middle of all this I tried to get on with filmic, my conclusion is filmic works for certain kinds of images, and I really could not get a handle on its controls. In general if a module in darktable has lots of features, one tends to abandon it and look for something simpler. In music you have a similar parallel - complex synthesis methods like Yamahas Frequency modulation which was difficult to program using menus and sub menus vs analog style synths where all the parameters - fewer of were immediately accessible using controls - one per parameter. Too many controls and sliders means I rarely remember what does what - so technically it may be a good tool, but from a human standpoint, maybe in future revisions it will be more accessible to humans like me.
You can see the amount of personal effort I have invested in LUTS and Base curves, generating my own sets by hand, so you can imagine I must have put in similar effort to get Filmic RGB to succeed in my workflow - Well you cannot win them all - win some - lose some.
The end result of this workflow improvement with custom base curves or LUTS taking on the heavy lifting, and tailoring as close as possible to how I like my images to look, so its a bit of a personal thing - to oneās own taste. Most of the time I end up using just a few modules .
- White Balance
- Denoise (I change which one depending on the image) with practice it takes only a few seconds to achieve a suitable result.
- Crop and Rotate (if needed)
- Exposure
- Base Curve OR LUT
- Tone Curve
- Highpass for sharpening
Note that at this point in time, my priority is to have an optimised workflow that quickly gets me to a very nice presentation of the image I took in camera, so I can quickly process lots of images, without losing the creative juice or objectivity which happens when you are staring at an image for too long.
And with darktable, thankfully its easy to copy from one pic to another, if these images were taken in similar conditions, and minor changes can then be adjusted on each one. 90 % of the time once I get the 6 aforementioned modules right for an image - in about a minute or two, so easy to just copy to all other images taken in the same shooting session/subject.
Just like when working on music for a long time, same with video, our eyes tire and lose objectivity. I think a really quick workflow as defined above has been critical to sustaining my interest in photography.
I still do not know how to export and share my base curves, so thatās something I must now learn,
Will be back and post links to download sites of these, when my learning is done.
I must add as a caveat, further to all of this muscle and eye correlation with images, has made me adept at using the tone curve for so many of the things that other would use more modules for. I use this (and the exposure module in concert) for changing contrast, brightening, darkening, not just the whole image but specific areas of the dark to light range - highlights, shadows, as it has become 2nd nature to more precisely do these things in tone curve, far more precisely than I can achieve in broadstroke tools like Contrast, Brightness and Saturation. Saturation is one of the exceptiions where the Saturation slider in CBS module does not have an equivalent in the tone curve module.
Been a long journey - about 6 or 7 months, but finally pretty happy with the results. Quick, repeatable and somewhat easy when you know how.
Addendum - the over exposure /under exposure highlight toggle tools are our best friends, helps to keep us in check. still learning the gammut one cos I am guilty of violating its limitsā¦all the time !! so I keep that one turned off for now.
2 Likes
Wow, that is quite a workflow!
True, it works phenomenally on well exposed, unclipped images.
I donāt find this to be true for me personally. I think the devs have given me every opportunity for control, which as an artist, I appreciate.
I think you have to use a tool until it becomes intuitive. Or take some notes.
2 Likes
Sometimes better to share the results, far more interesting than the long talk about the process. An image like this took me about 3 minutes to edit, and thatās cos it needed quite a bit of rescuing, taken on a dull rainy cloudy late afternoon in November in the Northern hemisphere.
Otherwise typical photos are done in less than 2 minutes, when taken in good light.
I guess its horses for courses, does one want to spend lots of time on one or two pics, or achieve really good looking pics with minimal effort, and take so many more pictures with the confidence that editing will be a pleasure, and easy to achieve.
I love all the tooling and this is great for creating digital works of splendid art, but for certain pursuits like a hobby which this is for me at this time, the real joy is in taking the pics and admiring the end result., rather than hours spent polishing one image to perfection. Its never perfect, and there are thousands of variations of perfect on the same original image, so sometimes all that slaving away on one image - especially if itās just a hobby, becomes drudgery and somewhat futile, ending up with several excellent versions of the same image, different but each of them excellent - some what futile cos each time you approach it - you end up with a slightly different result - so so no point in slaving on the pic, hit the main editing improvements in a few seconds, and spend more time admiring the result.
I guess its the same with cars, some like to tinker with them, I prefer to drive them, and only tinker if it saves me money.
1 Like
Just FYI Styles can append or replace just depends on the setting in place before they are appliedā¦
As a work around add all your base curves one after the other. Export the xmpā¦People can just load a step and save as a new preset in the base curve module. They can just do it for each oneā¦not pretty but should workā¦
So as an example if I has a style (Style 1)that modified 5 modules, and maybe introduced 1 new module as a 2nd instance, of an existing module.
And I want to choose another style.(Style 2) which makes its own changes, i.e when I apply the new style, I want the changes made bye 1st style to be completely reversed and replaced with changes made by the new module.
This way I can store a collection of module changes in each style and switch between styles in the same way I see on Lightroom demos of Lightroom presets on Youtube,
Is this possible in darktable?
This style/module append/replace toggle - where do I find this in darktable?
An interesting approach - never thought of that. pretty easy to achieveā¦
Here you go - three of my custom base curves - all turned off , Raw file and xmp attached.
I have done no further development to this image - attached only to test your suggestion as a way of sharing base curve presets.
DSC00999.ARW (16.4 MB)
DSC00999.ARW.xmp (94.4 KB)
Base Curve
Base Curve 1
Base Curve 2
Styles panel is one place in lighttable and second you can also just take an existing image and copy the history stack ctrl cā¦and if you use shift ctrl v on another or othersā¦you can select what parts get appliedā¦If you apply the style in dark room then it uses the current styles overwrite mode from lighttableā¦
Thanks - something to take a close look at, appreciated.
Wow I seem to have made a few replies today, as a ānewā user, so the forum policies have placed a 45 minute delay on posting this response - interesting. This explains any delay in my response.
I wonder how long it takes to become a non new user.!! I do understand - in a strange world, the conduct of some of us, causes these kinds of safeguards to be imposed on the others.
Display referred means tone curve comes early in pipe. Many raw editors work this way, but it is generally considered inferior because tone/colour mixing looks nicer done in linear, and a curve is non-linear. So modules after suffer. darktable has base curve for this.
Scene referred means tone curve comes late in pipe. Few raw editors work this way, but is generally considered superior because it allows lots of modules to work in linear before the non-linear curve. Darktable has filmic for thisā¦ But filmic is more complex than just a curve. First tab is log tone mapping. This means you can push data beyond clipping in earlier modules, and filmic can bring that data back. You do that by adjusting White relative. Black relative takes care of the shadows. These sliders also determine where the pivot point of the curve is. The curve shape is controlled using contrast, latitude and softness/hardness.
But you donāt have to use filmic for the curve - you can make the filmic curve straight, linear, and use another module for the curve. Tone curve, base curve (moved after filmic in the pipe), lut, and tone equaliser are all options. You can even use color balance, which has a few film like Presets.
Iāve been playing around with this recently, but it has only made me more aware of how remarkable filmic is. For instance, if I first make the filmic curve flat, set Black and White relative points, then go back to set contrast, latitude and hardness, I nearly always get better results than any preset or other module. And except for complex cases, its fast. The problem I was having initially is that default contrast is often too high and alters your perception shifting other sliders, which is why I now set it linear to start with. I also turn mid tones saturation off (prefer color balance), and use preserve chrominance: no, as default setting, then toggle through the others to finish. Now, I use a few favourite luts/Presets mainly for comparison.
But assuming a linear filmic curve, Iām wondering if anyone can explain the pros and cons of base curve vs tone curve (both placed after filmic in pipe)?
My workflow is usually pretty short, consisting of exposure/Filmic, local contrast, tone equalizer, color balance (saturation, contrast, and tweak shadows, midtones, and highlights), and sharpen/contrast equalizer, and the finish with crop. Most of the time I donāt even need to touch Filmic because the default settings are spot on at the start.
That usually takes me about two minutes. Darktable has a nice system to copy and paste history, so Iāll frequently work on one image and then copy it to the rest of that set. I can usually get through 150-300 images in about an hour. After that Iāll go back and play with the really special pictures to get them just rightā¦ that might take awhile.
So, itās great to have a program like DT (and Rawtherapee, too) that can accommodate so many different ways to edit photos.
1 Like
Base curve is just a tone curve designed to approach the tonal adjustment that would produce the camera JPGā¦I think that is all really esp now you can move modules aroundā¦before quite often it would produce blown highlights for many imagesā¦The other modification is done with DT -chart. That will create a style that has a tone curve and a lut combo to match your image to either your jpg reference or to the reference values for a color chartā¦if you donāt have a chart then for general lighting you can often find raw/jpg pairs for common cameras on imaging resources website. Its a crude approach but I tried it for a few people and they remarked that it was a better starting point than the base curveā¦
1 Like
Please note that a straight line in the filmic chart does not mean a linear mapping: the x-axis is logarithmic (itās scaled in EV).
Just consider this: the filmic mapping is defined by 5 points: mid-grey (fixed), white relative EV and black relative EV, plus 2 points for latitude. A truly linear mapping is defined by just two points. This means that if e.g. black relative EV is -5 (mapping it to LAB L = 0), and mid-grey (around which filmic builds the mapping) is always mapped to 50% LAB, white relative EV would always have to be +5EV (mapped to L = 100%). However, youāre free to change the white and black points, and filmic changes parameters of the non-linear mapping in the background.
1 Like
Sorry, I didnāt express this correctly: of course this is not linear, either:
if e.g. black relative EV is -5 (mapping it to LAB L = 0), and mid-grey (around which filmic builds the mapping) is always mapped to 50% LAB, white relative EV would always have to be +5EV (mapped to L = 100%)
Itād be ālinearā (a straight line) when mapped in log->linear space; however, what I meant is that changing the white and black points would change the slope in log ā linear, which would also move the y value mid-grey is mapped to; however, filmic always ensures a neural mapping for mid-grey.
Now this is advanced level stuff. Iāll come back to digest this. All in due course.
1 Like
Same Image processed three different ways. With some other modules like tone curve, and contrast equalizer applied in eachā¦ 1st time I ever did this. In this example, Not much to go on between the different approaches. Who knows, maybe it depends on the image. Without examining them side by side, they look pretty close.
- Filmic
2, Custom Base Curve
- LUT
1 Like
I like that LUT rendering! Which LUT did you use? And where did you find your favorite LUTs?
Can you expand on this? When I set contrast to 1, and highlights and shadows to āsoftā, I get a straight line. When I adjust white and black relative sliders, the orange circle moves up and down the line, and if I now add contrast, functions as the s curve pivot point. So is it correct that the log tone mapping part of filmic (scene tab) always pivots around mid grey (as determined by exposure/previous modules/mid grey luminance slider? While the curve (look tab) pivots around the orange circle (as determined by white and black relative sliders), which is not necessarily middle grey (is only middle grey when white and black relative are inverse of each other, eg. +5 and -5, +6 and -6, etcā¦)?
Check out this ramp. Filmic is shown, but itās disabled; the marked area (visible due to ādisplay sample areas on imageā in the global colour picker) shows mid-grey, or LAB 50%:
Take a snapshot of this starting state.
Hover over filmic and enable labels. Youāll see that the orange dot (18%) is mapped to 18% on the y axis:
Compare with snapshot: mid-grey is unchanged:
Make some crazy adjustments to filmic: mid-grey remains mapped to mid-grey.
This remains true even if the mapping is completely bonkers, like this:
5 Likes