- Why attempt to make a photo look like the JPG, at least for educational purposes.
a) I think it is a useful learning exercise, and for those who have never processed RAW files, like me for 12 years I had taken jpg photos like everyone else and knew no better, even though all this while I was aware of RAW, but never pursued this, cos it requires tooling - like Lightroom, a further expense. Unless you are pretty serious about the photography or curious, no one bothers to add the hurdle of processing in RAW.
b) The ooc jpg, typically taken using All Auto settings on the camera, cos seriously that’s what most snappers at this level do - leave almost everything to the cameras choices. So its a decent benchmark, to work against. in developing RAW files from scratch. Case in point RAW Therapee, which I did not get on well with, attempts to create a “base curve” from information gleaned in the embedded jpg stored in the RAW file, so there must be some value in this comparison.
I will have to read up on terms like scene referred or display referred - so cannot comment on these cos I have no clue what these are about.
What I find interesting is that each of these approaches while I do not use them together typically, complemented my learning, about the other.
- I STARTED WITH CUSTOM BASE CURVES
I spent a while frustrated by using custom base curves and was not really pleased with the results, in spite of all the effort.
- THEN LOTS OF LUTS
Then I discovered LUTS via the rawtherapee dump of LUTS, and other LUTS resources on the web.
And in my study of video I discovered that in the film world, using LUTS is pretty standard, everybody uses them - i.e. pretty much every single major or minor video we have ever seen in the last 5 years has LUTS on them. So I opined, there must be some value in looking at this approach for photos.
In simple terms, the typical film uses a custom enhanced form of the gamma curve which is used to preserve as much detail in the image, captured in the RAW file typically called a log (i.e logarithmic) so they have things like Canon’s Clog, CLog2, etc. and Sony has their own, in fact everyone making serious cinema cameras has their own logs.
They then provide LUTS to enable you transform an image captured in their log transformation, into an acceptable starting point, for your further tweaking of the image. This type of LUT is known as a Technical LUT. (there are other types of LUTS which one can call Creative LUTS whose purpose is more to enhance rather than transform from RAW).
So having played around with the LUTS in RAW Therapee, I found much greater success using the Technical LUTS from say Canon, which are free to download.
Typically it would be using a Canon LUT to convert from some Canon log format to Rec 709, of course one is welcome to see the effect of the other LUTS provided in the Canon LUT download, its art at the end of the day, underpinned by science(from inks in the days of hand paintings, to chemical formulas for films substrates, and now digital equivalents).
The advantage of this is it gives one a consistent starting point, from RAW to a pleasing image, that looks more like real life, rather than the contrast reduced RAW file.
This was the start, and over time you discover which technical LUT conversions you prefer the most, from the LUTS directly downloadable from the cinema camera manufacturers.
Because your original RAW file is not created with as much of a logarithmic transformation as the expected input of these technical LUTS, the result of applying a Technical LUT may tend to be oversaturated, so some adjustment is needed to compensate, such as
a) reducing Saturation using the Contrast, Brightness, saturation darktable module.
b) Altering Contrast using same, or using the Exposure module sliders.
Generally those in the Film industry have very different approaches to photography, cos their particular focus is on mid tones, so in my experience using Technical LUTS, the dark areas get crushed when applied to non cinema log raw inputs. Cos the Technical LUT was expecting an input file with a very different kind of curve. Typically some lifting of shadows is needed, and some brightening.
I read your concern about LUTS having an issue with interpolation. My response is - if this is good enough for the film industry who have millions of dollars to spend on the stuff, it should be good enough for photography. But there is also the issue of LUT quality, which is based on the size of teh cube, so what I have done from prior experimentation is use the largest size version of the LUT(cube) where possible, and this certainly has a positive effect on the image, however slight an improvement. Typically 33 x 33 x 33 should be good enough. Larger is obviously better.
Next step in the journey was discovering some web sites which generate Technical LUTs, using their own algorithms, and offer opportunities to tweak these LUTS. One of these has an off line app also - google LUTCalc. albeit I do not use these LUTS cos I have not figured out a way to fix their excesses - ok if you want that bright commercial look.
Another source is at the link below :
So the idea is - I generated lots of LUT conversions from various CInema Log input gamma formats to Rec 709. Hint - important hint - set Gamut of input and output to “passthrough”
Experiment with “auto adjust to optimise dynamic range” - enable this.
So using these approaches I generated sets of Technical LUTS, from each gamma log of interest such as Canon Log 2 - to every other target output gamma and also made sure i included one for the conversion to Linear - which has a sub option to select Rec 709 y1.90 -( exp 2.22).
Then went through a whole comparison process, to deduce what each of these transforms were doing - LUTCalc’s site helped me better in this regard, looking at the curves. But it is an involved process, experimenting and deducing which log formats were stronger and less strong and why.
At the end of the day most of the time I just use a Canon Log 2 to Rec 709 y1.90 (exp 2.22) Technical conversion LUT generated via one of these sites. an I can use other modules in darktable to tweak further.
The important aspect of using LUTs was it accelerated my workflow, to the point that I also created darktable styles to apply a whole set of modules to each image using my most preferred default LUTS - pretty much like Light room presets. The only thing I find is that unlike Lightroom presets, the styles in darktable are cumulative, if you apply a 2nd style it adds, not replaces the 1st style. Further more over time I stopped using styles, cos using LUTS already accelerated the workflow so much that it did not need any further acceleration to combine modules automatically, I ended up preferring not to use my darktable styles which I created.
- ALL YOU CAN EAT BUFFET - LUTS or Base Curves - the more the merrier
Then one fine day, thank to darktable’s awesome duplicate feature, I was comparing the LUT approach with some of my custom base curves - which I had abandoned, and in many cases I ended up preferring the base curves, so now, depending on my mood, and what the image needs, I have a choice - to try some Technical LUT conversions, which I generated from these aforementioned sites, or some of my custom hand made base curves
So much easier to flip through a bunch of presets to get you as close to where you want to be, and this varies with each image, and reduce the amount of further tweaking you need to do.
And it is good to have a range of starting points to energise creativity and keep things fresh, one makes surprising discoveries by trying out alternatives that would not ordinarily make any sense.
The discussion of LUTS above is predominantly about using them in a Technical conversion, any other use of LUTS can be considered as using them in a Creative way. to further augment a base conversion, including instances where I have used a base curve instead of a technical LUT, and can then augment with a Creative LUT for artistic enhancement.
- Filmic
In the middle of all this I tried to get on with filmic, my conclusion is filmic works for certain kinds of images, and I really could not get a handle on its controls. In general if a module in darktable has lots of features, one tends to abandon it and look for something simpler. In music you have a similar parallel - complex synthesis methods like Yamahas Frequency modulation which was difficult to program using menus and sub menus vs analog style synths where all the parameters - fewer of were immediately accessible using controls - one per parameter. Too many controls and sliders means I rarely remember what does what - so technically it may be a good tool, but from a human standpoint, maybe in future revisions it will be more accessible to humans like me.
You can see the amount of personal effort I have invested in LUTS and Base curves, generating my own sets by hand, so you can imagine I must have put in similar effort to get Filmic RGB to succeed in my workflow - Well you cannot win them all - win some - lose some.
The end result of this workflow improvement with custom base curves or LUTS taking on the heavy lifting, and tailoring as close as possible to how I like my images to look, so its a bit of a personal thing - to one’s own taste. Most of the time I end up using just a few modules .
- White Balance
- Denoise (I change which one depending on the image) with practice it takes only a few seconds to achieve a suitable result.
- Crop and Rotate (if needed)
- Exposure
- Base Curve OR LUT
- Tone Curve
- Highpass for sharpening
Note that at this point in time, my priority is to have an optimised workflow that quickly gets me to a very nice presentation of the image I took in camera, so I can quickly process lots of images, without losing the creative juice or objectivity which happens when you are staring at an image for too long.
And with darktable, thankfully its easy to copy from one pic to another, if these images were taken in similar conditions, and minor changes can then be adjusted on each one. 90 % of the time once I get the 6 aforementioned modules right for an image - in about a minute or two, so easy to just copy to all other images taken in the same shooting session/subject.
Just like when working on music for a long time, same with video, our eyes tire and lose objectivity. I think a really quick workflow as defined above has been critical to sustaining my interest in photography.
I still do not know how to export and share my base curves, so that’s something I must now learn,
Will be back and post links to download sites of these, when my learning is done.
I must add as a caveat, further to all of this muscle and eye correlation with images, has made me adept at using the tone curve for so many of the things that other would use more modules for. I use this (and the exposure module in concert) for changing contrast, brightening, darkening, not just the whole image but specific areas of the dark to light range - highlights, shadows, as it has become 2nd nature to more precisely do these things in tone curve, far more precisely than I can achieve in broadstroke tools like Contrast, Brightness and Saturation. Saturation is one of the exceptiions where the Saturation slider in CBS module does not have an equivalent in the tone curve module.
Been a long journey - about 6 or 7 months, but finally pretty happy with the results. Quick, repeatable and somewhat easy when you know how.
Addendum - the over exposure /under exposure highlight toggle tools are our best friends, helps to keep us in check. still learning the gammut one cos I am guilty of violating its limits…all the time !! so I keep that one turned off for now.