RAW sharpness with darktable

Software darktable
66

Just to clarify, the several weeks of torture that led me to create custom base curves, was because the default base curve assigned was clearly not matched with my ooc jpegs.

I have read of a method that uses some open source software that is a part of the darktable distribution in linux, as well as a product with colour blocks, to measure and create a custom curve for each camera. i.e you take a picture of the color card and it compares it with some other thing, to create the curve.

The process I followed for creating based curves was painful, all done by hand, about 20 to 30 points per curve, you create teh 1st one and try to get it as close as the result of a jpeg ooc image.

Then you do the same for some other image, e.g taken with different conditions like brighter light or less light.

And then over time you create variations of these, tweaking aspects of the dark parts, mid tones, and highlights. All done by hand. No automation.

and eventually you whittle down the set to about the top 20 that you find most useful, over time you tend to settle on about 4 o 5 favorites that give you the desired result quickest. You need something to brighten mid-tones, you kinda know exactly which one does this best.

Over time during and after creating your base curves, you discover and remember which ones to use (and you could go back to rename the base curve presets - using these observed transforms)

What makes it the more difficult is that in teh normal linear scale display, especially in the darker regions, base curve points are so close to each other, they are really hard to tweak. So the secret is to use the base curve in logarithmic mode during editing.

So setting a base curve after your library of hand tweaked base curve presets is ready, is simply a case of selection. you have a hunch which one is likely to work, and you try a few to get the closest to your final intention for the image.

Typically I never tweak any of these base curve presets, cos they are like set in stone, and reused across many images, so leaving them unchanged allows me to edit an old image, and ensure that darktable can “tell” me which preset I chose for that image. If I were to do any edits to the base curve in an image, this association by darktable would be broken.

If I had to do this again, it would take me about a week or less, devotig an hour or two each day. Much faster than the several weeks it took me when all this was new to me.

I had no cl;ue it would be such a game changer in my workflow, I was simply experimenting with how to solve what I saw as an issue with darktable’s workflow.

What I also found was that when you have a diverse set of custom base curves, I also changed cameras, from a Fuji to a Sony )both mirrorless) and the base curves were still just as productive a workflow accelerator with the new camera. So its really not about getting a perfect curve with one camera, but more like getting a set of candidate curves, that would work in pretty much any camera.

Typically any finetuning of base curves is done in the tone curve module. either using one instance or several. Why? darktable allows you to turn on and off a module, and sometimes I want to try out an incremental set of possible changes to the result of the base curve. Most productive manner is to break up these curve based tweaks into multiple instances of the tone curve, which I can turn on and off individually.

I wish I could have a tone curve and an rgb curve module, that enabled me to turn off or on any points added to the curve, this way I could assess the contribution of just one or two points on this curve., without having to turn on or off the entire tone curve module or rgb curve module.

In the same way that LUTS have really taken off, and Presets in Lightroom have also become quite a thing. I’d like to hope that in the near future we can have collections of Custom made Base curves, which we could publish as compilations and share with others to improve their workflow and accelerate this aspect of the workflow in darktable, I’d be happy to share a few of mine - as much as I feel quite precious about them, in the spirit of supporting the superb work of the darktable developers. One current challenge with darktable presets is they come in a long list., so it may be unwieldy to have too many, which is why I limit mine to about no more than 20 custom base curve presets. Maybe based on subdirectories, we could have darktable modified so that the presets are also grouped into sub categories, based on the directory structure in which they were “imported”, to avoid cluttering up the preset menu into one long list.

3 Likes
67

I’m sure many would appreciate them. I too am rather surprised custom curves haven’t become a thing, as they avoid the interpolation of luts.

It sounds like you are using the display referred workflow. Have you tried scene referred? Most here will use filmic, which comes with its own curve. Recently, I have taken to using filmic only as a tone mapper to control Black and White points, whilst making the curve straight. This allows one to use a custom curve in another module. I’ve been doing that either with either tone curve or tone equaliser. So is there any reason you chose the base curve module over those?

I also find it useful to Compare my custom curve to some other Presets, as there is nearly always a preset that does it better - but rarely the same preset each time. The Presets I’ve been using are the film emulation luts from rawtherapee, which are downloadable from rawpedia. That folder contains hundreds of luts. I’ve narrowed it down to about 30 favourites which produce the goods regularly. With more use I hope to narrow it down to 10 or less. But I wonder how easy it would be to extract curves from these luts and save as curve Presets?

This is a particular topic of interest for me at the moment (albeit off topic from original discussion).

68

Unless I am very much mistaken, unless you have a strictly parametric (mathematically well-defined) curve, interpolation happens regardless. The basecurve in darktable is not parametric afaik.

In any case, we’ve drifted off topic quite a bit here. Also, I notice that I’m getting increasingly irritated when I see people vehemently wanting to replicate their in camera JPEGs. If you like them all that much for how they look, why jump through hoops in editing the raw yourself? :man_shrugging:
Anyway I’ll try to keep my personal issues to myself… :slightly_smiling_face:

2 Likes
69

TBH I do not get it, why. I must admit, I did not check all vids from @s7habo, but I saw some and what I remember does not explain to me the contr-EQ stacking.

I downloaded the xmp from @anon41087856 and in a duplicate I swtiched off two of the contr EQ and increased the first one accordingly, to do it all in one. Is it my eyes? I did not see that much difference. Too subtle IMO to spend the energy (or do I still miss something?)

1 Like
70

You know, I asked that question a few times, and never got an answer…

1 Like
71

Suggest we create a new thread to discuss this further here :slight_smile:

To avoid derailing the current thread

Link to new thread below

Darktable - Workflow approaches - Custom Base curves, LUTS, Filmic

72

But the answer is quite simple. People like some aspects of their OOC jpegs, but they don’t want to lose all the possibilities of raw, like for example being able to recover shadows/highlights or tweak WB - basically to fix some occasional mistakes or difficult conditions.
If you like how OOC jpegs generally look like, but don’t want to lose all the possibilities of raw - I feel like it’s a very reasonable request to know how to make your raw generally look like OOC jpegs, maybe just as a good starting point for editing or just for fixing problematic things (e.g. shadows).

3 Likes
73

Regardless the motivation, I am very happy about all what I have learned here

74

replying to my self…
What I did is:

  • set one contrast euqlizer
  • snapshot
  • attenuate mix down to 0.333
  • duplicated times
  • compared to snapshot

There is an effect visible. Donno why, but it is like this…

75

That presupposes that users pick their camera (at least in part) for the jpg rendering. I know I didn’t, and I don’t like the Sony basecurve at all (look at what it does to the highlights).

And I think it’s quite unreasonable to expect any raw development program to be able to produce a look identical to the OOC jpegs of brand X, which is usually what is requested/complained about (after all, that’s what “replicate” means). One of the problems there is the multitude of scene settings modern cameras have (be they “smart” or not"), and even the possible settings for the “standard” renderning…

If you like the general look of OOC jpegs, you can use the basecurve approach, and pick a basecurve that you like, no need to stick with the curve corresponding to your camera brand. But that will not give you jpegs corresponding to the OOC rendering.

76

Perhaps the difference is just that x × 1.3333^3 > 2 x (it’s ~2.37 x )

77

Because in OOC jpg-s have a great color grade and look but you still have to pull out some shadows/highlights, correct exposure, retouch some parts of the image, correct perspective, crop, lens correct for third party lenses. dehaze or do something else etc.
The deal with the ooc jpeg is that it has a great contrast and colors if only it lacks a bit of saturation sometimes. But there are many other corrections that need to be done or creative editing if you want to do that.

1 Like
78

My feeling is that people view the OOC JPEG as a starting baseline. Put a little differently, if I can’t do better than my camera settings then why bother with raw, or what am I doing wrong?

1 Like
79

Interesting write-up. I came across this advice from @anon41087856 from at thread at Tactica.com easy ways to sharpen skin with darktable (hopefully I haven’t broken any cross-posting rules), where the sharpen module is actually exactly an highpass filter blended in overlay mode, and furthermore raising the right most node in the contrast equalizer falls back to a highpass filter.

So, it would seem that you could do everything and more in the contrast equalizer - plus new insights into stacking CE instances - if you can predict the results from the settings. For me, it might help if I had a better understanding of what’s going on under the hood of the CE module and how it relates to the others

1 Like
80

Interesting. Would you believe I have never heard of or used the Contrast Equaliser module in darktable

Did not know it existed. Using the highlight module has become so 2nd nature, and really quick, and also easy to tweak to taste, I suspect even after I have studied other candidate modules that can achieve sharpness, I most likely will continue using the highpass. cos its so quick and so easily tailored, using the opacity I can quickly reduce its effect. Now that is one gem in darktable, pretty much all modules have the opacity, so when you;ve gone overboard, so easy to reign it back, without changing the sliders in the module itself.

This is probably the piece de resistance of darktable, and I cant find this in any alternative tools - masking and Opacity in almost all modules. kinda like having Lightroom and a bit of Photoshop in the same image editor. un beatable.

1 Like
81

Dark table chart is pretty easy and it gives you a tone curve and an LUT in a style either to your matched jpg or your color card specs so color accuracy vs jpg match…https://www.youtube.com/watch?v=LufwQZx01gk https://www.youtube.com/watch?v=11nInNWJHWk&feature=share Harry uses a trick and applies the default basecurve before running darktable chart and so he claims this give a better match…he is rather meticulous and these are old videos but essentially you can both create your own basecurve and profile using DT chart to get a tone curve/LUT combo…PIXLS.US - Profiling a camera with darktable-chart

1 Like
82

I think it’s like a lot of things, where you learn an approach and then you optimize it for your use and it becomes second nature. The contrast equalizer may be worth checking out due to its versatility. If you look at the presets you’ll find effects for clarity, sharpening, and de-noise as well as some effects you wouldn’t expect like bloom.

From my perspective, there are a few tools that do similar and even equivalent effects, so I run the risk of over doing something by using multiple modules for the same purpose. It’s like baking… add some sugar in this step, a little more the next, and then more again, and pretty soon the cake is just cloying. At least, that’s my experience.

83

I did check out the contrast equaliser. Definitely worth the effort, even thought it takes a few more clicks. Very refined control of contrast, rather than the broad strokes of the highpass.

Like a Rolls Royce, all bells and whistles of contrast management.

I am reminded of the different levels of photographs and their purpose, in the good old days when people like me did not own a camera, and most people I knew did not (was born and lived in West Africa where only the really well heeled had these things), you either got a photographer to take a Polaroid pic, instant, or went to a proper studio paid some more money and had to wait at least a week for the results. Typically you went in a group of friends, actually got dressed up and posed and took individual snaps as well as group photos - it was an event in those days.

Darktable modules remind me of the effort required for different photo targets, some are really quick broad strokes, and others are quite involved for very fine control.

Nice to have options. Thanks for introducing me to the contrast EQ. Appreciated.

If there was one wish, I wish that the more complex controls in darktable had a way to turn off each section, cos toggling off/on an entire module does not give you an easy way to evaluate the contribution of each section within the module.

Even simple controls with only three sliders like Contrast, Brightness and Saturation, makes me wish I could turn off and on each of these individually, while keeping whatever values I had set unchanged in each slider. The current work around is to set up three instances, and in each of these I tweak only Contrast, only Brightness, and only Saturation.

But seriously - as darktable is open source, maybe one day, I’ll dive in and create a version with such enhancements, for personal use, who knows maybe… That would be such a cool thing to achieve…

1 Like
84

Have you tried using the snapshot feature? On the left panel you click “take snapshot” and it’ll copy your photo at that point in time. Then you can make adjustments and click on the saved screenshot to compare before and after. Plus, you can take several snapshots to capture different point in edit

I think many of us don’t realize how lucky we are and take things for granted. It’s good to have a reminder every now and then.

1 Like
85

Thank you for suggesting the snapshot feature.

I did use the snapshot feature extensively for a while, when learning darktable, great feature.

But now I prefer to either :

  1. Create a duplicate, which is easy in darkroom, and make changes in the duplicate, then compare them side by side with the original, in lighttable - click the images you wish to compare and use key X to enable side by side comparison.

This way I can also compare more than two versions at the same time, and immediately go to edit further the version I prefer.

OR

  1. The other benefit of the duplicate approach is that also as a quick comparison method, while editing version A, quickly view version B, by holding my mouse on version B in the darkroom duplicate management window on the left of the screen and momentarily the image displayed will be version B, until I release the mouse click. Advantage is if i do things like zoom in, the view of any other duplicates also follows my current zoom in the current edit, unlike snapshots which are a static snap in time… Furthermore when you have toggles like over/under exposure enabled in darkroom, the snapshot includes these highlighted areas in the snapshot.

Both of the aforementioned approaches I now use, instead of the snapshot, as described above, allow me to compare edits, and have the image in all versions compliant with the current zoom.

3 Likes