Introducing the filmic module in darktable

[anon41087856]

I know, I promise it will be on for the realease.

[agriggio (Alberto)]

very interesting @anon41087856! I’ll definitely try this out asap. I have been experimenting with log encoding and scene-referred editing inside RT, and I can definitely see some benefits. it’s hard to quantify (for me at least), but it seems to produce smoother tonal variations, that are more pleasing to the eye. the difference is not dramatic, but it is there, especially on portrait shots (probably because we can more easily perceive small differences in skin tones?). thanks for the informative posts! (and thanks also to the other people involved!)

[Elle (Elle Stone)]

@agriggio

I have been experimenting with log encoding and scene-referred editing inside RT

@anon41087856 - I followed the tutorial in this thread and tried to follow the logic:

Solving dynamic range problems in a linear way

Ok, please fetch the last version and follow this tutorial: open an image and ensure, in the exposure module, that you have no clipped highlights or shadows. Don’t hesitate to push down the black level and the exposure level [Capture%20du%202018-09-21%2018-00-01] in the unbreak color profile, click on the “middle grey target value” color picker, then draw a rectangle over an area that is supposed to be 50 grey (or just choose the whole image to get its average luminance as a 50 grey refe…

It looks like the code linearizes “something” which I’m guessing is the LAB TRC, because the camera input profile is already linear.

1. Is above correct? If not, what is linearized?

Then the code applies a log2 curve that takes parameters from the image, modified if the user wants by picking middle gray and etc. At this point it seems to me that the image is no longer encoded linearly. Edit: this question is meant rhetorically, in case anyone wonders.

2. Is above correct? Is the image no longer encoded linearly?

The next step is to apply a tone curve, which happens in LAB space, but using automatic RGB to restore the otherwise lost saturation.

3. Is the tone curve itself applied in LAB space even though the saturation is added back in using “automatic RGB”?

4. And what is “automatic RGB”?

Question 2 above is meant rhetorically. This new darktable algorithm linearizes “something” (that’s question 1), applies a log2 (or gamma, user choice) curve, and then modifies the result in LAB space, but add saturation using RGB. The result is hardly “scene-referred”. The result also has hue shifts all over the place.

Whether the new darktable algorithms produce “pretty results” or an easier, quicker way to get to “pretty results” is a matter of aesthetic opinion, and personally I don’t care whether an operation is done on linear RGB or otherwise as long as the user is happy (and preferably also has a clue regarding what they are doing to their RGB data).

@gez - several times you’ve expressed a rather intense dislike of editing nonlinear RGB, often in connection with doing so using GIMP. Is there something about the darktable “unbreak profile” algorithm that makes it OK, acceptable, etc to operate on nonlinear RGB?

[gadolf (Gustavo Adolfo)]

Sorry about this probably stupid question, but is this the right version?

[nosle]

I’ve been testing it a bit and found that, for instance, the contrast slider creates sudden large changes between 0.01 values whereas for the most part there are smooth smaller changes. See below screenshot for instance. In the below example there is a sudden loss of contrast when increasing the value of the contrast slider.

Edit: the change doesn’t look all that dramatic on pixls.us but working on them the step is huge and in an unexpected direction.

[Elle (Elle Stone)]

@anon41087856

Following up on the matter of making darktable’s editing workflow more efficient, and images look better, my last module was merged yesterday into master. It is based on Troy Sobotska’s plugin for Blender, and on my previous work on the logarithmic tone-mapping all boundled at once. Is is intended to fit with my other change of the color balance module . These two can replace a lot of silly things inside darktable and don’t mess-up the colors as most Lab module do, or the whole pipe as the base curve do (please, folks, stop using that nonsense).

I put up some sample metrics for “how messed up” are colors from a sample base curve compared to your “unbreak profile” and “filmic” modules here:

Messed up colors by the number - darktable's new unbreak and filmic modules

I’m assuming that the colors in a scene-referred interpolated raw file (no “make it pretty” algorithms) are as close to the actual scene colors as most of us can get given that when we take pictures we don’t also measure scene colors using specialized equipment. If the colors in the scene-referred interpolated image file are taken as the “not messed up” colors, then the degree that post-processing produces “messed up colors” can be quantified given suitable metrics. I used two metrics: How …

[gadolf (Gustavo Adolfo)]

@nosle I confirm that, same here.

Also, there are times when I try to apply automatic log TM, I have to click it twice, the first time nothing happens.

[billznn (Massimo Bill)]

@nosle

Edit: the change doesn’t look all that dramatic on pixls.us but working on them the step is huge and in an unexpected direction.

It doesn’t look dramatic on pixls.us, but the histogram seems to tell it’s a bit dramatic

[gadolf (Gustavo Adolfo)]

Shouldn’t color balance come after filmic in the pixelpipe? What led me to think that was the filmic suggested workflow.

[anon41087856]

@gadolf

Shouldn’t color balance come after filmic in the pixelpipe? What led me to think that was the filmic suggested workflow

Colors are, in general, better edited in linear space, hence why they come first in the pipe. However, as lightness adjustements change the colors and it’s super difficult to see colors properly on dark images, it’s better to edit them after.

@nosle

I’ve been testing it a bit and found that, for instance, the contrast slider creates sudden large changes between 0.01 values whereas for the most part there are smooth smaller changes. See below screenshot for instance. In the below example there is a sudden loss of contrast when increasing the value of the contrast slider.

That should be fixed now, or at least you get a view of the curve to understand how your parameters affect the result.

[phweyland]

The first try is promissing. Here are some remarks.

1. I experience some discontinuities as @nosle. Example (I can share the raw file if needed) :
   Latitude = 4,70 => Latitude = 4,71
   

   
   

2. When I change the middle grey from 4 to 8%, I expect to get back the original image decreasing black and white point by 1. It is not the case. Why ?

3. What are the neutral values for S-curve ? I would find easier to understand what it does knowing where it starts from.

4. For manual tweaking I think the view of the curve should show only the S-Curve (not including the log part). It would be a straight line at the beginning.
   Materializing the linear part (and fulcrum ?) on the curve would also help controling what we do (especially on limits).

[anon41087856]

Hi everyone,

I have written an extensive doc for that module, emptying my brain after weeks of tests, so I’m happy to RTFM you : https://hackmd.io/BoyDhxRwQFq3z3H2d6wjqA?view This is a collaborative work I do with @asn, so feel free to correct my bad English (that’s my third language, I do my best).

@phweyland

I experience some discontinuities as @nosle. Example (I can share the raw file if needed) :
Latitude = 4,70 => Latitude = 4,71

RTFM

@phweyland

When I change the middle grey from 4 to 8%, I expect to get back the original image decreasing black and white point by 1. It is not the case. Why ?

Because they are not related. That middle grey is a pre-amplification before the log, values below the middle grey values are just less “log-ified” than values above. We discussed that before on the unbreak color profile, it’s the same.

@phweyland

What are the neutral values for S-curve ? I would find easier to understand what it does knowing where it starts from.

Neutral values for the S curve would be an 2-based exponential function (the inverse of the log) with an y offset equals to ((black exposure) / (dynamic range)) / (output grey)^(1 / output power factor). That’s pretty much the default parameters, except you should set the contrast at 1, but that produces weird things sometimes.

@phweyland

For manual tweaking I think the view of the curve should show only the S-Curve (not including the log part). It would be a straight line at the beginning.
Materializing the linear part (and fulcrum ?) on the curve would also help controling what we do (especially on limits).

You only get a pure S curve if white level = - black level (that is, grey = 50 %). Otherwise, you get an S curve convolved with the exponential function mentionned above. I could set a log scale for the graph, as I did in the tonecurve module, but I’m pretty sure it would be misleading more than helpful.

[phweyland]

@anon41087856

I have written an extensive doc for that module,

Really great and helpful ! Thanks so much !

From TFM : some settings put nodes in bad places
Would it be possible & useful to show the nodes to the curve ?
EDIT: Same question for middle grey …

[gadolf (Gustavo Adolfo)]

Thanks for the whole work!
I’m not sure if this is the right place (if not, author and/or mods, just let me know), but I’m exercising the module with this image from a recent play raw, and I would like to know if the workflow, settings and final result are ok. (trying to be as close to the scene as possible)
1 - WB
2 - filmic
3 - exposure
4 - color balance (local adjustment on the tail of the storm, to stress it’s rainy nature)

080724-1822-100-1479_01.cr2.xmp (7.3 KB)

[Claes]

Hi rei,

According to Pierre [New module : Filmic by aurelienpierre · Pull Request #1811 · darktable-org/darktable · GitHub],
the recommended flow is

• exposure
• white balance
• color profiles / LUT
• filmic (tones)
• color balance (colors)
• denoising and enhancements as you please.

Have fun!
Claes em Lund, Suécia

[gadolf (Gustavo Adolfo)]

Hey, Claes!
I’m aware of that work flow. However, by reading the latest words on the module, it seems that you may want to tweak exposure after filmic:

> This module should be enabled after the white balance has been adjusted, in order to balance the RGB ratios. Then, if needed, the exposure module should be used to correct the mid-tones in a linear way.

[Claes]

Hm, it is not easy to follow the rapid development of the module.
Here is the latest module description I have found:

Filmic module for darktable - HackMD

[gadolf (Gustavo Adolfo)]

Exactly, the quote I did before is from that link.

[yteaot]

I apologize in advance for ignorance. But I would love to use this module. Is it possible and how is it happening? I can not install anything from git without guidance. So can someone kindly educate me.

[darix]

If you are on linux you are a lucky person and can use Install package graphics:darktable:master / darktable