filmic v4 on the way

[priort (Todd Prior)]

Thanks for that

[GuK]

Sorry, dumb question: is there a way to get a Windows exe with the new filmic for those who are not into programming? I’m a bit lost at Github…

[MStraeten (MartinSt)]

just a test build with a couple of further stuff - darktable-3.1.0+2359~g8332dedfc-win64.exe: File on MEGA
a backup of your darktable database is highly recommended …

[Egocentrix]

@MStraeten

a backup of your darktable database is highly recommended …

I would make that stronger: Using a separate .config dir and library for a dev build is highly recommended.

[GuK]

Thanks!

[paperdigits (Mica)]

I’d copy any raw files to a separate location, or turn off xmp writing in the Dev version. Xmps are not backward compatible. Nor is the database.

[GuK]

Too bad - application error, the application could not be started (0xc0000142).
(Win10,64bit)

[MStraeten (MartinSt)]

try to start it from cmd.exe
darktable.exe -d all
maybe it then gives some indications what might be missing.

[Carmelo_DrRaw]

I am trying to understand the details of the tone mapping formulas in filmic v4, to see if I can get some “inspiration”, and I have some observations that would like to get confirmed by the experts.

First of all I would like to understand the behavior of the tone mapping curve alone. I might come back to the saturation and hue control in another post…

Let’s put aside the highlights reconstruction, and consider the simplest version with channel-by-channel tone mapping (filmic_split_v2). For simplicity, let’s initially see what happens to the mid-grey point, step-by-step.

The black and white points are defined in EV values with respect to mid-grey. That is, a “white relative exposure” of +5EV means white = 32 * mid-grey. Right?

For a symmetric dynamic range (black_exposure = -white_exposure), the log conversion maps the linear values in the following way:

• log(black) = 0
• log(mid-grey) = 0.5
• log(white) = 1

Next, the spline function maps log values back to the linear domain: out = spline(log(in)).
We have:

• spline(log(black)) = 0
• spline(log(mid-grey)) = pow(mid-grey, 1/hardness)
• spline(log(white)) = 1

That is, the black and white points are mapped to 0 and 1 respectively, and the mid-grey point is mapped to mid-grey raised to the 1/hardness power. Am I right?

Here is a typical example of tone-mapping curve wit a slight contrast and a symmetric dynamic range:

The plot in the UI shows the log-to-linear mapping of the spline curve, but does not include the final power function, controlled by the “hardness” parameter.

I find it interesting to look at the linear-in -> linear-out curve as well, that is the result of the full chain out = pow(spline(log(in)), hardness), so I modified the UI to plot this:

Here is the same, but with the horizontal/vertical scales going from 0 to 0.25 to zoom on the region around and below mid-grey. The mid-grey point is also marked in the plot:

One can see that the tone-mapping curve is steeper than the diagonal line below mid-grey, and is flatter than the diagonal at the mid-grey point. That is, the contrast in the mid-tones is lowered, even if the contrast parameter is > 1. I have played a bit with the various parameters, and as far as I can see this is always the case, except for very high values of the contrast… is this intentional?

I was also trying to achieve a “neutral” rendering in which the shadows and mid-tones are left unchanged and only the highlights are compressed. This would correspond to a curve that lies on the diagonal up to the mid-grey point (or higher), and then bends down to compress the highlights. However, I have not found any combination of parameters that behaves like this.
Setting both the contrast and the hardness to 1 (that is, a flat spline curve that maps mid-grey to mid-grey without an additional power function at the end), I obtain instead a curve that severely clips the dark end:

Is there a way to achieve “neutral shadows” with filmic V4?

@anon41087856 I hope you find the time to read this rather long post…

Thanks!

[anon41087856]

@Carmelo_DrRaw

The black and white points are defined in EV values with respect to mid-grey. That is, a “white relative exposure” of +5EV means white = 32 * mid-grey . Right?

Yes.

@Carmelo_DrRaw

Next, the spline function maps log values back to the linear domain: out = spline(log(in)).
We have:

The spline maps from log to “gamma”, since you see log(mid-grey) -> mid-grey^(1 / hardness), and this gets later mapped to linear through x^hardness. It’s the same things as the log2/log10 film sensitometry curves, conceptually.

@Carmelo_DrRaw

One can see that the tone-mapping curve is steeper than the diagonal line below mid-grey, and is flatter than the diagonal at the mid-grey point. That is, the contrast in the mid-tones is lowered , even if the contrast parameter is > 1.

Yes, this comes from the filmic curve imposing a first-order derivative = 0.0 at 0%. You can relax that constraint by setting “shadows contrast” to “soft” in the options tab. “Hard” uses a 4th order polynomial constraining it to be tangent to the horizontal, while “soft” uses a 3rd order polynomial that only constrains the lerping conditions with the latitude.

[Carmelo_DrRaw]

@anon41087856

Yes, this comes from the filmic curve imposing a first-order derivative = 0.0 at 0%. You can relax that constraint by setting “shadows contrast” to “soft” in the options tab. “Hard” uses a 4th order polynomial constraining it to be tangent to the horizontal, while “soft” uses a 3rd order polynomial that only constrains the lerping conditions with the latitude.

Thanks! However, I see a similar behavior when using the soft option:

[anon41087856]

@Carmelo_DrRaw

Setting both the contrast and the hardness to 1 (that is, a flat spline curve that maps mid-grey to mid-grey without an additional power function at the end), I obtain instead a curve that severely clips the dark end:

@Carmelo_DrRaw

Thanks! However, I see a similar behavior when using the soft option:

You have the log transfer function at the beginning. If you set hardness & contrast == 1, you are asking the interpolation to pass through (0, 0), (0.5, 0.18) and (1, 1) while ensuring continuity of the the first order derivative around the latitude. My guess is that puts too much “tension” on the spline and produces overshoot/undershoot. Neutral blacks + highlights compression have never been a design goal for filmic, so they would need a different spline.

[Carmelo_DrRaw]

@anon41087856

Neutral blacks + highlights compression have never been a design goal for filmic, so they would need a different spline.

Ok, understood.

@anon41087856

Yes, this comes from the filmic curve imposing a first-order derivative = 0.0 at 0%. You can relax that constraint by setting “shadows contrast” to “soft” in the options tab. “Hard” uses a 4th order polynomial constraining it to be tangent to the horizontal, while “soft” uses a 3rd order polynomial that only constrains the lerping conditions with the latitude.

Regarding the slope of the tone mapping curve around the mid-grey point, I have tried to set the shadows contrast to “soft”, but I see no difference in the mid-tones:

Enabling the auto adjustment of the hardness gives a smooth transition to zero, but the tone curve flattens well before the mid-grey point…

I am insisting on this point because I have the idea that a contrast boost should mainly affect the mid-tones… not a criticism, just trying to understand

[anon41087856]

@Carmelo_DrRaw

Regarding the slope of the tone mapping curve around the mid-grey point, I have tried to set the shadows contrast to “soft”, but I see no difference in the mid-tones:

Yes, this is because the latitude is a straight line no matter what. You should read the details of the S curve here : Engineering | Filmic, darktable and the quest of the HDR tone mapping

@Carmelo_DrRaw

Enabling the auto adjustment of the hardness gives a smooth transition to zero, but the tone curve flattens well before the mid-grey point…

That depends on your latitude width, I think. But I don’t really understand your graph. It should pass through (1,1) if you applied the output power on it, why does it not ?

@Carmelo_DrRaw

I am insisting on this point because I have the idea that a contrast boost should mainly affect the mid-tones… not a criticism, just trying to understand

Contrast is the slope of the linear central part (midtones indeed), so it also affects extreme values because of the continuity of the gradient, in the way that… well, whatever the slope is, it should smoothly end up at (0,0) and (1, 1).

[Carmelo_DrRaw]

@anon41087856

But I don’t really understand your graph. It should pass through (1,1) if you applied the output power on it, why does it not ?

No, no… that’s the whole tone mapping curve, from linear_in to linear_out, so out=1 is mapped to in=white. Hence the curve passes through (1, white)…

[GuK]

@MStraeten

darktable.exe -d all
maybe it then gives some indications what might be missing.

Didn’t help, but there’s a new parallel thread with nightly windows builds. That worked.
Thanks anyway!

[GuK]

@anon41087856

Setting it to scene-referred will auto-apply filmic and exposure module altogether.

First off, a big thanks for the improved filmic!

When I select “scene-referred” like in your screenshot (with disabled base curve) nothing is applied on opening. I use the latest nightly from the parallel Windows thread.

[elstoc (Chris E)]

@GuK

When I select “scene-referred” like in your screenshot (with disabled base curve) nothing is applied on opening.

Please note that the changed setting will not take effect until you restart darktable. Also, note that the scene-referred workflow defaults are only applied to new edits and when you discard the history stack on an existing edit.

[GuK]

Discarding the history indeed works, opening an unedited pic not. But that’s a great workaround - thanks!

[elstoc (Chris E)]

To be clear when I say ‘new edit’ I mean an image that has never been opened in the darkroom. As soon as you open an image in the darkroom, darktable auto-applies some settings and this is considered part of the ‘edit’.