Filmic: Differences in curves (3.2.1 vs 3.3.0/dev) and out-of-the-box curve in 3.3.0/dev.

I’ve noticed two things that I would like an explanation for 'cause I don’t have a clue why this would be so.

  1. Every time I turn on filmic in 3.3.0 and look at the curve I see a small orange warning part on the left. This does not happen in current 3.2.1.
    Why is this the case?
    I distinctly remember that you need to make sure that, in general, the curve is white, 'cause the curve (result) isn’t optimal otherwise.

  2. Why does filmic show 2 different curves for the same edit?
    I made a very, very basic edit in 3.2.1 and did the exact same things/settings in 3.3.0. Made a clean duplicate in 3.3.0 and loaded the sidecar from 3.2.1.
    Both images look exactly the same in 3.3.0 (there’s a very slight luminance difference, not talking about that), the filmic curve, however, looks very different:

3.3.0:
3.3.0
3.2.1 (loaded xmp in 3.3.0)
3.2.1

Image I tested this on: cats portrait
Xmp made with 3.2.1: cats.portrait.321.xmp (7.1 KB)
Xmp made with 3.3.0: cats.portrait.330.xmp (7.5 KB)

So which guide/curve is correct? Or better: Which one can I trust?

One of the reasons I spend time comparing filmic in 3.2.1 and 3.3.0 is that I’m having a hard time using filmic in 3.3.0. I do not have any issues using it in 3.2.1 and before.

I do look at the curves during my edits and try to make sure they are optimal (read: white). The thing is that in 3.3.0 I keep getting bad/unsatisfactory results. Not sure if there is a cause-effect thing going on here though, but there’s enough of an uneasy feeling to post this topic.

Are the two points I mentioned above by design (the new eye-candy being the reason, maybe)? Seems to me that what I mentioned makes using filmic harder, not easier. The fact that the defaults got changed, again, doesn’t help changing from one filmic version to the other either.

Some extra info:

  • The curve view doesn’t matter. The only one where it is really hard to see is the look+mapping(lin) one.
  • this isn’t limited to the example I posted or my 2 cameras.

Please enlighten me :slight_smile:

2 Likes

I can’t answer your questions, I guess only Aurelien Pierre can. But I can confirm that in my subjective perception filmic rgb in 3.3.0 is harder to use compared to the version before. At least if one tries to avoid orange (warning) areas of the curve. Frequently getting a curve with a negative slope for the (deep) blacks, comparable to your first screenshot, is confusing me.
The only information I could find for the new filmic rgb is from http://darktable.secretlifeofmatt.com/module-reference/processing-modules/filmic-rgb/ but this does not answer the question (or I’m missing something).
As a “workaround” I’m actually using my old presets from version 3.2.1 and the new version only for experimental purposes and to learn how I have to use it now to fit my requirements.

Good to know that I’m not the only one that thinks the 3.3.0 version is harder to use.

I already had a look at the draft of the new/to-be manual for 3.4 and, like you said, it doesn’t shed any light on the above/general issue. It does mention that the look only graph is the traditional one. This is obviously not the case.

At this point I’m actually avoiding using 3.3.0. I used to do all my Play Raw edits in the latest development version.

This isn’t solely based on filmic though! The new clipping indicator, for example, has also changed in a rather unhelpful way. I really, really like the new multiple options a lot, but it seems to be unusable for determining the blackest one can go before it “warns” you. Blacks look really ugly and crushed compared to the current 3.2.1 settings if you do. It is almost as if the lower threshold (-12.69EV) is set for a 4/8k HDPI monitor.

I do mention the clipping tool although this topic is filmic based because this also seems to be a black point related thingy.

Seems that the new default settings needs looking at.

I don’t find the new filmic any more difficult to use than with 3.2.1.

Maybe on the options tab you have “contrast in the shadows” set to “soft” rather than hard? This “soft” setting places a looser constraint on the spline, which may be resulting in this gradient reversal you are seeing?

Switching to “soft” the curve gets worse. The overshoot of the spline gets much stronger.

I agree, the default setting was completely unusual. I switched to “any RGB channel”, "-11.44 ev" and “99%” to get a behaviour which is at least similar to 3.2.1.

Switching to “soft” the curve gets worse. The overshoot of the spline gets much stronger.

yes, that is my point. By selecting “hard”, you place a stiffer constraint on the spline, which will make this sort of overshoot much less likely.

Nope, they are at their default setting.

In 3.2.1 I sometimes change one or both to soft to get the result I’m after, but in 3.3.0 using soft isn’t at all workable.

I get the impression that there isn’t anything wrong/different with the underlying filmic engine itself, but that all this is a visualization problem. It is an important difference compared to 3.2.1 though; Me, and probably a lot of others, like to have some anchor/reference points I can count on when editing and changing these between version is rather annoying.

So, what is going to drive the shape of the spline is the hard/soft constraints at either end, and the contrast and the lattitude on the look tab, and the “hardness” aka gamma power.

I’m running 3.3.0+1046, and by default I’m getting default contrast of 1.350 latitude 25%, and the hardness is auto-calculated, and this seems to give good results. If you are using presets from another version, maybe it is pushing these numbers too far?

@Matt_Maguire: Please have a look at the xmps I provided: All the settings for 3.2.1 and 3.3.0 are the same and well within the bounds of “extreme”. The only 3 things I changed from its default setting:

  1. white relative exposure (+4.13)
  2. black relative exposure (-9.12)
  3. latitude (33%)

That’s it.

I opened your image with the 3.3.0 xmp on my 3.3.0+1046 version of darktable. I can see you have contrast 1.5, latitude 33%, auto-hardness enabled, hard constraints on spline for both shadows and highlights, and the spline looks fine to me. No sign of any overshoot.
filmic1 filmic2 filmic3

I’m actually using 3.3.0+1006~ge31529f63. Even just enabling filmic rgb or resetting the module (using the default settings, no presets) shows a curve with a (small) orange region for the blacks. Any tweaks of the “relative white exposure” to lower values make it worse.
As @Jade_NL I’m also asking myself if the visualization should just be ignored at this point? Or in other words : Is the spline shown in the graph only calculated for visualization or is it used for the mapping process itself.
What is bringing me to that question is the observation that swichtching from “hard” to “soft” changes the curve markedly, but no modification is visible in the histogram.
Or have we just to update to 3.3.0+1046?

The spline curve is used to actually map the tones. If you have orange bits, it means you’ll get some negative gradients in luminance in your image – how objectionable this is depends on the image. Histograms are a fairly blunt tool, and locked to 0-100%, so it is not surprising that it would be hard to notice any appreciable difference in the histogram display with these overshoots.

Thank you for clarification.

… should be avoided in tone mappings as I learned so far. But maybe I’ve overestimated the visualiziation in filmic at this point in the past. It’s the question of finding a balance of using technical tools and trusting personal impressions looking at the image.

I don’t know, it’s just very strange that with the same xmp file, we are seeing difference results in the filmic look curve. I didn’t see any relevant commits on filmicrgb between the darktable version you are using and the one I am using, so I’m really not sure why you are seeing these orange clipping indicators at all…

Really stange. I downloaded @Jade_NL’s image and xmp files and the look curves I get are similar to yours, no orange regions, different to @Jade_NL’s screenshots. Possibly he is using an older version?
But independent from that : just applying filmic with it’s defaults leads to a (small) “warning” in the look curve. And any decrease of “relative white exposure” enlarges this region.

filmic

I observe the same. Increasing the black relative exposure a little eliminates the shadows overshoot. I’m guessing that this could be due to the auto-hardness calculation.

Nope. One of the things I do every morning is pulling and rebuilding dt dev. So I’m on the very latest version (this is darktable 3.3.0+1046~g881097ceb)

I don’t find it very assuring that we are all seeing different results.

Here’s a little video I made.
dt.3.3.0.filmic.mkv (13.4 MB)

This is dt 3.3.0.

First cat shows the little orange warning. We seem to agree on that being there.

Both the second and third example (see below) have the same starting point as the first cat: Clean sheet.

The second cat is with the dt 3.2.1 xmp applied.
The third is the cat after manually applying the settings (1.46, 4.13, 1.5 etc) to both exposure and filmic.

The images speak for themselves…

Filmic default settings, for darktable master, in display tab have changed a bit : the black level used to be 0% and is now set to 0.015%.

The reason behind is sRGB output coded on 8 bits integers can only display about 12.7 EV of dynamic range. So, any EV below -12.7 EV at the output of filmic will end up rounded (because of the float -> int conversion) to 0 at the output of the pipe, which means an infinite number of low EV could be rounded to 0, resulting in gradients loss, about which many users have complained.

But that higher black level makes it easier for the curve to undershoot, in the other hand. Worse case scenario, just set display black luminance back to 0%. Otherwise, try to reduce the contrast or change the balance, or adjust the display black level.

Here are other black clipping values:

  • sRGB (sRGB OETF) coded on 8 bits integers : anything below 0.015118% gets rounded to 0,
  • sRGB (sRGB OETF) coded on 16 bits integers : anything below 0.00006% gets rounded to 0,
  • Adobe RGB (power 2.2 OETF) coded on 8 bits integers : anything below 0.000110% gets rounded to 0,
  • Adobe RGB (power 2.2 OETF) coded on 16 bits integers : anything below 0.0000000006% gets rounded to 0,
  • any RGB (no OETF, linearly encoded) coded on 8 bits integers : anything below 0.1961% gets rounded to 0,
  • any RGB (no OETF, linearly encoded) coded on 14 bits integers (RAW photo) : anything below 0.003052% gets rounded to 0,
  • any RGB (no OETF, linearly encoded) coded on 16 bits integers : anything below 0.007630% gets rounded to 0.

So much for people who claim 16 bits encoding is overkill because [name here stupid reason].

The formula to compute the black clipping threshold for any bit depth and OETF is EOTF\left(\frac{0.5}{2^{bit \, depth} - 1}\right), where EOTF(x) = x^{2.2} for Adobe RGB or EOTF(x) = x / 12.97 for sRGB, assuming rounding is done to the nearest integer.

2 Likes

@aurelienpierre: Your post explains the (computer) science behind the changes and I’m not fighting that at all, on the contrary: I welcome it and I thank you for that info!

What your post does not explain is the visualization part. Why aren’t the visible curves also adjusted to the new, better, situation?

As I mentioned in one of my previous replies: I’m certainly not the only one that uses these curves to obtain a “sane” result (what else are all these numbers for if not to have some boundaries and anchors).

EDIT: Is the -12.7 EV you mention also the reason why the clipping is set to this number (-12.69 to be exact)?

ok, but you understand the white and black relative exposure need to be properly set for the image. You have the black point set at -9.21EV, which seems to me to be quite low for this image, and you have quite a high contrast in filmic at 1.5. I think this combination of settings is leading to this overshoot on the curve. If you want to increase the contrast on the cat, it may be better not to be so aggressive in filmic, and instead look at the local contrast module, or perhaps the contrast in the color balance module.

1 Like