Proposal for updated Filmic curve parameterisation in Darktable

Right! I promised more maths, here is more maths:
the_maths.pdf (250.6 KB)
This is 8 pages of rather theoretical stuff but just in case anyone wanted to see how the curves are derived. This is not written to be very easy to process because this actually took me about as much time as solving the equations and making the Desmos mock-up. This also means there might be typos in it. The equations should be correct though, because they’re the same ones as used in Desmos.
@anon41087856: I’m sending you the TeX file, too (can’t upload that here).

Not covered: What’s the most elegant way of constraining the variables which need to be constrained?
This is relevant because the linear segment needs to be prevented from extending to 100% (99.999… is fine, but not 100) or to 0 (same thing), and contrast also has a lower limit. I have ideas for that, too, but that’s for another post, coming soon…
Next up: which parameters to expose, and how …

7 Likes

Okay, so… I don’t get it. The same settings every time? How does that work if exposure changes, or the light, or the dynamic range of the scene? I suppose you also don’t give yourself a lot of time per photo, which means you are probably not trying to make each one of them look as good as possible, but “good enough” in fairly little time.

But even then: There are high-key, low-key, in-between key photos, photos where the sky is blown out and you need to keep it blown-out or else it’ll look stupid, photos where it’s nicely exposed to the right and you want to preserve those fluffy white clouds, and photos where the fluffy white clouds will have to be squeezed a bit harder to make space in the histogram for the child under the tree … how can that all work with a single preset?

2 Likes

the setting i set is to get as near to what the camera output if you shot it underexpose it will give you under expose image. if you over expose it will give you over expose. you can choose the one you want to edit and do what ever edit you want to it. at least with this setting the lighttable preview before any adjustment are add will look very similar to when the image is first open in darkroom. Thats all this does. this might not work for you.

1 Like

There are long discussions about how complex filmic is to use and should be simplified - and then there’s a simplified workflow using the capabilities of the module by having a default being good enough in most cases —> even worse :wink:
A preset is a preset - and in difference to a simplified tool where stuff is baked in it can be tweaked if necessary.

Presets are not defaults. Tweaking presets (or actually any parameter) in a well controlled (orthogonal) fashion is a design goal. If not, LUTs should be recommended instead of filmic. Reducing complexity by reducing non-orthogonal behaviour is improving consistency while maintaining control, without giving up the preset-approach.

a preset can be defined to be a default in dependence of exif data. It’s up to the user and his experience. If there’s a setting “good enough” in given conditions it’s far more usable then a perfect theoretical curve that doesn’t exist in real world conditions.

…and you’ve just explained exactly what the problem is: Somebody posts an image and explains what problems they had, and a bunch of people who don’t have those problems are happy to jump in telling them that what the look they want to achieve is stupid, the way they’re trying to do it is wrong and they’re also bad photographers. Because you’re convinced of this before you’ve even seen the issue explained.

This is the same problem which lots of support forums suffer from: Someone asks about something unspecific, and people tell them to be more specific. Somebody asks how to do something specific and gets told that this is a stupid idea and if they’re doing that, they’ve already made a mistake and proven to be unworthy of any further advice.

This sort of thing becomes more likely when the topic includes to artistic decisions or matters of taste, and different people using the same tool for very different ends (do you measure your speed in photos per minute, or hours per photo?). Especially around filmic, especially because so many people struggle to get it to work comfortably (and people-in-the-know get annoyed with the large number of stupid questions) this risk is elevated.

That’s not to say you had no right to be annoyed about an issue which annoys you but which you don’t manage to understand, of course. I get that frustration. But what you just said doesn’t really encourage people to come forward, either, because whoever posts anything from now on will first have to defend themselves as artists and craftspeople before their question will even be taken seriously.

Any piece of software and any developer has to fight with RTFM-type questions. But if there’s a significantly higher number of those with regard to one feature, it stands to reason there is a systematic cause – independent of whether the developer or the users understand what that cause is.

10 Likes

wow, there’s another thing I didn’t know.
Combined with the fact that the scene-referred process needs to be activted in the settings, I would suspect that there are few users who don’t even notice they’re in display-referred mode and try to use filmic, or people using display-referred modules when they should not.

Which is to say: I do know that you have to switch to scene-referred, but I’ve accidentally worked in display-referred mode a few times when the setting reverted for some reason and I didn’t pay attention. And I had not noticed the tooltip, so I kept checking the advice in the manual about which modules are recommended and which are not, which is of course a lot more tedious than just seeing a direct indication in the GUI. Would it maybe make sense to put the scene-referred modules in a separate group? And/or to have an easily-visible switch between both modes which also changes the way in which modules are shown? You probably wouldn’t prevent people from using the “wrong” modules, but if they’re presented in a way that noticeably emphasizes the “good” choices (through grouping, sorting, highlighting, adding little coloured “scene-referred”/“display-referred” icons which go grey when they’re not recommended for the current mode …), that might prevent a lot of accidental missteps.

1 Like

Where are the images that people have posted? I’d like to see them.

I wouldn’t “so many people.” The vast majority of dark table users are people who we’ll never hear from, they’ll never participate.

It seems that we are still generalizing though, and we would sure like to see some images, videos of usage, etc etc to understand where the problem is so that we can address it. I make this last statement specifically as one of the maintainers of the documentation.

3 Likes

You can already do this with the module groups setting. There are pre defined groups for both display and scene referred. You can switch between them and define your own groups.

You can see the manual here: https://www.darktable.org/usermanual/en/module-reference/utility-modules/darkroom/manage-module-layouts/

2 Likes

Ah okay, makes sense. Thank you. If the aim is to replace the camera base curve, you “only” need to make sure that the dynamic range corresponds to what the camera does and the mid-tone contrast is similar to the jpegs produced by the camera. And the benefit over the regular base curve is that you can then tweak it in filmic, and you can use the other scene-referred tools.

2 Likes

To clarify: I was not saying that this was bad in any way, I just wondered how a single preset can fit all purposes. But I misunderstood what the purpose was.

I also do not think that filmic should have more stuff baked in (not sure if you thought I did) – I just think that the levers could be connected in different ways to make the inputs and their effects on the image more obvious, distinct and orthogonal. I.e. make it possible to tweak one thing without having to readjust another (as much).

4 Likes

Thanks!
That helps me, certainly.

…but the remark was meant more about UI defaults/changes which communicate to new users who’ve never used DT before, that:
a: there is a scene-referred workflow, and a display-referred one
b: that some types modules are specific to scene-referred or display-referred workflow
c: which ones belong in which group
Anyone who doesn’t understand what “scene-referred” etc means, can then go and look it up, but especially if the information is so close to the user anyway, I think there should be a way of making it directly visible without getting in anyone’s way.

I’d expect new users to spend some time with the documentation, some videos, or tutorials. Then they can hover over the module and the tool-tip will tell them what the module is good for.

If some modules are for scene-referred and others are for display-referred, perhaps they could be colour-coded.

Vague questions from users – well, that’s life.

One of my favourite user questions was nearly forty years ago. “I can’t get the floppy disk out of my PC,” she said on the phone.

In those days, floppy disks were about 5 inches square, thin flexible disks in stiff black envelopes with a window on each side for the read/write heads.

So I went to her office and sure enough, the floppy was stuck. I levered it out with the top of a pen. The disk was in backwards. I explained this.

She said, “But I don’t see how I could get it in, the other way round.”

I asked her to show me how she normally put it in. She held the disk carefully between thumb and finger, with the thumb over one read/write window, and the finger over the other.

I collapsed in laughter, which didn’t win me any brownie points.

“But those holes are exactly right for my finger and thumb,” she said.

And she was right, they were. She knew nothing about how computers worked, and had not been properly trained, so she used her intuition.

4 Likes

In a regular workflow, both scene and display referred modules are useful. The pipeline manages the order of operations. The trouble is users can change this order and duplicate modules. Some depend on others. Others do double duty (I think filmic can do both).

Colour coding will probably make it more confusing. The real solution is to denote dependencies, out of order modules and mark duplicates (i.e., alert the user that a scene-referred solution is available [e.g. in basecurve, indicate that filmic may be a better alternative] and also warn the user if both are activated [which should not be the case 95% of the time].)

The thing about FLOSS is that it gives you choice but with choice comes responsibility. :nerd_face: PS I have been playing a tactical RPG with help overlays, stats menus, symbols, etc. I wonder if our raw processors will become like that… :stuck_out_tongue_winking_eye:

1 Like

yes, you’re right of course. You can use pretty much any module in either case, and you can absolutely misuse them. Which I actually value in a tool. There’s a lot of great art that was made because someone ignored the rules, broke them by accident or just didn’t know about anything and just went and out and invented punk…

However: Would it not make sense to have some sort of scheme applied by default: Either you choose the scene-referred way, which deactivates the base curve and activates filmic etc., and favours the scene-referred tools (because anything you can do scene-referred should turn out better than the display-referred equivalent), but of course still lets you apply display-referred effects, or you choose the display-referred way, and have the scene-referred stuff moved out of view.

Making that choice explicit (not signals to a new viewer that there is a choice to make. And if they wonder what that means, they can go and figure it out. If they don’t notice that the choice exists, and don’t notice that some modules need it this way and others don’t – they have a high chance of just wondering why this thing is so complicated, why half the tools seem to produce weird results (as I will frankly admit I did the first time I came across filmic), … and either find some helpful information outside of DT (as I did here) or leave again.

Simply having a little something in the module titles could go some way to making even complete newcomers conscious of these aspects, and tell them where to look for answers.

2 Likes

Love it! Those holes absolutely do look like you’re meant to hold them that way, too!

I think one of the first 5¼" floppy disk boxes I had (or was it the box the disk drive cam in? I don’t remember.) Had a stylized drawing of a hand holding a floppy disk, and the disks came in paper envelopes with the label sticking out. So I never even had to think about how and where to hold them.
I was also shown how to handle floppy disks (carefully!), but that poor lady’s mistake would have been completely avoidable if her environment had contained the right queues.

I hope she was able to laugh about it later. This kind of humiliation can last a very long time.

1 Like

What I ask myself is if the users know that you’re expecting this of them, if they know what to even search for in the documentation, and if they have reason to believe that it could be worth their time.

And that’s not a rhetorical question, either. My experience is probably not representative(*) and I don’t know what the insides of DT look like, so I don’t know what “the average” user thinks, or what is easy/feasible to implement. But I am convinced that if a GUI can provide cues to users about the concepts it implements, then that is never a bad thing.(**)

(*)The first time I used DT was around 2012 or so, and I found my way around it without referring to the documentation. I’m sure I could have learned useful things and probably used it more efficiently if I had bothered to read it, but in no scenario would I have spent that time up-front. I used RT and DT in tandem for a while but then switched to mostly RT. Only after coming back to DT (to check out this “filmic” thing about which I’d heard wondrous things but which was supposedly hard to use), did I conclude that my usual approach would fail here.

(**)unless of course if those cues are getting in the way of people who don’t need them. Which is why I’m very much not advocating talking paperclips or fewer bigger buttons with different bright colours, and an “easy” mode with a single “awesomeness” slider to make your pictures more awesome.

6 Likes

Yes, the information is far more valuable if it is right in front of you when you need it.

3 Likes