Color calibration - colorfulness

Are you using the v2 versionā€¦maybe try the same things and see what you get. My original confusion was I thought colorfulness was doing something equivalent to lowering say the red in each channel etc etc so like taking the red from red the greens and the blues. So RGB of say 100 134 80 would lose red predominantly but the cross talk would also alter other colors. Obviously the impact of such a thing in a different colorspace would not be the same on the overall gamut much in the way you describe. I tried with v1 to look at a color checker image and move the sliders and understand the effect. It wasnā€™t obvious. In v2 it seemed more relatableā€¦

This is not a contradiction to that, but I think it was Feynman who said that if you canā€™t explain your topic to a layperson, then maybe you donā€™t understand it as well as you think. That, too, I consider an overstatement, because some stuff just is complicated. But my experience (teaching at uni) tells me that what appears intuitive to one person (A) can be completely random to someone else (B), but that there is likely another way of phrasing the same thing which makes it very obvious to B (while A might be wondering why youā€™d phrase it in such a funny wayā€¦).

Applied to the issue at hand (and actually to the filmic module, too, which has some people, including myself, guessing a lot, and guessing wrong): I think the same operations could be presented/explained in a way which could allow more people to develop an intuition for how they work, what will happen if they pull a certain slider, and which sliders they should pull to get a certain result. Often, the key to this lies in unstated assumptions which one person considers too obvious to mention, and another to obscure to consider.

When I write a paper and I get a ā€œstupidā€ remark by some reviewer who didnā€™t understand what the point of the paper was: I canā€™t deal with that by explaining to the reviewer that theyā€™re wrong (even though they probably are!) ā€“ this is a sign that the point of the paper was not clear enough, and I need to rephrase it to make sure people get it. I find this the hardest and often most tedious part of my work but unless I explain what I do and people can understand and appreciate it, Iā€™m not really done.

4 Likes

Yeah, just realized Iā€™m still on 3.4, so it still has all those bugs in it. I should have read the whole thread before respondingā€¦ but itā€™s a really long one :man_shrugging:

It is. Thanks for your perspective

This, and a further example: early on in my career I attended a presentation by a very thoughtful colonel who said something along the lines of ā€œI want something that can be used in the dark by a young soldier with minimal education, who is cold, tired, and hungry, while heā€™s sheltering in a trench in the rain. Heā€™s also frightened because a lot of people just over there are trying to kill him. This (waving a modern scientific calculator) is uselessā€. A bit over the top for this forum, perhaps, but it illustrates the point. My expertise (and degree) was in life sciences but that shouldnā€™t disqualify me from being to use something like the color calibration moduleā€¦but I just canā€™t get my head around the sliders (DT ver 3.4). I donā€™t think Iā€™m alone, Bruce Williams seemed to have some difficulty as well, and posts here seem to agree. Not all outstanding photographers (you can exclude me from that category) and artists have an understanding of advanced mathematics. So Iā€™m really sorry to say that the color calibration module (as in 3.4) was not up to the usual excellent, high, easily understood standards in the rest of darktable. Iā€™m looking forward to an update with a modified color calibration module thatā€™s a bit more intuitive to aged grunt amateur photogs like me.

Please donā€™t shoot me, this is a clumsy attempt to give constructive criticism. DT as a whole is an outstanding package, arguably more so because itā€™s open source, and huge thanks and congratulations to all those who put in hours of work to make it so,

4 Likes

I find the soldier analogy very easy to get, although I would not hold Darktable to quite the same standard :slight_smile:

Iā€™ve developed engineering methods and implemented them in software (lots of computation, a little GUI) ā€“ and what Iā€™ve understood from that is that you canā€™t teach the users everything you know (even if theyā€™re fellow engineers working in the same field), but you also canā€™t dumb it down to the point where they donā€™t need to understand anything. The trick is to present the method in a way that makes sense to them, and choose a way to control the system which makes sense according to their perspective, not your own*. The number of times (and different ways) people have naively tried to use my software in ways that obviously (to me) could not have worked but still blindly trusted the results is staggering. But Iā€™d like to believe that a developer canā€™t be expected to anticipate all of those ways. So the only solution (unless you have in-house focus-group testingā€¦) is to make something, see how it lands, talk to users, and modify the presentation/controls until most users can work it out in a short time. They learn, you learn, and eventually it clicks.

Thereā€™s a lot of discussion about both Filmic and CAT around here, and I hope that @anon41087856 stays patient for long enough that enough people get not just an intuitive grasp but a more general understanding of both the method and what it is that many users donā€™t get about it, that the implementation/GUI can be updated to make it easier to use and understand. Heā€™s demonstrated a few times how the fundamentals heā€™s put in allow for pretty amazing results, but I think there must be an easier way to ā€œdriveā€ it.

(*) Example: I non-dimensionalize everything first, then work in non-dimensional space. Theyā€™re way more familiar with dimensional values (in imperial units, can you believe it?!) ā€“ so thatā€™s what is displayed. Had to implement unit conversion just for that purpose.

2 Likes

As a software developer in my earlier life, I recall many times sitting with the targeted users for hours to understand what they wanted and what it should do. Then developing it and taking it back to show them. I would see that they were puzzled, then they would finally say, no, thatā€™s not it. I would go back over the notes with them, and I know this is clichĆ©, they would say, ā€œYes, thatā€™s exactly what I said, but itā€™s not what I meant.ā€

Software is All About Abstraction, that is, making something that is complex a bit easier to deal with by the next layer of user. if it werenā€™t, weā€™d have thrown away these machines years ago as too cumbersome to deal with.

The art of it all is finding an abstraction that both reliably wrangles the complex thing and provides a useful interface to the user. The most successful endeavors Iā€™ve seen in doing such puts the developers in the trench with the soldier, to use @SalisburyJonā€™s story. One really needs to understand what the user needs to do and the world in which theyā€™re doing it, and then tamp their ambitions and just build the thing they needā€¦

1 Like

This is what has happened for a lot of the scene referred modules, the initial reaction is negative, but after some use, some videos, and forum posts, the module is understood and then praised. So give it some time :slight_smile:

3 Likes

Please keep in mind - we donā€™t have paid software developers at darktable that also would code games instead if they were paid for it.
The key developers are photographers, implementing stuff they use.
So thereā€™s no gap between users and developers, just between users that understand in detail whatā€˜s implemented and users that doesnā€™t :wink:

3 Likes

darktableā€™s space invaders clone would like to interject here :wink:

3 Likes

Oh, I know the dynamic. At the extreme end of the spectrum, Iā€™ve written a comprehensive raw processor for an audience of one, me. Iā€™ll find it interesting to see how many, if any, other people take up my predilections regarding workflowā€¦

In our business here, the wrangling of complex technology to creative ends, thereā€™ll always be a tension between developers and users, I think. How much do the developers abstract, vs how much do the artists learn about their medium. Iā€™m fine with the tension; I believe all who participate in it learnā€¦

3 Likes

At the age of 4 to 5 I was on a visit at my grandparents. On the kitchen table there was a bottle of Strohrum. I read murhorts. Then they told me to read from the other side :slight_smile:

1 Like

haha i love that story. almost as good as ā€œredrumā€. in any case youā€™re saying reading saved your life i gather?

1 Like

Try developing an expertise in image editing, and come back to me. That kind of statement is all what is wrong about photography in general : college-educated men with some expertise in unrelated fields thinking, out of pure hubris, that this unrelated expertise make them somehow expert at everything else, and since image processing is so easy, it shouldnā€™t be a problem.

Well, think again. Image processing is not easy, itā€™s a job. For real experts. Who trained for it.

Besides, color calibration features are standard in many other software since the 1990ā€™s. So itā€™s nothing ground breaking, nothing new, nothing unheard of. Your average Netflix/Hollywood movie is graded using similar tools. Not sure if your average Hollywood colorist is a math geek, I wouldnā€™t bet on it, but they managed to make those tools work decently in production.

The keyword here is polar decomposition of a 3Ɨ3 matrix. I have to finish it, but that stuff takes time.

THANKS ! \o/ Portfolio - AurƩlien PIERRE, Photographe

All the knowledge I have in fundamental image processing comes from my very frustration toward darktable after years of being but a simple user getting semi-shitty results with cameras supposed to be awesome. I wasnā€™t born like this. And I learned C/OpenCL only to hack darktable. I really didnā€™t picture myself coding all day, I donā€™t like that anyway.

All the tools I develop come from a look I want to achieve, as a photographer, and that Iā€™m unable to get with current things. Itā€™s always feature to tech, and not tech to feature. And itā€™s actually look to feature to tech. Look comes first.

So trying to cast away the dev from the photographer is not going to serve your point. You just need to humbly discard your unrelated expertises, try developing the required skills from scratch, and stop making excuses.

Also, photography is bloody damn difficult. People should really stop assuming it is or it should be easy. Itā€™s not. Color alone is super difficult to grasp (forget your usual hue-saturation-lightness, it doesnā€™t exist outside of shitty GUIs), and that is the core of what we do.

Iā€™m looking forward to an update with a modified color calibration module thatā€™s a bit more intuitive to aged grunt amateur photogs like me.

Color calibration will be intuitive the day walking, writing, counting and speaking are intuitive for a baby or for an adult with brain damage. Try walking drunk holding a glass of water on a tray and make it a priority to not spill any water. Thatā€™s about as intuitive as life gets.

5 Likes

I wouldnā€™t say it saved my life. But shortly after I learned reading, I read one Karl-May book per day until my dad told me how to read and write punch cardsā€¦

Teehee, yes! Because what users say they want is based on their interpretation of what your program does under the hood, which may be wrong. What developers think users want is based on what they think about users, and what they themselves would want for the stuff they do with the software (have I ever fallen into that second trapā€¦). It sometimes helps to discuss things in terms of what users want to achieve (or which part of the current setup does not work, and why) rather than how theyā€™d like to do it, because itā€™s hard for them to imagine how the software could work differently, but itā€™s hard for the developer to know in what environment and to what end the user is using the software, i.e. which implementation works for them and which does not.

I like to believe that after some >5 years working on one piece of software I got to the point where most of the stuff the which users did not get was simply due to no time being available to implement it ā€“ but each new feature still had a design iteration planned in because either thereā€™d be some misunderstanding about how it needed to work to fit into their workflow, or about whatā€™s technically possible.

1 Like

What Iā€™m not getting is why people think they should be a math expert to use a module.

I get it,those people donā€™t really understand what the module is doing or why it is doing it. But why is the module wrong then?

There are sooo many sliders in darktable (and other tools) that Iā€™m sure people donā€™t understand and do something they didnā€™t expect.

What you then do is play around with the sliders, and learn. Or move on and try a different module.
You donā€™t go saying the module is wrong and youā€™re (patiently at least)awaiting a replacement.

Iā€™m sure a lot of people coming from lightroom see the shadows and highlights module and think ā€˜yay,a shadows sliderā€™ and are then disappointed

You then think ā€˜well this is not what I was looking forā€™ and you try finding a different wayā€¦ Without understanding what was happening. This is good.

Soā€¦ The color balance thing. It is something different to what people expect. Or they tweak sliders and something different happens. Accept it does something different. It doesnā€™t work on plain rgb values, it is not usable as a plain simple channel swap. Move on.

Now, for the people that donā€™t get it and want to learn it. Welcome, I think Iā€™m one of you :).

And yes, aurelienpierreā€™s explanation goes over my head. Maybe because my sleep deprived head in lockdown with kids around the house :s.

Anyway, Iā€™m loving what the first tab does. Iā€™m loving the simple gamut compression and what it enables for filmic. I guess I have no need yet for the other tabs :).

2 Likes

If you use it in channel mixer mode (thereā€™s a preset for that, and also for the channel swaps), itā€™ll work without adaptation, in working space RBG (e.g. Rec.2020).
The implementation in 3.4 had bugs, see:

Iā€™ve read that, and that the bugs are fixed is always good.

ā€¦ but it still does something different from what most people seem to expect, and after the reported bugfixes there are still quite some posts about people talking about how it is too complicated, and how it should be possible to understand it. And I donā€™t think thatā€™s right.