New Sigmoid Scene to Display mapping

PhotoPhysicsGuy · April 12, 2022, 11:58am

CAM16 in RT is akin to ZCAM but based on CAM16-UCS. The HDR-sigmoidic-JzAzBz I think is different to ZCAM. @jdc correct me if I am wrong!

jdc · April 12, 2022, 12:00pm

You mention the ZCAM (in Rawtherapee) module conceptualized by M.Safdar, Y.Hardeberg, R.Luo, I implemented it by strictly respecting the algorithm, the results tested by 2 contributors of RT and myself were disappointing (the code is implemented, but disabled).

So I “tinkered” in the CAM sense of the word, to bring to Jzazbz (implemented as JzCzHz) some CAM features : as mentioned by @PhotoPhysicsGuy

taking into account the “Mean luminance Yb%” of the scene
“absolute luminance” (in cd/m2) corresponding to the shooting conditions (speed, diaphragm, sensitivity,…)
“black Ev” and “White Ev”, which translates the dynamic range (DR) of the shot
as well as “surround” (average, dim, black) also characterizing the conditions of the scene.

The development of the JzCzHz module was full of pitfalls with problems that appeared, not documented by the researchers, especially for the curves using Hz in abscissa. But now everything seems to work, after many tests made by @Jade_NL and @Wayne_Sutton .

This part of RT (in Rawtherapee - Local adjustments mode, so that deltaE and transitions can be taken into account) also allows you to use “Log encoding Jz” and “Sigmoid Jz” in addition to the usual adjustments:

Brightness / contrast (absolute luminance) or Lightness/contrast (relative luminance), chroma, saturation, hue rotation
curves Jz(Jz), Cz(Cz), Cz(Jz) and also curves Jz(Hz), Cz(Hz), Hz(Hz)
shadows/higlight (Jz)
wavelet(Jz) (very simplified)
clarity and sharp mask
recovery base on luminance mask
mask (like the ones I developed…)

I don’t say it’s perfect, but it’s a step towards…and there is still a lot of work to do to make HDR an integral part of RT?

Excuse my bad english.

Jacques

RawConvert · April 12, 2022, 1:50pm

@jandren , can we go into clipping a bit further please. You’ve described cutting the top off, just what I had in mind. So I was envisaging a curve like below where instead of going asymtotpic at the highlights, it reaches a limit. Now I accept for nice results there shouldn’t be an elbow in the middle of a curve, and it’s said the slope should be a continuous function (I think). However, does this matter much at the boundary?, i.e. where y = 100% and x = a bit less than 100%. I’ve never noticed issues when doing this boundary thing in tone curve module, or read about people having issues. Can you produce an example of this going bad?

clip

I’m struggling to find a raw I can share that has this, and in does seem sigmoid blows quite fast! I still find it a bit strange that I can’t just adjust the blown highlights in a more “direct” way within the module, but that’s maybe just me, and it’s early days. Using the skew to do this feels like I’m abusing it.
Here the glint off the headlight is indeed (255,255,255).

This is a simple sigmoid edit.
_5D_03055-classic-car-V3-sigmoid-S-sRGB.xmp (45.2 KB)

Cheers

priort · April 12, 2022, 2:07pm

The man in the passenger seat looks a bit like a ghost…is that possibly a consequence of such a curve?? Maybe I misunderstood ??

RawConvert · April 12, 2022, 2:24pm

There’s no tone curve module, just sigmoid…and the man is a woman!

priort · April 12, 2022, 2:39pm

Sorry…was viewing on my phone… still curious then how sigmoid handled that area…was it blown out and compressed back to that??

RawConvert · April 12, 2022, 3:00pm

_5D_03055.CR2 (59.8 MB)

RawConvert · April 12, 2022, 5:05pm

@jandren , @anon41087856 , something is not right with filmic 6 in the sigmoid build, at least on my system.
In my pure Master build of 4 April, the Extreme Luminance slider is having an effect.
Whereas in my Master+Sigmoid build of 10 April, this slider is having no effect. (And that’s with at least 3 of the Preserve Chrominance options)

Can anyone confirm/deny?

jandren · April 12, 2022, 5:09pm

Not worth reporting to Aurelien if it works in the lastest master builds. I can see if I can reproduce it though (trying out your image right now )

priort · April 12, 2022, 5:11pm

Work fine on the one I prepared…it was from just before it was merged into the master ie v6 Filmic…I took it from AP’s GIt

EDIT you might be seeing a side effect of this…not sure

github.com/darktable-org/darktable

filmicrgb: fix norm handling in v6

darktable-org:master ← flannelhead:filmicv6-highlights-fix

opened 07:54PM - 08 Apr 22 UTC

flannelhead

+26 -6

We need to clamp the RGB norm before calculating ratios. Otherwise things like t…his can happen: 1. Consider the `max RGB` norm and input pixels `(1, 1, 1)` and `(1, 1, 2)`. Filmic white is set such that the source white is at `(1, 1, 1)`. 2. Norms are calculated and they are 1 and 2, respectively. 3. Ratios are calculated: `(1, 1, 1)`, `(0.5, 0.5, 1)`. 4. When the filmic tone curve is applied to the norm, it is also clamped to the `[0, 1]` range at some point. Now both norms have value 1. 5. Ratios are restored; now the output pixels read `(1, 1, 1)` and `(0.5, 0.5, 1)`. This is bad as the original blue pixel was brighter than the white! 6. Gamut mapping will preserve the color of the blue pixel and it results in a nasty-looking dark patch in the middle of a bright surrounding (think of e.g. lights where the center is clipped in raw). To remedy this, we already clamp the norm before calculating the ratios. This way when the ratios are restored, they will retain their brightness ordering. Gamut mapping takes care of finally clamping the brightest highlights back into the target range. I don't know if dumb clamping like this is the best solution, but the filmic curve is not really well defined outside the designed dynamic range i.e. it doesn't extend above the source white point value. Correct me if I'm wrong, please. From brief testing I find the white point adjustment in filmic behaving now much more like I would expect it to - the brightest parts will eventually turn white when bringing down the white point slider. Without this PR, this is not always the case. This behaviour exists also with the earlier filmic v5 revision but probably the mandatory desaturation has masked this issue.

paperdigits · April 12, 2022, 5:18pm

This is your own Frankenstein build. If you can reproduce with just the lastest master, please report, but you’re unlikely to get support for this custom build.

RawConvert · April 12, 2022, 6:00pm

I’m not looking for that, more reporting the possibility of some problem with the 2 sets of code when merged. Why Frankenstein? - no one has said the process I documented is invalid.

Entropy512 · April 12, 2022, 6:04pm

That’s probably better left as something for @jandren to handle as part of the PR process once he resumes it, as it’s more his responsibility than AP’s - if it is even reproducible.

Most notably, I believe your process includes a rebase, which can potentially cause unknown conflicts if you’re not careful. Your process is valid for experimentation, but you have to keep in mind that a rebase can have some unknown side effects, and reporting such side effects for something that does not even have a currently active pull request is not necessarily valid - especially reporting them to AP since handling any actual fallout would be considered the responsibility of @jandren

Edit: For example, If my memory serves me correct, I think there were some cases where older versions of sigmoid would print garbage in the UI due to upstream changes in core dt code. These didn’t cause git to flag a merge conflict, but did cause issues.

Edit 2: Specifically, see New Sigmoid Scene to Display mapping - #568 by phweyland - jakob later says “rebased and fixed”, but I suspect that rebasing alone was not the fix, additional work on his part was necessary.

paperdigits · April 12, 2022, 6:08pm

You’re merging out of tree code into darktable master. @Entropy512 is right, @jandren is probably your best bet for this build but seems unlikely that he would want to spend time on filmic debugging.

RawConvert · April 12, 2022, 6:11pm

What does that mean pls?

RawConvert · April 12, 2022, 6:18pm

@Entropy512 , ok. I’ll take note. So are you saying glitches like I reported have to be accepted as the risk of using code under development? -
Or more sophisticated git commands are needed to properly merge code? -
Or something else?

I think it was fine to include AP - it’s his baby and it’s theoretically possible he did something between the 4th and the 10th which caused it.

paperdigits · April 12, 2022, 6:22pm

@jandren’s sigmoid branch is not part of the official darktable code (yet). So when you merge sigmoid code (out of the main darktable tree of code) into the master branch, you have your own thing going.

Entropy512 · April 12, 2022, 6:29pm

You have to be very careful - I would only do this under the following circumstances:

You run @jandren 's code exactly as-is in its current state without rebasing - then it’s OK to report it to him
You are running darktable’s master branch with nothing additional merged in, then it might be OK to report to AP.

Reporting that X’s stuff breaks when Y’s code is merged is something that should just be left to Y, there’s kind of an unwritten rule of pull requests of “you broke it, you fix it” - especially if you break someone else’s code by accident!

Telling a developer that someone else’s code broke their stuff will either:

Get ignored as invalid and also probably annoy them
Get the developer pissed off at whomever broke their code in the event that the breakage is being propagated as something that resembles official (not what happened here, but often happens when Linux distributions break something that isn’t a problem upstream…)

jandren · April 12, 2022, 6:35pm

Had a try at your image @RawConvert and your comment is much clearer now!

Just default sigmoid:

Uff, that produces ugly pink areas in the highlights! That is what you want to blow out right?

So just adding a masked exposure module with a hefty boost on the brightest areas of the image:
(Note that I didn’t change the display transform type or settings, just edited data in scene-referred space.)

Much better!

Root cause analysis:
The pink cast that doesn’t blow out is caused by sensor clipping so the actual pixel data should be a lot brighter than it is recorded (its clipped after all). This causes problems in the sigmoid module as it expects bright stuff to be properly bright! The correct solution to this problem is to extrapolate the clipped areas using a scene-referred highlights reconstruction method, a feature that hopefully comes soon to darktable! An even better solution is off course to nip the problem in the bud and underexpose a bit at capture time, but that isn’t so good advise for an already taken image!

A comment about changing the clipping level using either filmic’s white point or the trick you showed in the base curve. I would call this a “display-referred” method for dealing with clipped areas, i.e. you relate the maximum level of your camera to your display maximum, a big no-no! In this case resulting in bright pale faces!

priort · April 12, 2022, 11:46pm

Frankenstein… because it has no acceptance yet and so really lives outside the master branch so you introducing it is on you and or @jandren to the extent that he responds… or any others that might be able provide insight…