New Sigmoid Scene to Display mapping

Yes! It is my firm belief that the transition out of linear is a subjective endeavor; attempting to lock it into a one-size-fits-all model just doesn’t cut it for all images.

This is indeed the basis for the ODT transform in ACES, and the calibrated device-specific profiles in the ICC workflow. I personally think that putting any user discretion in it is just convolution confusion of the intermediate process between linear RGB and the output interface, where filmic and the other tone strategies live.

Just my two cents…

2 Likes

If we are going by nits, I suggest we give the (enthusiast) user the flexibility to define it to suit their system and workflow.

This is my belief too. I hope that all my arguing above is seen as:

‘curve-X’ is an addition to ‘curve-Y’ and here is why…

If a separation of tasks into RRT and ODT would happen, I might be inclined to see the job of the BT.2390 EETF actually in a custom ODT to be a ‘last-resort’ option of how to leave certain luminance ranges untouched (1:1 translation of input-brightness to output-brightness ) and for example overbrights (100 nits to 1000+ nits) to be reigned in to whatever you want. This would tie in with:

So let’s say you have a display with peak-white at 450nits. Technically not really a HDR capable screen but the user wants to at least somewhat exploit the screens capabilities while leaving everything that is supposed to be 100nits at that. This is where the bt.2390 EETF could be used. (Same argument for ‘blackpoint’ adjustments)

Of course a different tonemapping of the ODT could be done with log-logistic+skew where actually all display-brightnesses would be changed except middle-gray, but this would technically interefere with the idea of a HDR workflow where source-brightness-intent is respected at least up to 100nits.

This is where it becomes really helpful to separate between RRT, ODT and possibly even a device specific EETF (if one whishes to use high-brightness high-contrast displays in such a fashion).

  • RRT: on the perfect Display, it looks like this!
  • ODT: on a calibrated Print, sRGB or HDR10 screen this is the most technically(!) correct rendering of the ‘perfect-display’. Dear printshop, change me if you think you can do better.
  • display-EETF: I’ve got this high-brightness high-contrast display here and I want to use it as a 8bit-DIY-semiHDR-solution and I have a colorimeter…let’s do this!..but I also want to display my jpegs-correctly!

After thinking about it a bit I could see log-logistic+skew in each of the above roles (for HDR’ish reasons the least in the custom-ODT display-EETF role). A hypothetical Dev might ask now: why the hell would you use up to three ‘simple’-curves consecutively to do what one 4th-order-polynomial can do?!?! My answer would be: because it prevents the conflation of Intents (best-case-intent, display-specific-intent and before all creative-intent), seems future proof and when separated into different modules, task specific development makes trouble shooting more fine grained and thus in theory simpler. Also: you don’t have to. take one module if you want.

But I wholeheartedly agree that modules which supply all of the above in a one-stop-shop solution, are easier to ‘sell’ to people. :man_shrugging:

I guess that is assuming that there is no display referred modules in the pipeline? Or if there is, where would they be? I think a graph similar to what I did above would help explain what you mean because I have some questions but they might just be misunderstandings.

Side note, this is kind of outside the scope of my pull request but nonetheless an interesting and valuable discussion in figuring out the specific needs of a future proof tone mapping.

1 Like

Indeed it is! (oops) I keep forgetting the legacy modules and compatability requirements of dt. In that case there would have to be a clear cut after RRT+ODT into display-referred space with a requirement of the ODT to be most likely sRGB.

This is my clumsy way of counter arguing some things that were initially brought up against log-logistic. ‘This is the old way!’ no, it isn’t, here’s why. ‘Not futureproof!’ no, it is, here’s how. ‘Not enough parameters!’ no-yes, depends on where in the pipe to decide what and what to use log-logistic for. Unfortunately arguing this involved to talk about a not-really-disclosed pipeline concept in comparison to other pipeline concepts.

Sorry for derailing!

2 Likes

Ok some images!
Trying to keep actual contrast constant and only changing skew. Colors made it much harder to see the effect so all pictures are desaturated using luminance before the sigmoid. Pictures are ordered as follows:

  1. Negative skew
  2. No skew
  3. Positive skew

Pushing the skew slider a bit more than it needs such that it actually is visible!
The difference is surprisingly small in some pictures. Especially after looking at the plots so much. Downloading them and fast flipping between the images will make the effect more apparent.

Also updated the PR but its a bit hard to use atm. Try if you want the results will stay the same but I need to work some on the user interaction.

Forest Mushrooms:
IMG_5032.CR2 (23.9 MB)
IMG_5032.CR2.xmp (19.4 KB)
IMG_5032_01.CR2.xmp (18.1 KB)
IMG_5032_02.CR2.xmp (18.5 KB)



Kitten:
20181030_IMG_5417.CR2 (23.5 MB)
20181030_IMG_5417_02.CR2.xmp (11.0 KB)
20181030_IMG_5417_01.CR2.xmp (10.6 KB)
20181030_IMG_5417.CR2.xmp (11.0 KB)



Snow:
IMG_6872.CR2 (22.0 MB)
IMG_6872.CR2.xmp (9.5 KB)
IMG_6872_01.CR2.xmp (9.1 KB)
IMG_6872_02.CR2.xmp (9.9 KB)



And an old favorite:
A study in blue
RPN_Kick-In_20180830_3392.NEF.xmp (14.0 KB)
RPN_Kick-In_20180830_3392_01.NEF.xmp (13.6 KB)
RPN_Kick-In_20180830_3392_02.NEF.xmp (14.4 KB)



7 Likes

Awesome!! Great work. The results speak for themselves.

2 Likes

Desaturating before tone curve with a saturation function that is neither perceptual nor physical is a nasty trick to hide the fact that decoupled RGB curves change color in uncontrolled ways.

Fix the problem, don’t hide it. The cyan skew in the blue picture is simply unacceptable. Chroma preservation is not perfect, but at least it adds some reliability.

Also, ACES are not a gospel. The logic of what they do is nice, the actual implementation is a monster that changes at every next version.

2 Likes

I’m aware of your feedback on how to handle colors in the scene to display transform. Adding ratio preserving as a processing option is easy but I have just focused on the shape of the curve so far. Adding skew as well as the other curves to compare with has taken all my time so far.

So I would love some feedback on the curve rather than the color at this point!
Some examples of helpful feedback:

  • How do the black and white pictures look to you? Good? Bad? Why good? Why bad? Etc.
  • Are the images bad examples? Do you have any better test images to evaluate the processing on?
  • Have you tried to tone-curve-explorer? In that case, what are your takeaways?
  • Does the skew log-logistic curve fulfill the mathematical requirements for a scene to display transform? I have added support for display white and display black, custom grey scene+display is also supported but not exposed in the explorer. Skew is still not orthogonal but the effect of it should be clear and I’m working on a solution to the orthogonalization.
2 Likes

You might want to consider test images sets from image processing research. The more types you throw at your curves the better. E.g., clean, noisy, blurry, hazy, rainy, blown, low light, underwater, detailed, SDR, HDR, compressed, with artifacts, multi-spectral and other types of imaging, artificial, cartoons, etc.

1 Like

Absolutely agree!
Do you have a link to a repository or similar?

Well, it is a long list I have above. Search Google Scholar or GitHub, etc., for listings of papers that address “innovative” “novel” “awesome” :joy_cat: data sets. They will have the links and info on the sets.

Here is the thing : I don’t care how it looks and neither should you. “Looks good” belongs to amateur empiricism, unless you are Kodak and can conduct extensive aesthetic studies with N > 100 in controlled conditions. Because it looks good on some pics doesn’t guaranty it will look good on all pictures. And also, looks good to whom ? Let’s ditch that thinking.

A tone curve is a mapping. The only relevant question is : how flexible can it be made ? The user will decide what looks good, so starting with his intent (target look), can we provide him a mapping with a reasonable set of parameters that reasonably matches his target ? That’s the only thing that matters, design-wise.

That’s also the primary reason for node-based tone curves. Draw points, interpolate, done. No intermediate assumption, just honor user input. Unfortunately, tone curve GUI is not suited for scene-referred where DR \in ]0; + \infty[ and the relevance of a 2D graph to represent a 1D mapping is disputable. Also, splines interpolations do overshoot.

But mapping in HDR poses an extra problem, compared to SDR tone curves, which ends up in a trade-off : how much do we want to let local contrast unchanged in midtones (aka the safe range that is commonly shared between SDR and HDR) while compressing global contrast ?

Because if you simply adjust contrast for midtones and harshly trim the output to the display DR without further mapping, the picture looks believable and correct. We have done for decades. But us, photo geeks, will mourn the lost details in deep shadows and bright highlights, and the wasted camera possibilities. I guaranty that replacing tone mapping at all with just a soft clipping (like the highlights compression you will find in “negadoctor” module) looks a lot better than all the shit filmic does. Unfortunately, you won’t be able to get skies back, so you need to love white skies with flat clouds to consider going this way. Also, no color handcuffs, so saturation and hues will go somewhere unexpected, depending on setting strength and RGB color space in use.

On the other end, if we do a simple log scaling, we can manage to keep middle grey where it is while bringing back both ends of the scene DR to the display DR, but at what cost ? That picture will look washed and ugly, not believable. Yet, if you go in “unbreak color profile” module, use it in log mode, and if your picture doesn’t need too much compression, it does the trick decently.

So we need to account for both ends of that trade-off : protecting midtones while squeezing DR bounds in a way that allows defining the weighting for each strategy. Now, the mathematical challenge is to devise a smooth, continuous, monotonous (aka bijective aka invertible) function to allow all that.

Starting from these specifications, with almost a decade of experience as a photographer and too many lost battles as a retoucher, and building on Troy Sobotka’s work (another 15-20 years of experience in pixel nonsense), I came up with the filmic spline. Not all filmic parameters lead to a sane curve, just like all tone curves nodes don’t lead to a sane curve, but it does what it is supposed to do.

I’m afraid you went the other way around : you started with a solution you found cool, sigmoids, and tried to retro-fit them in mapping problem you overlooked. Well, they fit the continuity, monotonicity and smoothness bill, ok, but what about mid-tones protection ? I’m sorry but we don’t start design by the solution and we certainly don’t settle for a solution that just “looks good”.

Empiricism works until it doesn’t, a stopped clock gives the exact time twice a day, and pictures that “look good” may well be a stopped clock that you just watched at the right time.

So if you want to contribute something useful, you either start again from the problem at hand, including all the constraints you have skipped, review the possible solutions, and then find the best suited, or do yet-another-arbitrary tone curve that could probably fit in the “base curve” module as a “parametric” mode, but in the latter case, don’t bother calling it HDR-whatnot.

And, for the last time, all the ITU BT something and other ACES stuff usually aims at being usable on embedded TV chips at 60 fps, so they are willing to sacrifice a lot of accuracy to that purpose. We don’t do 60 fps and we do GPU processing, so forget about sacrifices. The only reason for the lack of decent gamut mapping everywhere is using proper color adaptation models is simply too expensive for a TV chip, so at best they do nearest-neighbor mapping on pre-computed Y’uv LUTs. Standards and recommendations have a scope in which they apply, and we are lucky to be out of the scope of HDR TV, so let us not get dragged down by limitations we are not subjected to.

I have quickly played with it, but I’m already swamped with work and I’m afraid that is low on my stack. What I take away from that is you can tweak parameters to approximate any curve with any other curve ± \epsilon, which is expected.

4 Likes

You are starting to fooling yourself, the rgb ratio preserving technique is knowed from 1994 and the only reason why nobody in image and video industry use it as a primary way to manipulate contrast is that it doesn’t works.

You could easily see that a lot of people in this forum don’t like the hue shift introduced by v1,v2,v3 and v4 “color science”.
I must have to say that isn’t color science at all, it’s just desaturation and midtone resaturation like YOU like, and it doesn’t looks good 90% of the time.
Too much desaturation in shadows, total highlights desaturation, grayish hair, innatural skintones, sunset and flames.
Actually rgb ratio preserving works well only in blue sky photography.
Not enough to consider it the new standard, it should be treated just like an alternative over per-channel rgb.

I don’t understand what is your problem with a new tonemapping module, with filmic is hard and sometimes impossible to spot on brightness and contrast.
I’ve tried to match the base curve and i could tell that filmic has some serious limitations.

Open your mind.

4 Likes

Let’s stay civil, please.

5 Likes

It is meant to preserve chromaticity, and it does just that. So it works at what it is supposed to do. And it was part of ACES 0.1 or something, so the “nobody uses it” argument is void. The main reason for not using it is because it’s not possible to put that in a 3D LUT, which, again, is a big deal for the 60 fps guys.

And they have an option to bypass them. I also see a lot of people who are happy with that, and even some analog photographers that switched to digital because of filmic. So, some people like it, some don’t, that leaves us with zero actionable information.

Grow some visual and art culture.

Until you provide a portfolio of photographs that look even remotely publishable, you are a nobody with coding skills and opinions backed up by no practical field experience. I’m fed up by IT guys and physicists who have opinions on stuff they don’t master just because they understand code and equations, and yet they can’t see a gamut escape when it’s right in their face.

3 Likes

To close on the “hue shift” issue, let’s look at 2 pictures from hdrihaven :

They both have colored highlights, one in blue, the other in orange-yellow, so they sit on opposite sides of the color wheel.


Linear correction of -4.5 EV for reference color of highlights (linear correction is supposed to preserve color perfectly) :

Linear correction of +2 EV for reference color of shadows :

Filmic v3 on individual RGB, no norm, no desaturation :

Filmic v4 on power norm, desaturation to 0 outside of latitude :


Linear correction of -3 EV for reference color of highlights :

Linear correction of 0 EV for reference color of shadows :

Filmic v3 on individual RGB, no norm, no desaturation :

Filmic v4 on power norm, desaturation to 0 outside of latitude :


Whoever finds the norm versions more color-shifting than the individual RGB versions should either snort better shit or get their eyes checked. Notice that I don’t care which one looks the best, where filmic happens in the pipeline, the color grading is supposed to have been done before and we are to honor any color decision taken in there, not to enhance or beautify stuff. Meaning we are to retain original colors as much as possible with no additional color opinion.

1 Like

Thanks for pointing out that resource, there is a lot to experiment with true HDR images!
Having said that I am a seasoned amateur photographer (started with Canon film cameras 40 years ago) with some understanding of code, what I personally don’t like of filmic v4 is the midtone resaturation, I find it harsh and prefer to resaturate using color balance. Apart from the fact that in filmic saturation is applied to midtones only, are they two different algorithms, and if so, why?
Maybe, also as a side effect of you recent work on color balance rgb, saturation in filmic can be improved ?

Also one (maybe stupid) curiosity of mine: we say that in scene referred workflow values are unbounded [0, +inf], but RAW files have normally 16 bit and sensors DR is 14-15 bit. This is not going to be improved substantially any time soon, so in reality values should be considere bound to [0,65535] or?

Cheers,
Marco

Can we please stop with the condescending tone, it isn’t good for any of us.

24 Likes

It’s all coming down to a difference in values.

@age values a tool that directly makes a good “look”.

@anon41087856 values a tool that is utterly neutral and objectively accurate and allows you to use other tools to achieve a look independent of dynamic range management.

I have my own preferences on the matter but the key thing is that both opinions are valid and there’s no point in insulting each other over it.

15 Likes