Solving dynamic range problems in a linear way

Ok, the new feature is finished and ready to test: Unbreak input profile : add log profile by aurelienpierre · Pull Request #1730 · darktable-org/darktable · GitHub

I have added an auto-optimizer that computes the best parameters, allowing 2 clicks setting.

I will give it a try when someone makes a Windows build with the module. :slight_smile:

You’re that kind of person ? :stuck_out_tongue_winking_eye:

Hi Aurélien
I’ve made a build (windows) from your branch color-grading. The interface I get is this one:
image
The last slider (contrast correction) seems not to have any effect.
I don’t understand well the behaviour of color-pickers. The first click should activate it (it does) but the second one should deactivate it (it doesn’t), correct ?
The contrast correction color-picker sends a message I cannot read properly:
image
Is the auto-optimizer still available ?

@phweyland are you using the right module? Doesn’t look the same…

Edit: oops, never mind. That is what I get for skimming. :blush:

@afre yes, he is, I have made some changes. I’m trying right now to change the color pickers behavior to see if I can make them like the color balance ones.

@phweyland I will give you a full answer when I fix the color picker behavior. Hold on.

1 Like

Ok, please fetch the last version and follow this tutorial:

  1. open an image and ensure, in the exposure module, that you have no clipped highlights or shadows. Don’t hesitate to push down the black level and the exposure level Capture%20du%202018-09-21%2018-00-01

  2. in the unbreak color profile, click on the “middle grey target value” color picker, then draw a rectangle over an area that is supposed to be 50 grey (or just choose the whole image to get its average luminance as a 50 grey reference
    Capture%20du%202018-09-21%2018-01-25 )

  3. click again on the color picker to fetch the actual luminance value (it won’t be saved until you click again on it)
    Capture%20du%202018-09-21%2018-04-08

  4. reproduce the same process on a black or near-black area, with the black relative exposure color picker (careful : it takes the average lightness of the area, not the minimum, to reduce the sensitivity to noise in low-lights, so you have to be accurate in your selection). It’s hard to have real black, so you can tweak the value a bit after the picking (which gives you a first idea to speed-up the process)
    Capture%20du%202018-09-21%2018-06-49

  5. to automatically adapt the dynamic range, use the last slider (contrast correction) and, again, the color picker. This time, it has to be on the whole image (dark and light areas) because it takes the extrema of it. Click again on the color picker to have te computations done or begin adjusting the contrast correction . A negative will give you a better contrast but will sacrifice the extremes of the dynamic range, a positive % will protect the extreme values better, but will look very dull.
    Capture%20du%202018-09-21%2018-15-33

  6. Now, the optimizer is automatic but not magical. It is sensitive to noise in low-lights, so you may have to tweak a bit more the black relative exposure so your histogram in log mode is well centered.

  7. This is now a sane base to color-grade, and tune the contrast in a linear way. However, you have to re-add contrast and fix he lightness before saving the picture in JPEG with a S tone-curve.
    Capture%20du%202018-09-21%2018-20-15

Note that I haven’t used anything else here than 3 modules (exposure, unbreak input profile and tone-curve), and yet I have clipped whites only in the sun (in red), and almost no clipped blacks (in blue) even though I have a nice contrast.

The photo is © rawfiner.

Benefits :

  • no over-saturation problems in highlights, thus no gamut clipping
  • no staircase effect in sharp contrasted transitions (branches/sky, bokeh rings, etc)
  • fully parametric log function, relatively easy to remap the histogram where you want it
  • clean linear base to perform any adjustements in a safe way without clipping (especially : local contrast, high-pass filter, etc.)
  • no need for the shadow-highlights module that can have a very damaging effect in many cases (local contrast reduction, loss of shape and transitions, borders effects, desaturation)
  • no need for fixed reverse-engineered manufacturer-like basecurves
6 Likes

The manual definitively helps ! :slight_smile:

Step 1 to 3 are ok. Just a bit of trouble because the selection frame remains active. On the other hand changing the selection and clicking again of color-picker gives a new adjustment of middle-grey.
But even if the plugin is deactivated, the frame is still there…
1.


2-3.

4. select dark area

4.click - error message - selection as gone

I can repeat 4 several times with same result. What should have to do in that case ?
4. last try
if, after the first click I let the selection over the full image and click again I get this (dynamic range and black exposure modified, selection gone)

5. I confess that I don’t understand the instruction 5. (to automatically adapt the dynamic range, use the last slider (contrast correction) and, again, the color picker …)
In any case the slider contrast correction, -100% or +100%, has no action on the image nor on the sliders.

Some thoughts. If the color-pickers appearance changed depending on their state (like in tone curve for example), that would help the user to know what the next action should be.

I find this plugin really interesting.
Please let me know if I can help in any way.

Thanks for your feedback ! By the way, you are not using the latest version, the error message should be corrected in it (and the black relative exposure has now a color picker).

The slider contrast correction is the security offset for the dynamic range optimization. You have to have the corresponding color picker active to have it working, or set it, pick (click on the icon), and unpick (click again) to get the measure.

All the color picker works the same : you have to click to draw and pick, then click again to validate and read the value.

I will remove the picker frame when you exit the module in the next update and auto-activate the picker when you change the value of the contrast correction if I can.

What a great step-by-step tutorial which explains so well how the module works!!! :bouquet:

1 Like

I have worked on a logarithmic blending mode that utilize log2(src,dst) for Krita. In my case, I did something like this

template<class T>
inline T cfColorBurnLogarithmic(T src, T dst) {
    using namespace Arithmetic;
//Also known as Darken from EffectBank/Illusions.hu. IFS Illusions had used this blending mode.

    qreal fsrc = scale<qreal>(src);
    qreal fdst = scale<qreal>(dst);

    if (inv(fdst) == zeroValue<T>()) {
    return scale<T>(log2(1.0 + abs(fsrc)/abs(inv(.999999))/8));
    }  

return scale<T>(log2(1.0 + abs(fsrc)/abs(inv(fdst))/8)); 
}

Just enough for float precision for float value. But, I don’t think that this would apply in your case, @anon41087856 .

Log2 1 is 0. I guess you could use 1 as the base for 0 value. Ultimately, what you do is up to you.

You could also replace 0 with epsilon as well or use maxvalue - epsilon. Epsilon is the minimal number after 0 which depends on color depth.

1 Like

Still learning git commands … (I thought it was up to date … !) :frowning:
image

I think that I’ve got the way to use sliders and pickers.
Following the procedure with color pickers I get this:


The contrast correction doesn’t help for that particular image. But what surprises me is that contrast slider changes the dynamic range… Intuitively the dynamic range is intrinsic to the image and the slider allows me to consider the range I want. And with contrast correction I would adjust the output. But in the current case, i don’t understand the added value of the contrast correction compared to the dynamic range one ?

Another point I would like to understand (but there is also an explanation for sure). If I adjust this image with Tone Curve, it becomes desaturated.


Usually this kind of curve tends to increase the saturation, doesn’t it ?
EDIT: this not correct. This kind of curve desaturates whatever the image…

Adjusting manually the 3 first sliders I get easily this (which seems to me a very sympathetic starting point for tweaking further the image) :slight_smile: :

An other thought. As the plugin works essentially with EV units, wouldn’t be interesting to see the shooting EV ?
Something like EV = log2( 100 * aperture**2 / (exposure * iso)).

The contrast correction IS the dynamic range. You can set the dynamic range manually or use the optimizer to guess it. If you do so, you can use the contrast correction to add ou substract a certain offset to the computed dynamic range, for example if you have important details in the border of the dynamic range. The contrast correction is essentially saying “optimize the dyramic range and add ± x % to the computed optimum”.

That’s because you have to use the RGB auto mode (see my screenshots). Otherwise, you are working in Lab auto mode and that’s not suitable for this job.

The exposure settings and camera dynamic range are not relevant here. This module comes after the exposure, lens correction, denoising, etc. modules. The operation it does is:
( log2( coef * input / grey_level) - log2(coef * min_input / grey_level) ) / (log2(coeff * max_input / grey_level) - log2(coeff * min_input / grey_level)).

log2( coeff * min_input / grey_level) is the black relative exposure.
(log2(coeff * max_input / level) - log2(coeff * min_input / level)) is the dynamic range at the output of the module.

So it sums up to ( log2( coef * input / grey_level) - BLACK_EV ) / DYNAMIC_RANGE. In auto mode, DYNAMIC_RANGE = (1 + contrast_correction) * DYNAMIC_RANGE_computed.

This dynamic range here depends on what correction you did in the previous modules, where you want your black exposure, and the grey level you use. It’s no longer camera-dependant.

Thank you for these explanations ! That makes things clear.

If I understand what you’re saying, we could put the color picker beside the dynamic range slider and remove the correction slider. I can still work manually on dynamic range or with the computed one and adjust it afterwards with a unique slider.
But maybe I’m still missing something … :slight_smile:

yes, it would be possible, but I thought it was more reproductible this way.

I fixed the UI with your observations :

Is that more clear ?

1 Like

Where it goes the old linearity factor?

Gone. It’s computed internally now, no need of user input on that part.

2 Likes

I think this is not a question of presentation but just principle.
Where one slider is enough, two are confusing. That’s my feeling.
Other inconvenient here:
When I move the correction the dynamic range follows the move. Good.
But when I move the dynamic range the correction doesn’t. Then where is the right tuning ?
Last, why dynamic range would not work as middle grey and black exposure work ?

it’s similar to the auto-exposure thing, where you can tweak the exposure by giving the amount of pixels you allow to be clipped : one slider moves the other, but not the other way around. It’s a settings optimizer, not a an automatic correction.

middle grey and black color pickers take the average reading of the zone, which need to be carefully chosen. The dynamic range one takes the extrema of the zone (min/max), which can be the whole image except if bad things happen on the edges.

Understood… sorry for the slowness. :slight_smile:
Is that correct to say that the correction slider modifies the behaviour of the dynamic range color picker ?
If so I would just put the color picker beside dynamic range slider (in that sense it works as the other color pickers) and name the correction slider something like “dynamic range color picker correction (or bias)”.

Actually, if I apply a correction to the automatic dynamic range calculation, then move the dynamic range slider and then the correction one again, dynamic range is back to the automatic one with current correction applied.