You shouldn’t regard this specific branch (every forked repo for that matter) as having a stable or development (somewhat stable) status. It is a developer’s, in this case Aurélien’s, personal working grounds. Experimental would probably be a correct tag and that comes with all kinds of warnings and uncertainties in and of itself.
2 Likes
I thought the setup was a “unit testing” kind of environment, so I agree about stability, but I thought it still worthwhile mentioning the anomaly in case someone saw it who had been working in that area and said “oops that could be me…”.
OT:
Image preview is not updated in lighttable, if a style is applied. That is true for me in current git master as well. If lighttable contains several rows, scrolling up and down refreshes the preview for me. So here a redraw is missing to me.
@pk5dark hi Christian, I just double-checked in my older version, and it’s working as I said, that build is dt “3.0.1+dirty”. From 10th March I think.
I didn’t try anything with several rows though, just did import | image twice, same result both times, went from green UniWB to daylight as per my style.
_6D_10543.CR2 (25.7 MB)
_6D_10543.CR2.xmp (8.7 KB)
@anon41087856 , I’ve been having another play with V4 and would be interested in your comments. In particular, I’m asking about the new “reconstruct” controls, I wonder if I’m using them sensibly. So with the attached, which has a clipped area according to the raw toggle button, I’ve adjusted the Scene tab first, then onto Reconstruct.
I adjusted the clip threshold using the mask view so that the area selected was smaller than that at value zero. I did this because the clipped area according to the button at the bottom of the window was rather smaller, and I thought why would I need to reconstruct parts that don’t seem to be clipped anyway?
Then I put the transition slider over a bit to soften the mask limits. Though when I look at the effect of quite large movements here, I can’t really see any change in the actual image.
Then I tried structure/texture and was looking to see if the twigs in the clipped area might change, but again this slider didn’t seem to make a difference. Similarly bloom/reconstruct and grey/colorful.
These are not criticisms; I don’t properly understand the controls, and my image may not be suitable for showing the effect of some of the controls, e.g. there’s not a lot of colour around the clipped area.
Any comments and tips gratefully received.
The reconstructed area doesn’t need to be 1:1 the clipped area, sometimes it helps for smooth blending, to get a bit more from the surround.
The wavelets inside also helps to smooth transitions, it depends on your region.
The reason is your image doesn’t have any texture to reconstruct, it’s smooth highlights from the sun disc. The maximum you can do is to remove the magenta highlights and blend valid/clipped regions without sharp edges.
Generally, here is how it works:
- we split each RGB channel into a pyramid of blurry pictures (low-pass filters of increasing radii), and for each blurry scale, compute the difference between the higher scale and the current blur (high-pass filters of increasing radii). The stack of high-pass filters and the coarsest scale of blur is a wavelet decomposition.
- Then, at each pixel:
- the texture term is the sum along all the scales of high-pass filters of the maximum over all RGB channels. This represents the most “sharpness” we can grab from non-clipped channels at this pixel.
- the structure term is the sum, for each channel, along all the scales of high-pass filters, of the interpolated high-pass scales. This “fills” the voids we have in the high-pass scales and ensures smooth gradients, even if it is not as sharp as the texture. It has a chromatic variant (for each RGB channel) and an achromatic variant (as the max of all channels).
- the reconstruction term is the sum of texture and structure terms (again with chromatic and achromatic variants), which represents the most high-frequencies we can restore.
- the blooming term is the sum, for each channel, of all the scales of low-pass filters. It represents the gaussian blur at this pixel. Again, it has a chromatic variant (for each RGB channel) and an achromatic variant (as the min of all channels).
If we take the coarsest scale of blur (called the residual), and sum all the high-pass filters stack on top, it is a straight wavelet synthesis, and we get the original image back. So, in the same logic, we replace the residual by the blooming term (which avoids solid color backgrounds you sometimes get in color-reconstruction algos), and replace the high-pass stack by the reconstruction term, then sum them back altogether.
Then, as a user, you can weight every term in the final mix to favour a sharper or blurrier reconstruction.
TL; DR:
- texture inpaints high frequencies with valid RGB channel if any (in 1D, along channels depth),
- structure inpaints high frequencies by interpolation of the neighbouring pixels (in 2D, along the image plane),
- blooming inpaints the lowest frequency with a pure gaussian blur (in 2D, along the image plane),
- final reconstruction is a × (b × texture + c × structure) + blooming, where a, b, c are user weights (with b + c = 1 no matter what). If a = 0, then you get only blooming, if a = 1, you get full reconstruction.
- each term is again split in two such that term = d × achromatic + e × chromatic (with d + e = 1 no matter what).
Finally, the high-quality reconstruction method runs an extra step of reconstruction but with RGB ratios, instead of RGB channels directly.
1 Like
Yes, I see the white streak becomes pink.
Many thanks for the comments and detailed explanation, I will be re-reading it several times I think! (And just looked up TL DR - not TL!)
Filmic v4 got merged in darktable master today, along with other changes.
We introduce “workflows” in place of the “auto-apply basecurve” setting:
By default, workflow is set to display-referred, which auto-applies the base curve.
Setting it to scene-referred will auto-apply filmic and exposure module altogether. Exposure will auto-apply +1 EV by default to prepare overall brightness for filmic, plus revert the camera exposure bias in case you exposed to the right with an in-camera exposure bias. Then, filmic v4 will auto-adjust its defaults depending on the exposure bias.
Alternatively, setting workflow to “none” will auto-apply nothing.
OpenCL is for now disabled for filmic v4, since the v4 kernels are not written. Please test it and report weird behaviour, and I will start on the OpenCL in a couple of days, to have it shipped in 3.2.
Please notice that filmic defaults will not fit every needs, and it is impossible to do so. Basically, we have only “classical” assumptions (statistically valid on a large sample of pictures) to work with, no precise information about the true value of the middle-grey for each picture. Filmic defaults aim at fitting most cases but don’t expect anything magical in there. They are only a starting point and you need to tweak them for your special cases. However, I’m confident that these defaults should match OOC JPEG roughly, therefore provide a good base for culling.
Noticeably, Fuji owners will need to add +0.7 EV systematically in exposure module.
27 Likes
You DON’T merge some crazy new shit to master when I had hard working day and badly need to rest!!! Now I ain’t gonna get any sleep because I gotta play with latitude. 
BTW. V4 feels like when you finally get enough experience points and can unlock new skills to take down the big boss with ease.
9 Likes
Good 
Tutorial is out too :
23 Likes
Aurelien,
THANK YOU for all your work.
A particular the most significant feature to my mind is the descriptive tooltips you have placed throughout.
[One of the biggest hurdles I have found with most modules in darktable is lack of helpful documentation - a cryptic tooltip has often been duplicated (with grammar) in the official documentation, without explaining adequately what is done]
Your work with respect to filmic v4, the tweaked exposure module and the preferences is in complete distinction to this. This makes the power of the module ACCESSIBLE.
Once again,
6 Likes
Great! - thanks
Hello Martin,
darktable is an Open Source project, this means that the documentation is also Open Source. One way to contribute to an Open Source project is to improve the documentation!
For tooltips, you can open bugs and suggest an improved text. For the manual you can download the git tree and improve it directly.
2 Likes
Even just opening a bug saying you don’t understand some part of the manual would be helpful. @elstoc started one for chapter 1 of the manual.
3 Likes
OK … I guess I asked for that, Andreas! However I was not so much berating the existing documentation as much as praising Aurelien’s very helpful tooltips.
I can see a conundrum in that a developer who knows and understands the source well will likely write terse documentation, effectively as an aide-memoire; whereas a plodder like me – who could do with more spoon-feeding, hand-holding, whatever metaphor you choose – will struggle to be able to adequately document something s/he does not understand well.
Yes, I could take the time to become familiar with the code and understand exactly what it does. This assumes I have the capacity (not just time, also intellectual, conceptual, mathematical). But, assuming for a moment that I have that capacity, I would then become more prone to writing terse documentation.
To quote the timeless “Real programmers don’t use Pascal”:
Real programmers don’t document; the code is obvious.
None of this is meant to be an excuse for not contributing to the project, merely a “Don’t hurt me” kind of weak defense! Maybe I’ll be able to make a few suggestions on documentation, but I’m feeling rather out of my depth at the moment.
Anyway, thanks for the encouragement to get onboard.
2 Likes
Fantastic! Good job!
I have a question: If a higher value on the “hardness” slider makes the highlights brighter and the shadows darker, is that not another way of saying you increased global contrast? So what is the difference between the hardness slider and the contrast slider?
PS: Why don’t you promote your librapay campaign harder in your videos? I think you deserve a decent wage for your job.
Not really. Hardness is more like the “texture” of the contrast, it’s a bit hard to explain.
Because I always forget. Arti-scientist here, not marketer ^^
5 Likes
This is an interesting point. If you look at the filmic “graph”, you see that increasing hardness results in a greater Y value at every X point, except where Y is zero. So surely the shadows would go lighter not darker? And yet I think the preview is showing darker. Please can someone explain!
No, it’s more complicated, because the mapping at the end of filmic takes the hardness into account, such that scene 18% get remapped to display 18% no matter what.
There is really no intuitive way to explain what hardness does, it makes sense only if you compare it to paper grade when doing darkroom analog printing. That’s why I compare it to a contrast “texture”.
Just don’t mistake it with the “true” contrast parameter.
1 Like