When there is no anisotropy or variance regularization, probably yes. But those are not the interesting cases.
I think that in the general case, the most important low hanging fruit would be controlling the number of layers it decomposes to and most importantly how it does that. Eg if one is after an effect that can be described with 2-3 layers, the speed gains could be substantial.
Another speedup could come from a more nuanced implementation of the heat diffusion. With adaptive methods maybe it could take larger steps, and thus use fewer iterations.
(Note that my understanding of the precise role of the wavelet decomposition is still sketchy, so I could be missing something.)
Here I beg to differ, as the 4 speeds can vary, as well as the (direction of the) gradients between the two layers considered.
(Also, in my reading, the two layers used are not two subsequent wavelet scales, but one of the decomposition layers and the low-frequency residual layer.)
Correct, but you have to go one step further: those values depend on the “edge threshold” and “edge sensitivity” sliders. The “raw” variance is the sum of squares of the calculated changes. That means that the final value will be controlled by the largest changes, i.e. the steepest gradients.
For “flat” areas that value will be relatively small, so the effect of the “edge threshold” (which influences the variance_threshold") will dominate. For pixels on steep gradients, the raw variance will be large, so this will dominate the result, especially after multiplication by the regularization factor, which is based on “edge sensitivity”.
Net result is that the downscaling depends on the steepness of the gradient and the settings for the edge sliders.
Note that from the way those sliders are presented in the tooltips, they are supposed to be used for correction purposes, after speeds and anisotropies are set. As such, they are not essential to the understanding of the basic functioning of the module.
You already have two sliders controlling the number of layers: “center radius” and “radius span”. Given the nature of the beast, I’m not sure you could add another slider to control the number of layers.
Also, the number of layers is proportional to \log_2(r) where r is the sum of “center radius” and “radius span”. So the number of layers doesn’t grow all that fast. Low values for the two radii would mean few layers, large values => many layers (but with the soft limits, still less than ~10).
What could be possible is replacing the two radii by one, going back to the first version of the module (look at the parameter sets defined in the code, one of the radii has been added after a first version was included in dt’s source). But I assume there was a good reason to add a second radius parameter…
NO
The number of iterations is determined by slider “iterations”, and step size by the four “speed” sliders
So, since there’s no iterative “solving” of any PDE, you can’t really use adaptive methods. We are not solving a system here in the sense that we want to converge (i.e. change between iterations is lower than a giving limit)
As far as the manual is concerned, if you need to explain derivatives and trigonometry, you have lost your non-mathematical audience. Part of them won’t understand it, and part will go “Ooh, math, that’s way to difficult for me”
I think we misunderstand each other. I think that the module is definitely solving a PDE from a Dirichlet boundary condition by using a finite differences scheme. It is not an “iterative method” in the sense of looking for a fixed point etc, but it iterates in the time domain. But this is the simplest of iteration schemes for PDEs and perhaps it could be improved.
The same applies to a lot of sections already in the manual (wavelets, color properties, etc).
Please also consider the people who would benefit from a simple explanation using very basic math, especially if supplemented by graphs.
Right. So you want an adaptive scheme. First question that springs to mind is “adapt to what, exactly?”
Oh, I know many would benefit from an explanation. The question is not there, but whether such an explanation should be in the manual. The explanation as proposed will run over the equivalent of several pages, once you start several graphs and figures.
And derivatives and trigonometry are not “very basic maths” (at least not for the general population where I live, this forum might be different).
I am not advocating for an adaptive scheme, but adaptation usually happens until a desired accuracy is obtained. I would imagine that for image processing, that can be on the order of what can be visually perceived.
However, I imagine that an implicit method (eg Crank-Nicholson) would improve things too. But I have no experience with PDEs on this size/resolution (basically, the number pixels in an image) so I would have to experiment before suggesting something concrete. Maybe others have a more concrete suggestion in the meantime.
As the maintainer of the documentation, I don’t intend to merge anything that tries to explain the underlying mathematics. The documentation should be a statement of how to use the module and what it does (the effect things have on an image). I don’t understand the mathematics and therefore wouldn’t be able to assess its accuracy or maintain it when the module changes.
Honestly I’m uncomfortable with how technical the documentation is at the moment but it was written by the original author and so is at least “authoritative”, and I’m reluctant to change it without a much better understanding of it.
If you want to delve into the mathematics that’s fine but it would be better as a blog post (either here or on darktable.org) or just a forum post here.
That was my take on the issue. I think I have a decent understanding of what’s happening, I’m not completely sure about the how and why of a number of steps.
I’m very happy that progress is being made with deciphering how the module works (not for me, it’s way over my head; I only know the simplest basics of signal processing and calculus). I also agree that the deep mathematical / technical details should not be part of the manual, or should be in an appendix.
I hope for an outcome where, based on a better understanding, a more user-friendly explanation can be added, probably with graphics as proposed by @Tamas_Papp. One can only distil the essence of something and explain the practical aspects, if one has a thorough-enough understanding of the details and background.
I have worked with a lot of software over the past five decades. Often, software documentation will have both a user manual and a technical manual. A separate technical manual seems to be what some are calling for.
I understand, of course, that this is FOSS and those who strongly desire such a technical manual need to step up and create it for the community. This could be either as individual contributors, or they could create some sort of “working group”.
Better for such documentation to sit with the code (in terms of the code being readable and commented where it can’t be made clear). But for extra technical info, perhaps a wiki page (or pages) on github might be the best way. Having the documentation alongside the code makes it slightly more likely to be updated (though often even comments beside the code get out of line with it)
The code of diffuse and sharpen is rather well documented. What makes it hard to understand is the optimisation, with intermediate buffers that are reused, and calculations that are divided in several parts, which are not close together…
Just to be extremely clear: we would be super happy to publish an article on the darktable.org blog. We’d be happy to publish a vast variety of things in the darktable.org blog
A lot of the existing blog posts were written by the people who authored the code. As with the documentation, there is a high likelihood (verging on certainty) that the person who merges a new blog post will not be qualified to judge its accuracy, so some sort of disclaimer might be a good idea. Alternatively (if you’re feeling brave) you could try to contact the module’s author and get him to review.
Well, the worst he could do is say no, but the response might be colorful
I think most of a write-up could draw from the manual and this thread, so that would help to maintain accuracy.
My main question is how it would conclude. Don’t get me wrong, I greatly appreciate the insight that this deep dive has provided, it’s answered a lot of my own questions and I’m very impressed that someone other than the developer could work through the module.
But even with this insight I’m still not sure I can independently work through the controls to achieve a desired effect without relying on the module presets.
I agree. This thread has produced some great insight. I think what a lot of us hope for is that this research has been a first step towards potential interface improvements so that it’s not just a “presets module”. I still feel we need some visualization about what wavelet frequency we are working on, rather than just putting the module in difference blending mode. Hopefully @rvietor’s and @Tamas_Papp’s excellent research can spark ideas.
What I still don’t understand from the code is how large-scale effects are achieved. Specifically, with a large radius span, pixels far away are affected, but the kernel is local.
Is it the mult parameter in the heat PDE solver? Is it correct to say that the PDE is then applied as if they “neighbors” were \text{mult} = 2^s pixels away?
For the larger scales, pixels from further away are sampled, so the kernel convolution is no longer “local” in that sense. This is based on so called á trous algorithm, described in this paper:
