i also like modularity. you can tell by looking at darktable’s architecture i think. the processing modules are in a way separated such that you can easily add and delete them. unfortunately they are not easy to create and hopelessly entangled with gui code. also the interface isn’t super general and doesn’t allow for graph processing, just a linear list.
in vkdt, there is a module interface too. it’s based on a node graph though, so you can connect the same output to multiple inputs (see the arrow connections on the right):
and these module graphs are further decomposed into a more fine grained node graph (sorry can’t see it if you’re on a dark theme due to transparency):
and the interface is much simpler, gui code is completely out of the loop, and modules can be created by just some text files describing the kind of inputs and outputs plus the actual processing code, see this simple example:
so. in a way most of what you’re asking we always had in a form or another (i suppose natron modules are similar).
what has not worked well in the past though is
different languages (dt: sse/opencl/i386) introduce a lot of mainenance overhead
processing history compatibility guarantees with 3rd party modules is not really possible
high performance using a mixture of languages does not work (even c+opencl is a problem due to all the copying!)
and really, i don’t understand how you would put down the one liner exposure compensation in the above example in more simple terms in any language. given that you can just copy/paste the 25 boilerplate code lines, what’s the difference.
so i think allowing everyone to write modules in any language will give you a neat toy for rapid prototyping, but i’d expect it to grow into an unmaintainable zoo of scripts and clutches between them, like packages in the latex ecosystem. if you care about shipping a coherent package with some minimum standards when it comes to code quality and performance as well as backwards compatibility you’ll probably want to keep all supported modules in one single place.
I think the hard part in image processing is not the programming language but the image processing. Image processing and colour is hard, it doesn’t get easier in JavaScript or Python.
Right, I think the next order of business is to state the scope of the solution and the intended audience. Because sentences like
frighten me.
The fact that a bunch of geeks armed with i9 can make the solution work and find the runtimes acceptable to edit a couple of pics each week with no real pressure to deliver a result is by no means a proof of usability in a production context.
Spend a week with a commercial/wedding photographer who has to work 2, sometimes 3, jobs to join both ends and spits roughly a third of their income in taxes & social insurances, and another third in various fees, you will understand what “precariat” means and where the tolerance level toward speed and efficiency lies.
Photography as a business is getting more and more precarious as customers are less and less willing to pay for it, while still expecting state-of-the-art results.
That’s interesting; I have the opposite emotion. That G’MIC allows for a wide variety of use cases IMHO makes it one of the best case studies for a plugin architecture. I think it represents a realization of a lot of the trades to be considered, from extensibility to efficiency. Just because it’s probably not the best tool for processing weddings really points to the implications of those trades.
No. But nobody sad that G’MIC is a tool for photographers doing mass editing. G’MIC is an tool for and from researchers, that is even usable for other use cases.
Agree, but on a lower level I think there’s utility in studying such scripting tools as infrastructure. I think it’s interesting that G’MIC at its origin is a scripting tool, but the majority of its users, in GIMP, have no idea of that.
Disclaimer: I never really used the admin privileges on this website so please don’t treat my comment as that of a Pixls admin.
It’s a great idea. I mean, if I were building my own raw processor I’d be a fool not to — every once in a while — track what everyone else is doing and what great algos I can legally reuse.
Would I give this to a real photographer? Not really, no.
If you don’t actively manage your product/project and instead rely on the “vibrant ecosystem” and on trusting users to build custom per-picture pipelines, you easily get to a place where:
the saturation slider and the color temperature / tint sliders are in entirely different parts of the UI
you have to spend half an hour setting up quick task panel to have simple access to the most used controls
you have to patiently construct a basic pipeline that gives you a sensible output
So a question I would ask is what problem are you actually suggesting to solve and for what demographic.
Is it tinkerers who have the luxury of tweaking one photo for hours on end? Sure, I can totally see these users taking a lazy Sunday afternoon to compare three or four implementations of a denoiser.
Is it for people who have maybe an hour at best to sift through an entire photo shoot and then another hour or two to quickly edit the good shots, send them for exporting and go charge the batteries and prepare for the next shoot while their TIFFs/JPEGs are being spat out one by one? You’ll find it difficult to grab their attention.
If you want this dream project to have maximum impact for professionals, I do genuinely believe that you should aim at giving these people the gift of the only resource that cannot replenish: time.
Good points …this is why we will always need a range of solutions for the various users out there. One size fits all will always be a compromise for some group or other. And the time vs 'tinkerability" spectrum will also be different for every user. Users and not developers will have to decide what is the best option for what they need and participation and interest should help to drive those projects… Some will think a vast array of plug-in offers the best option and some will see that as a mess.
I think you have a lot of people here that are programmers and interested and knowledgeable as it pertains to digital image processing, and some of those also photographers or have an interest in photography as a hobby, and finally there are others that are more concerned about a tool that serves photography as a task or it is their vocation and they have much less interest in being exposed to all the sliders and variables involved in tweaking the images and the time and complexity that it can introduce.
Can one project be expected to serve the needs and expectations of such a diverse group. I don’t really think so and therefore decisions have to be made about the true direction for any group or single developer so that you don’t have a jack of all trades master of none scenario… I think there is room for multiple solutions as there are multiple needs. In the end it will still fall along the lines of tools or projects that are really digital image manipulation used to edit photographs and then tools or projects designed to edit photographs that use and expose the end user to varying degrees of digital image editing complexity…
I don’t know how interested you are in programming languages, but both Rust and Julia can produce code comparable in speed to C or C++, while being much more expressive.
Thanks for the feedback! I’m afraid that I may have given the impression that I was offering a fully-fledged design proposal. All I have is an “utopian ideal” which I threw out there to see if it inspires anyone. That’s all.
I’d just like to expand on something I had already said, but maybe not very clearly: a framework with the ability to use plugins is not incompatible with offering tried-and-true workflows out of the box, designed by experts and known to work well in most cases.
Making an analogy with Lego: you buy a Lego set and it comes with clear, well-designed instructions to build a particular model. If you want, though, you can deviate from the instructions. (You can also buy a Lego bucket with assorted pieces and no instructions; this, I agree, is useful only to a small subset of model builders).
To give a more concrete example or a potential benefit of plugins: Some people here were (are?) working on a sigmoid mapping curve. If I wanted to try it out for myself, I’d need to pull and compile a patched darktable parallel to the main version. If their code is not accepted upstream, it becomes very hard for me to keep using it. A plugin architecture avoids all these problems, both for developers and for users.
Thanks for the detailed explanation. I need to take a closer look at vkdt, it looks awesome!
I’d just like to add that I’m thinking of plugins as one step further than modules: the difference is that plugins can be easily added to the framework without recompiling, and they can be distributed and shared on their own. An example is a web browser, where you can easily add/remove plugins, which may be developed independently of the browser framework.
Does this ignore the concept of a pipeline and having things controlled…if for example one module controlled gamut or made some adjustments and then subsequent modules were integrated to expect that could you introduce all kinds of issues by just throwing in a random plug-in… I can’t think of a concrete example and I know you could mess things up as it is by moving modules around willy nilly but could it not be that you would have to know quite well what each module did with respect to input and output and how that tied in with the other modules… Just thinking out loud…
One of the advantage of using something like Julia is that you have instant access to good math and machine learning libraries. I’m afraid that at some point ML-based processing that is now mostly available in commercial software or hard to use research tools will become almost a requirement.
The CUDA stack looks good but not too sure outside of that. Another issue is the GUI.
For the pipeline side of things I think a node based approach (à la blender) works the best for graphics.
… in vkdt (as in any real time rendering framework, i think it’s the first feature that people implement) you can hotswap your modules. it’ll reload the changed shader code and rerun it in place, keeping your camera position/parameters/etc as they were. i know this is not what you’re asking but it’s great fun for debugging.
If I understand correctly, a pipeline is a special case of a node system. If so, nodes seem a better idea.
Basically, a plugin takes as input image data (in some format, encoding, color space, etc) and a mask, and it outputs a new set of image data.
This seems unnecessarily restrictive. Perhaps a plugin could take any number of inputs, and create any number of outputs. Similarly, any number of UI inputs, and metadata input/output.
I suggest thinking carefully about version control, both of the core system and of plugins. For example, what happens when changes to one plugin breaks another plugin?
Do you want to restrict this to processing raws? Why not generalise to allow any image?
