it’s pretty dumb now. it just processes the whole thing. there’s a #define in the demosaic/main.c file that switches to half size demosaic (i do that when on the intel laptop). need some smarter way to switch this (maybe using the region of interest negotiation phase). but to answer the resizing/performance question: it doesn’t really depend on resolution any more (in contrast to darktable). not really motivated to make the pipeline depend on resolution now, but with more and slower modules this may become necessary again.
Personally I consider the possibility to interactively add/remove/reshuffle the modules the #2 top priority (#1 being processing speed). Defining the pipeline layout through scripts and/or editing configuration files will IMHO let 2/3 of the potential users walk away immediately…
I still find the layers/groups/masks paradigm quite efficient in terms of the amount of information you need to put in the UI to properly describe the node graph. But I am open to any suggestion!
I fully agree on this, and that’s also how I implemented things in PhF. On the other hand, UI panels should probably be aware of which tool the are controlling…
The connection between modules and UI in PhF is made through node “properties”. When for example a slider is moved, the UI widget changes the value of the associated “node property” object, which in turn triggers a re-computation of the pipeline. During computation, the node fetches the new value from the property.
Does it make sense?
yes, that makes a lot of sense. such a subsystem still needs to be put in place. there’s a similar mechanism in darktable, but with additional complications due to multithreading. is PhF serialised in that regard? in dt, there is always gui thread, full pipeline, preview pipeline, and potentially an export thread working on potentially the same data. currently dt_graph_t are not thread safe (because fast enough anyways, i was hoping at least). the node properties would be the parameters here (see module/params file). these are modified using dt_module_get_param or const float *dt_module_param_float for instance. computation gets these values as uniform buffers.
for me at this point, too. still trying to find out what would be the best pipeline, which modules should go into the linear part, which need a more perceptual space or work best in display referred. i’m anticipating that end users won’t want to go through all that trouble again, at some far point in the future when there might be useful software employing this computational pipeline. so the gui needs to be super flexible and configurable.
From my experience:
- most of the color/tone manipulations should be done on linear RGB data
- some tools (like edge-aware blurring) might work better in log space - in that case, the module should convert the input from linear to log, do its job, and then convert back the output to linear
- I have no example of a tool that requires display-referred input. Some tools can however generate display-referred data, which can be further manipulated by other general-purpose tools (like RGB curves)
- I am staying more and more away from Lab as an editing space. However, some Lab-like color model is still needed to define things like Hue and Saturation. I would propose to go for Jzazbz or HDR-IPT, as they provide a more modern and robust model than Lab, which is not limited to display-referred ranges.
In PhF the GUI runs in the main thread, while the pipeline re-building and processing runs in a separate thread. The parameters are modified by the GUI and fetched by the processing. Therefore, a parameter could potentially be modified by the GUI while the processing is still ongoing, leading to weird results. However, I did not consider this a big issue because the pipeline will be anyhow fully re-computed when a parameter is changed, so the corrupted output is only temporary…
Another observation - If a particular algorithm needs to operate in a colorspace other than linear RGB, it may be best (even if it has a small performance impact) to transform to the operation’s needed colorspace at the input to the operation, and return it to linear RGB at the output. That way its position in the pipeline becomes more flexible. (It may have an ideal position in the pipeline, but not a required position in the pipeline, whereas if it required a nonlinear RGB or non-RGB-at-all colorspace as input or output, that would fix its position in the pipeline.).
Me too. That’s the impetus behind rawproc.
For processing single images, the “pipeline” (or toolchain, as I usually call it) organization is clean and efficient regarding UI display space. Most toolkits have a tree widget, reflective of disk media subdirectory organization, and I found that to be quite useful in managing my processing toolchain. When I decided to include a group tool to collect multiple operators working on a single copy of the internal image, the tree just gained “grand-children”, no muss or fuss.
It might be worth studying the node graph implementations of Natron and Blender; there appears to be a lot of work behind those that would be informative of this effort.
…but we are nerds trying to develop processing pipelines. this is a different use case than processing 2000 images from a specific photo shoot. in which case you might want to setup some pipeline for one shot and re-apply it in similar shape to a group of other images, and then only tweak a few key settings. potentially hiding all of the complexity that is irrelevant at a certain stage.
good thing is that imgui is really low profile… it’s very easy to generate the ui adaptively.
fwiw there are node graph editor frontends for imgui Node Graph Editors with ImGui · Issue #306 · ocornut/imgui · GitHub but i feel like this would probably be overkill.
(personally i think tree widgets are ugly, because of the tiny arrows you sometimes need to hit with your mouse, and because they usually come with some weird kind of indentation that makes the gui look unbalanced because sliders are shorter depending on tree depth etc)
That’s if you intend to hang all the widgets associated with the tool on the tree item. My implementation has a separate parameters pane with all that, and selecting a tool in the tree shuffles the paramters pane so that tool’s sliders are on top.
I think the main point is, does the serial nature of still image processing require consideration of a node graph organization, or would a list-oriented organization suffice?
I might end up utilizing a fixed-pipeline derivative of this for a future version of Filmulator. So I don’t care about being able to reorder the nodes.
Not sure if I would use the imgui stuff; apparently Qt 6.0 will let me embed Vulkan in Qt Quick.
This! There is maybe a steep learning curve to using these pieces of software, but I’ve always found their node based approach very intuitive.
Also I believe their modules are fairly (if not completely) decoupled from their GUI.
And additionally on the topic of “what goes where”, your GUI could reflect the type of operation that is permissible by a certain node. I mean: if it requires linear RGB input, it should only be possible to match certain outputs to this node.
You could also go free for all, but then the user should have a pretty good idea about what is going on internally.
A pretty large downside to nodes is that they require a lot of screen space. With say blender it’s fine because a small preview works for most of the blender use cases. For photography I like my preview large.
I feel, perhaps wrongly, that nodes are good when you’re building something that will ‘last’. Say a material in Blender, you’ll use it all over the scene and perhaps store it for other projects. Much of photo processing is custom tweaking. With this line of thinking nodes are better suited for building auto apply profiles than for actually editing on specific photo.
i agree. so maybe have a user facing node editor to develop certain pipeline templates, and then these can be applied and only a few relevant sliders could be tweaked. i was going to call node editors in this context “bloatware”, but while still working on pipeline order it’s actually fairly annoying to always change the input text file to reorder modules… and since there are already imgui node graph implementations this might not be much work.
This would make it a sort of modal raw developer. Despite being a vi user I’m not so confident its a great idea for photo editing. I’d really like to find out though, because it could be great!
random update: i have wired fake-lighttable mode with thumbnails, to get a sense of speed there. still need to battle-test it with a few thousand images (wget -r raw.pixls.us or so).
rawspeed loading becomes a bottleneck during thumbnail creation. most of this could be i/o but it’s also an issue on my system with nvme ssd. need to experiment with interleaving disk io/cpu processing/gpu upload/gpu processing in multiple threads. loading a raw takes anything between 10–5000ms.
short video about image thumbnail list + entering darkroom mode (gui is completely constructed from scratch/config file for the specific processing graph which is loaded per image when you enter dr mode, also the whole full-res image is processed all the time, even when just panning around):
thumbnails are pre-created for this video (being lazy with threading i made a command line utility for this test). also you’ll see that it really isn’t a raw editor but a performance testing prototype (no colour management whatsoever, doesn’t clip negative values in noisy images, no highlight reconstruction, doesn’t even flip image orientation, thumbnails destroy aspect ratio, …).
This looks awesome!
Whoa! What?!
That’s ludicrous speed.
I see it’s the same old Karfiol laptop.
Too bad that I am too stupid to compile this.
I’ve compiled vkdt today. I get funny stuff with the amdgpu driver. It loaded at least once and that was really impressive. The rest of the time I got funny colors and a shutdown of the system.
It is probably a bug in the amdgpu driver. Lets see if someone from AMD will fix it:
https://bugzilla.kernel.org/show_bug.cgi?id=205089