Future changes to the processing pipeline

I have encountered these before. Seems to me that I can crash PF by changing values or toggling options too quickly in succession.

I’m really happy you like it!

I have changed again the default for mask layers. Now the initial layer in a mask will take by default the output of the associated non-mask layer. The flow looks more or less like this:


This is obviously configurable, but for me this represents a more common default use-case. What do you think?

I will look into this… although hiding the RAW loader layer is not really what you would normally do. I will probably introduce the concept of “input layers that cannot be hidden”.

After a little bit more testing, I think I will merge the changes into stable without waiting too much, if we all agree…

I suspect that this has more to do with threads synchronization. GTK is not thread-safe, and making the UI work correctly with a multi-threaded processing pipeline is a real PITA. Most of the crashes I have been fixing over the past couple of years were related to threaded access to GTK functions… and I still have tons of such issues to fix!

1 Like

Ah, you give me a different experience to consider regarding my random crashes in rawproc. I’m only seeing such with the GTK rendition, maybe one more consideration in choosing my next GUI toolkit…

@paulmiller @afre this is how the “simplified” UI looks like right now:

I would prefer the default to take the input to the mask from the input of the associated non-mask layer rather than the output:


My reasoning for this is that then the mask does not change as you edit the settings on the layer.

I wouldn’t normally hide the RAW loader layer, it was just the simplest way I could get the problem to occur. It happens when I use the shadows/highlight layer, but not 100% reproducably.

To me it could go either way. I understand your method: process input into layer, then dial it back using mask.

It’s been a while since I used photoflow.
Today, I decided to use it on a couple of family photos and I must say I’m impressed.
It seems way faster than before (but I still get the annoying “caching…” that swallows 100% CPU, sometimes).
I got excellent results with a minimal set of nodes, and I was also happy to see that an old preset you did (Edge Sharpening) still works and rocks. Btw, it would be nice to have a tool for that. I lost some time trying to remember which nodes among many were the tweaking ones.
Also, the tone mapping tool gives wonderful results.
Besides the caching thing, sometimes I can’t see the close button from a floating tool (a big one like tone mapping), but that’s not really an issue.
Thanks to all that contribute to this good work (the developer, obviously, but also the testers).
Just a feedback.

1 Like

I’m very happy to hear that, you truly made my day!

The caching is needed to pre-compute cpu-intensive layers in the background, so that the processing time does not explode… ideally this should not be needed, but for the moment that was the best compromise I could find.

Indeed I have been putting quite some efforts into backward compatibility, and it looks like it was done properly :wink:
I have some ideas to use the guided filter for that, and as you say eventually put that into a separate tool.

That should improve with the UI layout I am proposing in this thread. For example in the tone mapping case:

I have just pushed a change that reverts back the code to what you suggest. Moreover, the input list for mask layers will also contain the output of the parent non-mask layer before blending, so that my previous idea can still be realized…

Pre-compiled packages will be available in a short time. I’d appreciate some feedback on this new version, as I would like to merge those changes into the stable branch as soon as things look OK.

Thanks!

1 Like

Will check it out. The branch is ready for merging as far as I can tell.

I’ve given 9c527 a quick try. The user interface works really well, but there are a some strange problems that you may want to fix before merging:

  1. If I add 2 layers and put a mask on the first one, the result is odd:
    11

If I turn the mask off, or explicitly choose the input for the top layer, I get the result I expect.
IMG_9010_TEST.pfi (22.1 KB)

  1. Still not reproducable, but I got the Processing…Caching…Processing loop the first time I tried to add a Shadows/Highlights layer.

  2. Dragging to reorder layers doesn’t seem to work any more.

I pushed a fix for this, the code should now avoid using a mask layer as the input for a non-mask one…

Good catch! I am now trying to fix this… thanks!

Right. What hurts more, though, is the swap usage. When it starts caching and I still have plenty of ram available, only high cpu usage is noticed, and I can perfectly deal with that. But then ram is filled up and it begins to swap. Then my computer becomes useless. The only way to stop that is a hard reset. Not sure if this is an Ubuntu stuff. I will play with swapiness values the next days to see what happens. Currently, swapiness is 60 in my Ubuntu settings.

Been wondering for a long time: could we get a panic button or key combination that would stop operations when things get out of hand? E.g., Esc. After which everything is cached from scratch. It is inelegant but it would save the app from stalling or crashing, and we wouldn’t lose our cool, esp. when we are in the zone. :sunglasses:

Working now.

Another thing I have just noticed - as you select different layers, some controls above the layers list appear and disappear:


This makes editing layers (especially when editing the mask) awkward because the layers move up and down as you select different things so you have to keep re-aiming the mouse.

This should be fixed now, as well as the layers drag 'n drop functionality.

1 Like

Yes, both are now working. :grinning:

I’m still getting the Processing…Caching…Processing… loop a lot. (usually if I add or edit a layer which takes a long time to execute e.g. Shadows/Highlights or RL sharpening). Saving a .pfi then closing the document and reloading from the .pfi fixes the problem, but it can reoccur if you edit any settings (unfortunately, still not reproducable from a saved .pfi state). This happens much more frequently on the simplified-pipeline branch than it does on stable.

I will now look into the caching issues, also to overcome the swapping that @gadolf reported.

Thanks. If I understood, the latest simplified build already mitigates those issues.
I haven’t had much time to test it, but so far haven’t got any freeze (due to swapping. Can it be because the change in swapiness I did from 60 to 1?)
Now it seems to set some kind of memory usage threshold, as the memory doesn’t fill up, even if there’s still room for growth. In the image below, the second indicator from left to right is memory usage. Is this what you expected?


It is taking a lot of time to process this pixel pipe but, at least, the freezes are gone.

What does default (..., blended) mean again? The floating window has in/out input: as well but with sub-image. Should this be duplicated? (Well, the panel doesn’t have sub-image…)