PhotoFlow: new caching mechanism - TESTING NEEDED!

Still got time to build it and repeat the delete layer test case and I confirm it works now, thanks.
As for the warning, it still shows now and then

(photoflow:15587): Gdk-WARNING **: 08:18:09.017: gdk_window_set_icon_list: icons too large

Later on I’ll do more edits.

EDIT: I completed an edit of the dreamy creek play raw without a crash. It’s very fast. The only thing noticeable happened when I inserted a film emulation layer in the middle of the pipe and chose a film: it took some time to process, at a very high cpu rate. Different from all other layers I manipulated.

This is exactly the kind of thing that the code in the improved-pipeline branch will solve once merged, that is to provide a correct and meaningful default input to all layers (including mask layers).

Good to know. Although the fixes I introduced for the crash upon layer deletion should have no impact on the other crashes that occur when moving sliders very fast. If that happen again, please let me know and provide logs if possible…

Glad to know the processing is getting faster with the new code!

The first time a film emulation preset is used, GMIC has to load and decompress the CLUT from disk, and this step takes a bit of time. However this is only executed once, and the processing should be much faster afterwards. Is that the case?

Correct.

Now I’m trying to edit this play raw but the computer is swapping again and completely freezing, so I don’t have a way to send you anything.
I tried using valgrind the way you mentioned above, but it takes forever to do the initial rendering.
The steps I took before freezing:

  1. Opened the raw
  2. Added a scale layer to fix the angle
  3. Back to the raw dev layer, I set highlights recovery to blend,
  4. then I switched to the advanced tab, and pushed the white level to around 55,000.
  5. Back to the input tab, I started tweaking with the exposure slider and, at that point, ram was filling up and swapping began, forcing me to reboot.

Look at the green indicator, at the top, which is ram usage (screenshot after completing step 4 above)

Confirmed. Moving the exposure slider without any additional layer already increases the memory. There is a big memory leak somewhere, I will look at this later today.

1 Like

There was an important memory leak in the demosaicing phase, which got amplified by the recent changes in the processing pipeline. It should be completely fixed now.

I could reproduce the odd behavior and I started to look into it, but I do not have a fix yet. Hopefully tomorrow…

:clap: It has been bothering for a long time. Thanks @gadolf for providing more concrete evidence; not good at doing that myself. :blush:

2 Likes

I have also fixed the stable branch, so no need to download the more experimental new-caching version.

New packages should be ready by now…

1 Like

ok, thanks. One question: if I want to send you debug info concerning the stable branch, is it just starting with this and repeat all other steps?

git clone https://github.com/aferrero2707/PhotoFlow.git --branch stable --single-branch

Yes, that should be enough.

image

@afre indeed the win64 builds are failing, apparently due to some changes in the MSYS2 pugixml package. I am looking into this right now, more news ASAP…

@afre win64 builds are back, you can find an up-to-date package in the usual place.

Thanks for the heads up!

I still get symptoms of memory leaking (green indicator: ram; purple indicator: swap):

I was just tweaking white level and exposure/blend.

EDIT: Maybe something is different, because after the rampant ram and swap usage (and some good seconds of freezing), the program gets responsive again, but in a more slow pace. Before your fix, once the computer froze, only a hard reset to eliminate it.

Does the RAM/swap increase happen before or after you start tweaking the sliders? How much ram do you have? The software should use a bit more than 1GB of memory, constant over time…

After.

12 GB, where more or less 8 GB would be free, since Gnome uses around 4 GB.

So PhF is taking 8 GB? That’s really too much… could you send me the specific raw developer settings that eat so much memory?
Are you using Amaze or something else? Anything activated in the corrections tab?

Sorry for the flow of questions…

I just opened the image, set a white spot for wb, then started tweaking with white level (forth and back a couple of times), then increased exposure, then get back to while level and do some forth and back slidings, that’s all. I didn’t tweak anything else.
As for the available ram, now that phf is closed, I have this:

image

So, except for swap, that would be more or less the available ram I had at the time I started phf.

Next time could you try to monitor the exact memory usage with top?

top -p $(pidof photoflow)

I will repeat your steps from my side, and see what I get…
Thanks!

I believe this won’t work with an appimage, correct?