PhotoFlow: new caching mechanism - TESTING NEEDED!

Hello @Carmelo_DrRaw

Just tested the new vesion for around 30 minutes…
No crash whatsoever on Windows 10 (CPU I7, 8 gb Ram)
RAW file NEF (Nikon D700).

My deepest congratulations because your software is packed with interesting features!

The zoom (in -out; 1:1 etc ) and pan (toolbars) works fine.
I have tried quite a lot the basic adjustements and their tools look quite responsive on my computer.

2 little bugs:

  • the icon which looks like a thunder lacks a tooltip (its is placed under the curves tool and the uniform fill, to be more precise)
  • the soft proofing setup lacks a way to close its window. When you click on the X button, on the right, nothing occurs (the window remains open). To close it you must press ESC on your keyboard

Here is the screeshot:

SOFT_PROOFING

Relight tool.

Yep, missing a close button. May have mentioned it before.

I’ll take a look tomorrow, its getting a bit late now.

Are you logging the value of tile->ref_count when the assertion fails in pf_tile_unref() in the new version? It might be interesting to know if it is 0 or <0. From a brief scan of the code in tilecache_pf.c, it looks like pf_tile_ref() and pf_tile_unref() are called when the tile pool mutex is locked, but there are other places that zero the ref_count where this mutex is not locked.

I will do it, but this won’t be the same use case as before, since I won’t be executing an appimage, right? Or it doesn’t matter?

Anyway, today it crashed when I
1 - Opened this raw (IMG_1093.CR2) from this thread;
2 - Added a Basic layer;
3 - Increased exposure by 8 EV;
4 - Exported as tiff 32-bit rec2020.
It crashed while exporting. It left behind the tif file, but a malformed one.

For Ubuntu 18.04, that would be:

sudo apt-get install libexiv2-dev liblensfun-dev libgtkmm-3.0-dev libpugixml-dev libtiff-dev libpng-dev libjpeg-dev gtk-doc-tools gobject-introspection libgtkmm-2.4-dev

I couldn’t successfully build it:

buildoutput.txt (1.2 MB)

Are you sure? Here a Dell Inspiron N4050, the only plus being 12 GB of ram. At 1:1 I confirm the slugishness.

I can confirm I only have 4GB RAM and cheaply replaced parts. :stuck_out_tongue:

I can also confirm that my earlier usage isn’t reproducible: 1 because I haven’t tried again and 2 because the log didn’t record the moments before the crash or instability.

Overall, I like the new direction, since PF won’t be stalling with every minute change in the settings, which has been extremely off-putting.

1 Like

I removed the offending compiler flag… could you pull the latest version and recompile?

The AppImage and the user-compiled code should behave in the same way, at least if the issue is in the thread logic and not due to some memory corruption…

Thanks!

Another error:
buildoutput.txt (1.2 MB)

Here’s the log from a crash of 3f5a3:

(I opened a NEF file, added a shadows/highlights layer and then adjusted sliders, zoomed and scrolled like crazy)

photoflow_log.txt (48.3 KB)

photoflow_crash.txt (72.2 KB)

Edit: Another crash log
photoflow_crash2.txt (17.8 KB)

Ok, I have temporarily removed all treatment of warnings as errors, so that the compiler will be less strict. Meanwhile I will try to fix the warnings…

Sorry for this…

1 Like

Still building, now I get this:
buildoutput.txt (1.7 MB)
EDIT: If you think a remote session will be easier to you, just pm me

Hold on… I am building the code on Ubuntu 18.04 myself, will get back to you once I have fixed those issues… it makes no sense to waste your time sorting out compilation problems! It will hopefully not take too long…

It looks like my code is not adapted to the newer version of LendFun that you have installed… please try this, it worked for me:

BUILD_TYPE=RelWithDebInfo CMAKE_EXTRA_PARAMS="-DBUNDLED_LENSFUN=ON" ./build_all.sh

I have added some more logging messages. I need to understand if the ref_count=0 is simply due to an extra unref, or if it is due to a wrong tile disposal…

The new package is on git as usual.

Thanks!

Thanks for checking and for the encouraging words! I think that right now there is probably even too much, so I am trying to simplify the tools… but first I need to get what exists to run rock solid.

Photoflow 85565:

(same method as before, open .NEF file, add shadows/highlights, zoom to 1:1 and scroll around with the track-pad until it crashes. I’m changing direction a lot so it doesn’t have time to complete the window refresh)

photoflow_log3.txt (34.5 KB)
photoflow_log4.txt (18.9 KB)
photoflow_log5.txt (35.0 KB)

@paulmiller @gadolf the crashes are all similar, which is quite good. Next thing I would like to ask you to check is if the reference counting mismatch is due to thread synchronization problems, or just some naive programming mistake…

You can turn off multi-threading by setting the following environment variable:

VIPS_CONCURRENCY=1

Thanks!!!

Just tried with VIPS_CONCURRENCY=1. I’ve been scrolling for 15mins, no crashes yet.

I was able to build it but now nothing happens when I try to execute it:

gustavo@N4050:~/PhotoFlow/build/Relwithdebinfo/bin(new-caching)$ valgrind --tool=memcheck ./photoflow >& /tmp/phf.log
gustavo@N4050:~/PhotoFlow/build/Relwithdebinfo/bin(new-caching)$ 

EDIT: I tried executing phf without valgrind and that’s what I got:

gustavo@N4050:~/PhotoFlow/build/Relwithdebinfo/bin(new-caching)$ ./photoflow 
./photoflow: error while loading shared libraries: libvips.so.42: cannot open shared object file: No such file or directory

So it is likely related to thread synchronization… let me have a look at the code once more. Maybe @gadolf will also catch something through valgrind.

@gadolf indeed you should set he following variable before invoking photoflow, given that VIPS was compiled by the build script:

export LD_LIBRARY_PATH=$HOME/PhotoFlow/build/VIPS/Relwithdebinfo/lib:$LD_LIBRARY_PATH