PhotoFlow: new caching mechanism - TESTING NEEDED!

Carmelo_DrRaw · October 5, 2019, 4:32pm

It should, the executable name is just a bit different (should be photoflow.bin or photoflow.real, if I remember correctly).

The appimage is really just a package that runs the executable in a custom environment.

gadolf · October 5, 2019, 4:35pm

There is photoflow.bin and PhotoFlow.AppIm.
The first one?

Carmelo_DrRaw · October 5, 2019, 4:42pm

photoflow.bin for sure. You should see it in the list when running top (or htop) without parameters.

gadolf · October 5, 2019, 5:08pm

Carmelo_DrRaw · October 5, 2019, 6:22pm

Which branch is it? I just tested the new-caching appimage, and processing a Nikon 810 RAW with spot WB and exposure correction required about 600MB on my Linux system.

gadolf · October 5, 2019, 6:35pm

Humm, I think I was running an older version, I’m really sorry.
You know what may have caused this? The desktop integration thing.
When I hit the Super key and type photoflow, I see this:

The one with the nice icon (“1”) is the one I was using. The one with the ugly icon(“2”) is the one that corresponds to the latest stable version.

Really sorry to have wasted your time on this.

Carmelo_DrRaw · October 5, 2019, 6:49pm

No problem! I will check what is going on with the desktop integration… I must admit I am not really using it myself.

gadolf · October 5, 2019, 7:03pm

I wouldn’t bother with that if I were you. This older version but nicer icon must be a left over from some manual tweak I did in Ubuntu in the past. Let it be. Just double check the program version on the next print screens users like me post around.

afre · October 6, 2019, 8:09pm

Latest stable doesn’t run because

Carmelo_DrRaw · October 6, 2019, 9:03pm

That’s strange… I will have a look tomorrow.

Carmelo_DrRaw · October 7, 2019, 3:43pm

@afre The problem with the missing dll should be fixed now, could you please check?

@paulmiller @gadolf I have also fixed the “assign profile” bug in the new-caching branch, and prepared new packages. There is still something wrong with the Spot WB that I am trying to fix…

afre · October 7, 2019, 6:27pm

It opens properly now. I noticed an unrelated problem. I may have reported this before. When I open the app, there is an open dialog. If I close it without opening a file, I am left with an empty window without any GUI. I cannot do anything besides close the app. Same as when I open a PFI of which the original file no longer exists. The app just continues to load without error and without a way to reenter the open dialog.

Carmelo_DrRaw · October 7, 2019, 7:37pm

Yes, I am aware of this… however, one of the next steps in the UI development will be to replace most of the buttons with menu items. At that point, the problems you describe will be solved, and we will gain a bit more of space for the preview and the layers list.

I will start working on this as soon as the recent improvements in the processing pipeline will be merges into the stable branch.

afre · October 7, 2019, 7:45pm

Great! If one closes the open window or opens a malformed PFI or one that goes nowhere, the user has no choice but to quit the app. Not a good experience for a first time user or a non-techie.

Carmelo_DrRaw · October 8, 2019, 8:47pm

@gadolf @paulmiller @afre
New update:

I fixed the image updating when clicking on the preview area in spot/area WB mode
I added a configuration option for setting the size (in MegaBytes) of the global tile cache. The default is 1000 MB, which gives a reasonable speed without eating too much RAM. A larger size does not necessarily provide a huge speedup, it depends on the image size and the set of tools being used.

Packages are available for download in the usual place.

paulmiller · October 9, 2019, 1:23pm

Assign Profile: working
Spot/Area WB mode: working

When I check the settings, the tile cache size is set to 10,000 not 1,000.

Carmelo_DrRaw · October 9, 2019, 2:11pm

I just fixed this as well. Thanks for checking!

Next step is to re-introduce the histogram visualization, after which I think the code might be good to be committed into the stable branch.

gadolf · October 9, 2019, 5:34pm

(To avoid the same mistake as before, I’m running new-caching-24343 from 2019/10/08)

Crash when trying to open the country sunset play raw:

Some notes:

I have already a .pfi file in the folder from a previous edit, but I chose to open the .CR2 file;
A couple of days ago I clean up my / partition because it was getting full. I did a couple of packages uninstalls, although I don’t remember having uninstalled some of the build required packages. But I always feel uneasy to do sudo apt autoremove (i had problems with this once in the past)
Follows gdb output last lines and log.

RawImage::RawImage() finished
… layer “RAW loader” finished
Building layer “RAW developer”…
RawPreprocessorPar::init_wb_coefficients() called
… layer “RAW developer” finished
PF::Image::do_update(): pipeline #0 updated.
PF::Image::do_update(): preparing pipeline #1
PF::Image::do_update(): updating pipeline #1
PF::new_operation: creating operation of type “raw_loader”
PF::new_operation: creating operation of type “blender”
PF::new_operation: creating operation of type “uniform”
PF::new_operation: creating operation of type “raw_developer_v2”
PF::new_operation: creating operation of type “blender”
PF::new_operation: creating operation of type “uniform”
Building layer “RAW loader”…
… layer “RAW loader” finished
Building layer “RAW developer”…
RawPreprocessorPar::init_wb_coefficients() called
OpParBase::build_many_internal(): adding tilecache for output image #0, padding=0
phf_tile_pool_init(): called
OpParBase::build_many_internal(): added tilecache for output image #0, padding=0
… layer “RAW developer” finished
[New Thread 0x7fffda7fc700 (LWP 13340)]
PF::Image::do_update(): pipeline #1 updated.
PF::Image::do_update(): preparing pipeline #2
Pipeline: force_rebuild set to true
PF::Image::do_update(): updating pipeline #2
PF::new_operation: creating operation of type “raw_loader”
PF::new_operation: creating operation of type “blender”
PF::new_operation: creating operation of type “uniform”
PF::new_operation: creating operation of type “raw_developer_v2”
PF::new_operation: creating operation of type “blender”
PF::new_operation: creating operation of type “uniform”
Building layer “RAW loader”…
… layer “RAW loader” finished
Building layer “RAW developer”…
RawPreprocessorPar::init_wb_coefficients() called
… layer “RAW developer” finished
PF::Image::do_update(): pipeline #2 updated.
IMAGE_REBUILD finished
Pipeline: force_rebuild set to true
IMAGE_REBUILD started
PF::Image::do_update(): preparing pipeline #0
PF::Image::do_update(): updating pipeline #0
PF::Image::do_update(): pipeline #0 updated.
PF::Image::do_update(): preparing pipeline #1
PF::Image::do_update(): updating pipeline #1
Building layer “RAW loader”…
… layer “RAW loader” finished
Building layer “RAW developer”…
RawPreprocessorPar::init_wb_coefficients() called
OpParBase::build_many_internal(): adding tilecache for output image #0, padding=0
phf_tile_pool_init(): called
OpParBase::build_many_internal(): added tilecache for output image #0, padding=0
… layer “RAW developer” finished
PF::Image::do_update(): pipeline #1 updated.
PF::Image::do_update(): preparing pipeline #2
PF::Image::do_update(): updating pipeline #2
PF::Image::do_update(): pipeline #2 updated.
IMAGE_REBUILD finished
IMAGE_REDRAW_START started
IMAGE_REDRAW_START finished
IMAGE_REDRAW started
[New Thread 0x7fffdb7fe700 (LWP 13341)]
[New Thread 0x7fffdbfff700 (LWP 13342)]
[Thread 0x7fffdb7fe700 (LWP 13341) exited]
[Thread 0x7fffdbfff700 (LWP 13342) exited]
[New Thread 0x7fffd9ffb700 (LWP 13343)]
[Thread 0x7fffd9ffb700 (LWP 13343) exited]
[New Thread 0x7fffbffff700 (LWP 13344)]
[Thread 0x7fffbffff700 (LWP 13344) exited]
[New Thread 0x7fffbffff700 (LWP 13345)]
[New Thread 0x7fffd9ffb700 (LWP 13346)]
[New Thread 0x7fffdbfff700 (LWP 13347)]
[New Thread 0x7fffdb7fe700 (LWP 13348)]

Thread 29 “worker” received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffbffff700 (LWP 13345)]
0x00007ffff697f645 in g_mutex_lock ()
from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
(gdb) set pagination off
(gdb) set logging file log.txt
(gdb) set logging on
Copying output to log.txt.
(gdb) info threads

log.txt (46.9 KB)

Carmelo_DrRaw · October 9, 2019, 7:42pm

There is a problem with the tile allocation. In the log I only see

phf_tile_pool_init(): called

without any further message afterwards. What is the tile cache size in your settings?

This means I need to better handle the errors in the tile cache initialization…
Meanwhile I have added some more messages to better understand the source of the problem.

gadolf · October 9, 2019, 9:38pm

I’ve built it and this time I didn’t have any issues and could complete an edit until the end.

It was 1 GB and I changed it to 2 GB before starting this edit.