Help for testing latest PhotoFlow version (particularly on Windows)


(Carmelo Dr Raw) #61

At this point, a gdb back trace would again be very helpful! Even if more rare, I still would like to get rid of this residual issue… and maybe the other one with the recovery file is related and will be fixed at the same time…

Thanks!!!


(Silvio) #62

@Carmelo_DrRaw : Even if more rare, I still would like to get rid of this residual issue…

Nope… :frowning:
At present, I have tested much better the clone stamp tool. And I was wrong in that it looks like this current version is prone to crash as the previous ones (not less…).
Sometimes it crashes sooner. Sometimes later but the end result is the same…
I suppose on Windows the threading stuff is different from Linux - Mac. Therefore, everything works fine for you…
I have tested this crash on Windows 7 + 8.1 + 10 versions on 3 different computers all of them with CPU Intel I7, 8 gb Ram, Nvidia graphic cards.

Here you can get two backtraces [1] [2]: Windows 7 - 64 bit.
The gdb tool acts odd IMHO because whenever I press enter PhotoFlow always continues to writing into the log text file. Generally, the gdb tool should stops sooner or later its backtrace into the log txt file…
In short in the end I have always closed myself the gbd tool by pressing Ctr+C.

BTW, on Windows 7 (64 bit), PhotoFlow sometimes acts weird :slight_smile:
I suppose it is related to the underlying Gtk backend (I suppose you are still using gtk 2 to be compatible with Gimp 2.8 and 2.9.5).
For instance:

  1. I have added a new layer > chosen the detail option and the sharpen tool.
    In essence, while moving the slider for the radius I was able to move it even though my cursor mouse was extremely far away, at the very end of my display.
    I have tried to repeat this later with a different image but this time everything was right
  2. When I have crashed an image I was unable to click OK (to restore it) or NO. The window to allow me this option was freezed.
    I have tried to repeat later this weirdness but everything was fine…

BTW, as usual PhotoFlow crashes when you open many different Nef images in a row.
It is really on random. Sometimes it crashes with the second image. Others times it crashes later.

[1] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/log_1_stamp_clone_crash_20170128.txt
[2] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/log_2_stamp_clone_crash_20170128.txt


(Carmelo Dr Raw) #63

Maybe, but your tests have definitely spotted another place in the code where a mutex was needed but not used.

One more version for testing, hopefully the good one: https://filebin.net/d9wo6to2i2muaypi

This will be the next thing to investigate, once the clone/stamp tool has become rock-solid :wink:


(Silvio) #64

Hello,

Currently, my testing with this new build is extremely easy because it is definitely, by far, the worst :slight_smile:
Nutarally, I am joking :slight_smile:

Windows 7 - 64 bit
Cpu: Intel I7
Ram: 8 gb
Gpu: Nvidia graphic card

I have tried for once to clone some jpeg images.
They are taken with a smartphone: Galaxy s5 mini.
As soon as I try to clone them, PhotoFlow crashes.
Here is the backtrace [1] and the video [2] showing my attempt

As usual I have also tried with my Nikon D700 Nef images.
Here is the backtrace for this crash [3]

This version even crashed at the very begining when trying to open an image.
The window to open them was stuck (freezed)
Here is the backtrace [4] I have managed to get (well, sort of…)

[1] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/bis/jpeg_crash_clone_20170129.txt
[2] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/bis/JPEG_CRASH_CLONE_TOOL_20170129.avi
[3] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/bis/nef_crash_clone_20170129.txt
[4] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016-01_29-logs/bis/crash_while_opening_image.txt


(Carmelo Dr Raw) #65

Improvement through degradation is my new slogan!!! :wink:

Seriously, this issue is starting to drive me nuts… :thinking:

There must be something still wrong in the code, otherwise it would not crash… but I cannot really find the problematic piece of code.

So I added again some console message, trying to trace down the places where the container structures get corrupted. Here is the new package: https://filebin.net/wxw3jhag2g4cboir
I have a Win7 machine at home, but there I cannot reproduce the issue… so I would need the full console output, in addition to the GDB backtrace.

Sorry for this lengthy debugging procedure, but I really don’t know what possible shortcuts to take.


Concerning the problem with opening several big files, I suspect that this is due to the fact that the executable is 32-bits, and that the program allocates quite some memory at loading time and then keeps the RAW image data in memory for later used.
I have reduced the amount of memory that is temporarily used during loading, hopefully this will help at list a little bit. Next step will be to make s 64-bits executable.


(Silvio) #66

Hello ! :slight_smile:

I have tested today’ s version on 3 different computers:
Windows 7 + Windows 8.1 + Windows 10
All of them with Intel I7 + 8 gb Ram + Nvidia graphic card

I have always tested jpeg images. Otherwise PhotoFlow is really slow when using the clone tool. Gimp 2.8.16 is blazing fast with the same jpeg images and its clone tool.

Needless to say, I am looking forward to testing the 64 bit version : To take advantage of my 8 gb Ram :slight_smile:

As usual it is extremely easy to crash PhotoFlow with the clone tool (on all Windows versions…).
Sometimes it crashes soon after a few clicks. Sometimes it takes quite a lot of testing. It occurs extremely on random…
You need to click very fast while using the clone tool : to “choke” PhotoFlow, which is unable to store all these inputs I suppose…
My jpeg images were small as size (1-2 Mb) and I have crashed PhotoFlow with differente jpeg images.
As mentioned above, with Nef images (8-9 Mb) the clone tool is too slow to work with and I didn’t test them today. With 32 bit softwares I don’t think it does much sense to work with Raw big images :slight_smile:

Here you can take a look at 3 gbd txt files (Windows 7 + 8.1 + 10) [1] [2] [3]

I have also uploaded the prompt messages for Windows 7 and W. 10 (but they do not like useful from what I have gathered looking at them…).
Here it is the link [4] [5]

[1] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016_01_30-logs/gdb_20170130_windows_7_clone.txt
[2] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016_01_30-logs/gdb_20170130_windows_8.1_clone.txt
[3] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016_01_30-logs/gdb_20170130_windows_10_clone.txt
[4] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016_01_30-logs/prompt_20170130_windows_7_clone.txt
[5] https://dl.dropboxusercontent.com/u/3095134/PC_TRUCCHI/BUG_REPORTS/2016_01_30-logs/prompt_20170130_windows_10_clone.txt


(Carmelo Dr Raw) #67

@Silvio_Grosso the last hours seem to have been fruitful… following the description of your crashes and using valgrind under Linux, I’ve been able to reproduce the crash and catch the reason.

Hopefully this new version of the clone/stamp tool is finally crash-free: https://filebin.net/xo6pa3u4if0ojgzm

I have also simplified a bit the code, with the goal of making the tool more responsive… maybe you can give me your impressions.

Thanks!


(Silvio) #68

Hello Carmelo_DrRaw ! :slight_smile:

Just tested today’s new build:
Windows 7 - 64 bit
CPU: Intel I7
Ram: 8 gb
Gpu : Nvidia graphic card

At present, it does NOT crash anymore !
Congratulations for your last improvements :slight_smile:

@Carmelo_DrRaw: I have also simplified a bit the code, with the goal of making the tool more responsive… maybe you can give me your impressions.

As usual it is not smooth. You do notice this behaviour when you clone big images.
For example, with jpeg images more than 2 Mb. Generally, it is really a pain…
Needless to say with Nef images you can only clone small parts of your images unless you have a lot ot fime to wait for PhotoFlow procesing your changes…
When you zoom-in or zoom-out, while cloning, it is even much worse.

Sometimes you even have weird results while waiting:

This being said I do not think it is your fault as regards this slowness.
I have tested quite a lot Krita [1] on Windows.
For starters, they provide 64 bit installers.
The threading stuff is excellent: It was one of the first implementations they did.
In addition, with Krita the canvas (zooming, panning etc) is handled through OpenGL (by way of your graphic card).
On Windows, due to the different file system (NTFS), Krita is slower than on Linux though.

I am not a software developer but I know for sure that usually fine-tuning a programme to improve its speed is by far the most difficult and time-consuming task :slight_smile:

[1] https://krita.org/en/


(Carmelo Dr Raw) #69

I wonder is something could be wrong elsewhere in your windows set-up… today I tested the smae windows version using wine running inside a Linux virtual machine with as little as 1GB of RAM.

Here is the result of editing one D810 RAW file with the clone/stamp tool: https://www.dropbox.com/s/0m8gh48qz1dcq0f/clomne_stamp_test_wine_1GB_ram.mov?dl=0

Everything runs quite smoothly as soon as the caching of the RAW file is finished (the status indicator goes from orange to green), even if I inserted a curves layer below the clone/stamp.

Also, in terms of memory 1GB is more than enough to process a D810 file, the peak memory usage reported by the program being about 350 MB…


(Silvio) #70

Hello !

Just tested yesterday’s version on Windows 8.1 (64 bit):
CPU Intel I7
RAM 8 gb
GPU Nvidia Graphic card
This computer even has got an SSD (not the usual HDD)
When testing PhotoFlow only the anti-virus was running. Namely Kaspersky Internet suite.

With a jpeg image 1,2 mb and the size of the stamp tool 120 a single “stroke” of the clone tool took 1:28 minutes. RAM usage was always around 50-60%

With all my images I have noticed that PhotoFlow at first is always extremely fast: no problem at all indeed.
However, as long as you continue working with the clone tool it becomes more and more slow. After a few minutes it is so unbearable slow I always stop my testings…
It looks like PhotoFlow “stores into the memory” all my changes and when there are many changes in the current image with the clone tool it is unable to cope with them any longer.
This version crashed on Windows 8.1 twice: With a Nef image and, later, at the very beginning with a jpeg image (Segmentation fault). However, now it is indeed rock-solid compared to the past ! :slight_smile:

On Windows 7 I have tried the same Nikon D810 image of your video but it is too slow to work with…

I am not a developer but I am not surprised that PhotoFlow.exe runs fast on Linux with Wine because I have read this same remark for others .exe in the past (they were faster on Linux - Wine than on Windows).
Perhaps, It might be the different file system, for instance.
In addition, on Linux you don’t have any anti-virus running in background.

BTW, Gimp 2.8.16 (64 bit) version is extremely fast with the same jpeg images when using its clone tool. Compared to PhotoFlow there is no contest. Aside from speed, Gimp does not become slower when I work very long with its clone tool.
As mentioned above PhotoFlow at first is extremely fast and responsive. It is only a few minutes it becomes extremely slow.


(Carmelo Dr Raw) #71

There is a remarkable difference between the clone/stamp tool in Gimp and PhF: the PhF one is non-destructive! What you mentioned is actually true: the tool is literally “storing” all your gestures in order to reply them on-the-fly when needed. That’s probably why it gets slower the more strokes you add.

At the moment I have no solution for that, and therefore I would say that the PhF clone/stamp is fine for small retouches, not for fixing a whole picture.

On the other hand, the big advantage is that if you do changes to the layers below the clone/stamp, those changes will be automatically be taken into account in the areas that you cloned. With gimp, if you change something you need to redo the cloning work completely…

This might sound academical, but there is in fact a practical reason why this approach can provide better results: the cloning process should preferably be one of the last steps in the editing, and specifically one should avoid any increase in the global or local contrast (curves, levels, sharpening) after the cloning, otherwise the transitions around the cloned areas will become more visible.

PhF provides a way to put the clone/stamp at the top of the layer stack, and still be able to tweak the underlying layers without having to redo the cloning each time you make some changes. But this has its own drawbacks…


(Silvio) #72

Hello Carmelo DrRaw ,

@Carmelo_DrRaw: At the moment I have no solution for that, and therefore I would say that the PhF clone/stamp is fine for small retouches, not for fixing a whole picture.

Yep, I totally agree.

Generally speaking, with Raw images, in truth , IMHO, I am not even sure a “clone tool” is really “useful”.
In short, when you decide to shoot Raw, especially as a photographer who works with full-frame Reflex (Nikon D700, Nikon D810) you are supposed to take pictures which do not need much fine-tuning later.
At work, I only take macro pictures (plants diseases) and I have never needed the clone tool later (only some basic color corrections) :slight_smile:


(Assaf Toledo) #73

Maybe I miss something but I don’t see why PhF’s approach is expected to be slower when more and more gestures are added. If you maintain a cache of the output of the clone/stamp layer and update it after each time that the user added a gesture, then adding a new gesture requires only a single update to the cache. Consequently, adding the 2nd gesture or the 20th gesture shouldn’t make a difference at all - in each case there is a single update to the cache of the previous sequence of gestures. So the only time when the number of gestures should make things slower is when all gestures need to be replayed, like when the PFI file is opened, because then the whole chain of gestures needs to be applied from the first to the last.


(Carmelo Dr Raw) #74

The fact is that there is no cache of the clone/stamp output. The pixels are re-computed each time something changes (a strokes is modified/added or one of the underlying layers is modified), so the complexity of the computations depends on how many strokes have to be rendered.

The issue with caching is that as soon as you have a relatively large number of layers, it requires quite some memory. At that point, you need to swap to disk, and deal with all the thread synchronisation issues when accessing the cache in read/write mode.

PhF approach (but this is really the approach of the underlying VIPS processing engine) is to limit as much as possible the amount of caching, and instead let the CPU re-process the pixels when needed. So the goal will be to make all tools as fast as it is reasonably possible, and to re-computation in most of the places.

Still, some “slow” tools like the RAW processor keep a cache of their output, which gets computed in the background. This cache is either kept in memory or on disk, depending on the size (I have a 500MB threshold for this, although is memory cannot be allocated the system falls back to disk caching anyhow).

However, the cache is only accessed when its processing is completed, so that there are no read/write synchronisation subtleties to take care of…


(Kevin Payne) #75

I have a bug with Uniform Fill layers. It seems that I can save a preset and when loading it again, the colour of the layer is correct, but the GUI hasn’t been updated:

so If I want to adjust the colour, I have to start from the beginning again.


(Carmelo Dr Raw) #76

@paynekj Hi Kevin, thanks for catching this! It should not be too hard to fix, will look into that as soon as I manage…


(Carmelo Dr Raw) #77

@paynekj Hi Kevin, could you test this new version and check if the problem with the uniform fill tool is solved for you as well?

Thanks!!!


(Kevin Payne) #78

I’ll try it this evening from home, Firefox is complaining about filebin.net’s security certificate and there’s corporate security getting in the way as well.


(Kevin Payne) #79

@Carmelo_DrRaw Thanks, that works as required now. :slight_smile:


(weyland) #80

I’ve tried the version 20170227 on windows 10.
It crashes sometimes.
I’ve attached the gdb messages. But maybe you need only them starting backtrace command ?

Fail20170315-2.txt (231.2 KB)