PhotoFlow and sidecar files

Software PhotoFlow
21

I think I have found what is causing the problem. I checked the files on raw.pixls.us and they opened fine on my mac with version 20170608. On examination of them and the files I was having problems with the only difference I could see is that mine had IPTC data. Most of my older E-1 and E-620 files are in an Aperture file system and on export for reworking with photoflow I had the include IPTC data ticked. Sure enough when I exported without the IPTC data the files open fine.

The files with IPTC data do open with version 20170401 so something has changed.

The files (with and without IPTC) and terminal messages are included at the following dropbox link in case you need to looking why the files with IPTC donā€™t work.

Again many thanks.

22

Updating to the latest version of EXIV2 seems to be the right solution. Here is a new version that correctly opens the .ORF files with IPTC data on my OSX system.

1 Like
23

That works for me also. Once again thanks for your help.

24

Unfortunately there seems to be a minor problem with this version (20170610) which I have not seen before. When I export a jpg file the correct colour profile is not attached. So far on testing Panasonic files always have Adobe RGB and Olympus files seem to have no profile attached. The actual colourspace conversion seems to be working fine as if I attached the correct colour profile to the exported file it displays ok in other programs. Sorry for yet another problem!

25

I will check! No need to apologize, I really appreciate the feedbackā€¦

1 Like
26

I fixed the issue with output color profiles, they are now correctly attached both for TIFFs and Jpegs.

That was a major problem, so thanks again for reporting it!

Updated OSX package: https://github.com/aferrero2707/PhotoFlow/releases/download/continuous/photoflow-20170612.app.dmg

1 Like
27

Thanks that works fine.

1 Like
28

Coming back from a wedding, Iā€™ve used DigiKam and PF, to develop my images ending up with 150 of them.
This is a journey ā€¦ :slight_smile:
First the performance and stability are the main issues, but thatā€™s not the topic. I didnā€™t include ā€œlocal contrastā€ and ā€œnoise reductionā€ to speed up a little bit.
I did not succeed in getting good result with some high iso, but thatā€™s not the topic neither.

Beside that, the repetitive questions are a bit boring

  1. A pfi has been detected. Do you want to open it ?
  2. After having exported a jpg file, exiting PF:
    2.1. Modified buffers existing, do you really want to exit?
    2.2. Image ā€œxxx.jpg.pfiā€ contains unsaved data. Do you want to save it before closing?
    The last 2 questions should occur only if I havenā€™t not saved anything, am I correct ?
    I do not get the right tuning the first time, so I did that 3, 4 or 5 times for each single image ā€¦
    If some questions must remain, I think a way to avoid them will be more than welcome.

Here is a confusing case. When we have a pfi plus jpg.pfi like:

When I open the jpg file PF asks the question about name.pfi. It should ask about name.jpg.pfi (at least use it). I want to open the treatment which created the jpg :

If I answer yes, I donā€™t get the actual treatment which has built the jpg. The worst part of it is that I may not notice I use the wrong pfi fileā€¦
If I answer no, PF opens the jpg image without any treatment. The only way to reedit properly name.jpg is to delete name.pfi.

A general topic. From DAM, when opening an image with PF, a new instance of PF is started. Would it be possible, or even desirable, to just open the new image in the current PF instance ? If that would not save any click, that would be more convenient to switch from an opened image to another.

When Iā€™ve finished to tweak all images, I would have loved to be able to batch generate again all the images adding a bit of ā€œlocal contrastā€ and, for high iso, some ā€œnoise reductionā€ before publishing the images.
Still on batch side, I think it would help to batch generate at the beginning a jpg file for each raw. This time, that would speed up the DAM and help to sort out the images.

Among the things which could speed up also the development process is a color picker for curves, white balancing, ā€¦

What did help a lot is the default.pfp, and even more, the ability to copy layers from one image to the other. The copy itself is quite fast ! :slight_smile: , then just be patient, if the previous image is good the next image is (nearly) good too :slight_smile: .

Iā€™ve probably made a lot of mistakes, not used the best method. I would love to learn how to go faster (4 times would be a good start) because all the tools are there to get the best from our images.

Thank you Carmelo !!

1 Like
29

Actually I think the original logic works well. If I want to open the previous image development steps I open the .pfi, if I want to rework the original raw file from scratch I open the .raw and if I want to include or work on the resultant image I would open the .jpg or .tiff. May not be the way other programs work but it seems good to me.

30

Speed is still one of the bottlenecks, and one of the big things on which I need to work once the next release is out. I have some ideas, but they will require some re-factoring of the processing pipelineā€¦ The goal is to get very close to DT and RT for all the tools that are somehow in common.

Today I will have some time for working on the file handling logic, so expect some update by tomorrow :wink:

True, but having sidecar files for the exported images has the big additional advantage of keeping track of how a given image was exactly generated. How the sidecar files are exactly handled is still perfectible though. Anyhow, the whole sidecar files business can be completely disabled via preferences (and is actually disabled by default), in which case the old logic that you describe applies.

Thanks for all the suggestions!

31

Iā€™m making progress on the batch side :slight_smile:

This pfi creation doesnā€™t work on Linux neither :

  1. produce pfi in PFs for all nef
    for i in *.NEF; do /usr/bin/photoflow --batch $i default.pfp PFs/name.pfi; done

However those work pretty well:
2. produce jpg in PFw for all NEF
for i in *.NEF; do /usr/bin/photoflow --batch $i default.pfp PFw/name.jpg; done
3. produce jpg for all jpg.pfi in PFf
for i in *.jpg.pfi; do /usr/bin/photoflow --batch $i name; done
4. produce jpg in PFw for all jpg.pgi in PFf with additional presets
for i in *.jpg.pfi; do /usr/bin/photoflow --batch $i ā€¦/PFw/preset.pfp ā€¦/PFw/name; done

With 2. Iā€™ve no pfi to edit the jpg from Digikam. So either the batch follows the configuration (save sidecar file) or I should use 1. (with name.jpg.pfi), if works, then .3 to produce the jpg.
We see also here that the double extension (.jpg.pfi) complexifies a bit the logic.

1 Like
32

It should workā€¦ Iā€™ll have a look.

33

This should now work in the stable branch, I just need a couple of days to update all packagesā€¦

Thanks for the heads up!

34

This works indeed, I get now the pfi files. But ā€¦ with this version I can just open jpg without pfi. Opening nef (with default.pfp) or jpg.pfi makes PF crash before showing anything. Impression that crashes already with Open Raw Image.

35

Iā€™ve just made a try with Digikam 5.6 which has the function to maintain the name correspondance during renaming operation.
It works for name.jpg and name.jpg.pfi. But doesnā€™t work for name.nef and name.pfi.
Renaming name.jpg to new_name.jpg renames also name.jpg.pfi to new_name.jpg.pfi.
Renaming name.nef to new_name.nef lets name.pfi unchanged.

36

Could you give a try to the latest windows 64-bits package? I have hopefully fixed a number of issuesā€¦

So the solution is probably to propose the .nef.pfi pattern when saving an opened RAW image. What do you think?

37

You are right, .nef.pfi pattern would solve issue of Digikam files renaming. But DK association is one point, PF association (recognition of pfi starting from image) is another one. I would not create any pfi file associated with an original image. I know, Iā€™ve changed my mind on thisā€¦

Let me try to explain. Having played around, in the DK context, which shows only the images (and not the pfi files), I think that is misleading to have a pfi associated with the original image as we donā€™t see the result in Digikam. On the opposite, the expectation is to open it with default.pfp, and not with a not expected and not visible pfi file. Even more, it is then impossible to open the original image with default.php.
There is another reason. When the orignal file is of the types of image that PF can create. In that case opening a .jpg.pfi file will lead to lose the original jpg file.

So the logic I would propose is this one :

  1. Opening an original file (without .ext.pfi), PF opens the image with default.pfp, if any, nothing else. After changes, Saving and Exporting, work as today, but preventing any creation of .ext.pfi and overwriting of the original file.
  2. Opening an image produced by PF (with .ext.pfi), PF opens the original image with the .ext.pfi treatment, exclusively (if PF uses .pfi (as today I think), instead of .ext.pfi, this is also misleading). After changes, Save updates both .ext and.ext.pfi. Export warns that .ext and .ext.pfi already exist and asks confirmation to overwrite them, letting chose a different file name.

What do you think ?

38

When opening a nef file it stays updating. I started it under gdb and let it run 30 mn. It consumed 2.5Gb memory. Iā€™ve tried to kill it, but this did not work properly. Attached the backtrace I could get.
Let me know if I can make anything else.

PF_stop-20170625.txt (4.9 KB)

39

Your backtrace corresponds to the point where the application was being closed, however things went wrong much earlierā€¦ if you can, I propose the following:

  • start the application with gdb, and open the nef file
  • the loading should take few seconds. if the loading phase takes again longer, stop the application with Ctrl-c (I hope this works on Windows as wellā€¦)
  • generate the backtrace, as you did before, with
    thread apply all backtrace
    (redirection to pf.txt clearly does not work, so please paste the terminal output as you did already).

Thanks a lot!

40

Ctrl-c doesnā€™t work. Iā€™ve stopped by PF exit button and also by the windows exit cross. Both trigs the segmentation fault.
PF_stop-20170625-2.txt (4.9 KB)