PhotoFlow and sidecar files

What you can already do (apart from possible windows-related problems that I need to debug) is the following:

photoflow.exe --batch img.nef preset.pfp %name%.pfi

The special %name% prefix will be automatically expanded to the name of the image file, without extension. In the example above, an img.pfi should be created…

The tif or jpg part in “jpg.pfi” doesn’t describe actually the intention or the work done on the image. In your example I would suffix the path or the name “_M” for master or “_W” for web. But we can have plenty of other goals which are not described by jpg or tif. doing this each pfi file keeps its own *_M or *_W piece of information.
Only the user know and can do that properly, following his own need, process or organization.

I decided to try the latest version 20170527 on MacOS rather than 20170401 I have been using to try out the sidecar file approach. It saved an image.jpg.pfi file as described but exactly as the previous version when you open a image.raw, image.jpg, image.pfi or now the image.jpg.pfi file with photoflow it simply takes you to the ‘open existing file’ folder list last used. To be honest this is not a major issue in how I’ve been using photoflow as I tend to be in the same folder for most or all of one session and work directly with photoflow rather than opening via a DAM. It does however mean that the dialogue to select the sidecar file does not appear.

As an aside I noticed that although the newer version continues to work with the .RW2 files from my Panasonic TZ100 the .ORF files originating from my Olympus E-1 and E-620 no longer work, Photoflow just crashes. Has the database of camera profiles used by photoflow changed?

Anyway thanks for the great work on photoflow which I’m really getting to like a lot!

At the moment I am fighting with some crashes in the windows version that occur for any opened image… once that is fixed, I will finalize the sidecars support.

Indeed I have updated the RAW files definitions to the latest darktable version, but that should not introduce any regression… I will check.

Thanks!

No problem, there is no rush for this. Thanks for your help.

This seems to be a problem of integration with the OSX system… does it happen only when you try to open a file from Finder, or also from Digikam?

Anyhow, I will look into that after the next release is out (hopefully by next week), because there are few other OSX integration issues I need to fix (for example, the Cmd-q shortcut does not work for closing the application).

Meanwhile I have prepared a new version that fixes an issue when directly opening .PFI files (and not indirectly via sidecar files). If interested, you can get it here.

I don’t use Digikam much but gave it a try and the problem is the same.

I tried the version you linked to. Not sure what problem it has fixed as I did not have a problem with opening .pfi files directly with version 20170401. It still has a problem opening the .ORF files but then you did not say you had managed to do anything on this.

Many thanks for your continued support.

1 Like

Next item in the todo list for next release :wink:

1 Like

I’ve just tried loading the E-1 and E-620 samples available at raw.pixls.us, and they open without issues… could you maybe provide me with an example of .ORF file you cannot open?

Also, could you maybe send me the terminal messages you get when you run photoflow from the console? On my system I can start it this way:

/Applications/photoflow.app/Contents/MacOS/photoflow path_to_Olympus_file.ORF

Thanks!

1 Like

I think I have found what is causing the problem. I checked the files on raw.pixls.us and they opened fine on my mac with version 20170608. On examination of them and the files I was having problems with the only difference I could see is that mine had IPTC data. Most of my older E-1 and E-620 files are in an Aperture file system and on export for reworking with photoflow I had the include IPTC data ticked. Sure enough when I exported without the IPTC data the files open fine.

The files with IPTC data do open with version 20170401 so something has changed.

The files (with and without IPTC) and terminal messages are included at the following dropbox link in case you need to looking why the files with IPTC don’t work.

Again many thanks.

Updating to the latest version of EXIV2 seems to be the right solution. Here is a new version that correctly opens the .ORF files with IPTC data on my OSX system.

1 Like

That works for me also. Once again thanks for your help.

Unfortunately there seems to be a minor problem with this version (20170610) which I have not seen before. When I export a jpg file the correct colour profile is not attached. So far on testing Panasonic files always have Adobe RGB and Olympus files seem to have no profile attached. The actual colourspace conversion seems to be working fine as if I attached the correct colour profile to the exported file it displays ok in other programs. Sorry for yet another problem!

I will check! No need to apologize, I really appreciate the feedback…

1 Like

I fixed the issue with output color profiles, they are now correctly attached both for TIFFs and Jpegs.

That was a major problem, so thanks again for reporting it!

Updated OSX package: https://github.com/aferrero2707/PhotoFlow/releases/download/continuous/photoflow-20170612.app.dmg

1 Like

Thanks that works fine.

1 Like

Coming back from a wedding, I’ve used DigiKam and PF, to develop my images ending up with 150 of them.
This is a journey … :slight_smile:
First the performance and stability are the main issues, but that’s not the topic. I didn’t include “local contrast” and “noise reduction” to speed up a little bit.
I did not succeed in getting good result with some high iso, but that’s not the topic neither.

Beside that, the repetitive questions are a bit boring

  1. A pfi has been detected. Do you want to open it ?
  2. After having exported a jpg file, exiting PF:
    2.1. Modified buffers existing, do you really want to exit?
    2.2. Image “xxx.jpg.pfi” contains unsaved data. Do you want to save it before closing?
    The last 2 questions should occur only if I haven’t not saved anything, am I correct ?
    I do not get the right tuning the first time, so I did that 3, 4 or 5 times for each single image …
    If some questions must remain, I think a way to avoid them will be more than welcome.

Here is a confusing case. When we have a pfi plus jpg.pfi like:

When I open the jpg file PF asks the question about name.pfi. It should ask about name.jpg.pfi (at least use it). I want to open the treatment which created the jpg :

If I answer yes, I don’t get the actual treatment which has built the jpg. The worst part of it is that I may not notice I use the wrong pfi file…
If I answer no, PF opens the jpg image without any treatment. The only way to reedit properly name.jpg is to delete name.pfi.

A general topic. From DAM, when opening an image with PF, a new instance of PF is started. Would it be possible, or even desirable, to just open the new image in the current PF instance ? If that would not save any click, that would be more convenient to switch from an opened image to another.

When I’ve finished to tweak all images, I would have loved to be able to batch generate again all the images adding a bit of “local contrast” and, for high iso, some “noise reduction” before publishing the images.
Still on batch side, I think it would help to batch generate at the beginning a jpg file for each raw. This time, that would speed up the DAM and help to sort out the images.

Among the things which could speed up also the development process is a color picker for curves, white balancing, …

What did help a lot is the default.pfp, and even more, the ability to copy layers from one image to the other. The copy itself is quite fast ! :slight_smile: , then just be patient, if the previous image is good the next image is (nearly) good too :slight_smile: .

I’ve probably made a lot of mistakes, not used the best method. I would love to learn how to go faster (4 times would be a good start) because all the tools are there to get the best from our images.

Thank you Carmelo !!

1 Like

Actually I think the original logic works well. If I want to open the previous image development steps I open the .pfi, if I want to rework the original raw file from scratch I open the .raw and if I want to include or work on the resultant image I would open the .jpg or .tiff. May not be the way other programs work but it seems good to me.

Speed is still one of the bottlenecks, and one of the big things on which I need to work once the next release is out. I have some ideas, but they will require some re-factoring of the processing pipeline… The goal is to get very close to DT and RT for all the tools that are somehow in common.

Today I will have some time for working on the file handling logic, so expect some update by tomorrow :wink:

True, but having sidecar files for the exported images has the big additional advantage of keeping track of how a given image was exactly generated. How the sidecar files are exactly handled is still perfectible though. Anyhow, the whole sidecar files business can be completely disabled via preferences (and is actually disabled by default), in which case the old logic that you describe applies.

Thanks for all the suggestions!

I’m making progress on the batch side :slight_smile:

This pfi creation doesn’t work on Linux neither :

  1. produce pfi in PFs for all nef
    for i in *.NEF; do /usr/bin/photoflow --batch $i default.pfp PFs/name.pfi; done

However those work pretty well:
2. produce jpg in PFw for all NEF
for i in *.NEF; do /usr/bin/photoflow --batch $i default.pfp PFw/name.jpg; done
3. produce jpg for all jpg.pfi in PFf
for i in *.jpg.pfi; do /usr/bin/photoflow --batch $i name; done
4. produce jpg in PFw for all jpg.pgi in PFf with additional presets
for i in *.jpg.pfi; do /usr/bin/photoflow --batch $i …/PFw/preset.pfp …/PFw/name; done

With 2. I’ve no pfi to edit the jpg from Digikam. So either the batch follows the configuration (save sidecar file) or I should use 1. (with name.jpg.pfi), if works, then .3 to produce the jpg.
We see also here that the double extension (.jpg.pfi) complexifies a bit the logic.

1 Like