Turns out I wasn’t doing the display transform right, so that’s now fixed. I also multithreaded most of the colorspace transforms, LittleCMS made that fairly easy. Also, a few fixes.
With this version, I now have tools that support a workflow I’ve messed with for a couple of years. In our recent vacation, I shot raw, then used the img command line program every evening to produce 800x600 jpegs for posting. With those, I can later open-source the raw file and make a better jpeg. So, the only full-sized images I save are the raws, unless I need to take something to GIMP, which is then a 16-bit TIFF with the working profile. This workflow works well, and 0.6.2 has a few changes to make it work excellently.
Your workflow step of using a command line program to make 800x600 jpegs from the raw files sounds like a good way to review files. I wonder if the resulting jpegs could be embedded in the raw file in place of whatever jpeg the camera might or might not have embedded - this would make it easier to review the raw files in digiKam or other image viewers. The camera-embedded jpegs never look anything like what I’d want to use for reviewing raw files because of the heavy S-curve that compresses shadows and highlights, plus the white balance I use when shooting isn’t too likely to be the white balance I’d use when processing the raw file.
Have you considered writing a plug-in for GIMP, which would allow GIMP-2.9 to open raw files using rawproc? RawTherapee, darktable, PhotoFlow, and also @partha 's version of nufraw all have example code to follow.
Yes, I removed the library you had problems with in Gentoo; it’s in the Ubuntu distros already. If any others have library issues with the AppImage in other distros, let me know.
Turns out for most things for which my family uses the images, 800x600 is all they need. That includes printed products such as calendars. They’re almost impossible to talk to about color management…
About the only way to do that I can contemplate short of writing a binary file-level library would be to use exiftool. For my favorite raw format, NEF, it has read/write capability. It also has a C/C++ API, so I can envision a “Save To…” menu option. Hmm, I’ll consider it for the next major rev.
Actually, yes, just last week I had to save a TIFF to take to GIMP, and the need for that very thing just sat on my desk and stuck it’s tongue out at me. Since rawproc uses 32-bit floats for its internal representation, it would be particularly nice to import that to >= GIMP 2.9 directly. I just added it to the rawproc Issues list in github.
The way I’m using working profiles now involves adding a colorspace tool as the first tool following opening the NEF as raw/linear from libraw, convert operation, relative_colorimetric inten. I’m using your Rec.2020 V4 g18 profile, and I’m finding the resulting images are quite nice without further modification. I save them to JPEG converted to a V2 SRGB g22 profile, and they seem to display consistently across all the computers to which I have access.
Now, none of this is available in the img command line program, but I’m starting to think about those modifications. However, one of the last mods I put into 0.6.2 was for rawproc to recognize img-generated JPEGs and TIFFs for the ‘Open Source’ menu, where the metadata is searched for a rawproc/img tool string and, if found, will open the original image and apply the tools to make the result image. I find this VERY useful for reviewing the img-generated 800x600 pictures; I can right-click->Edit in geeqie, select rawproc, and it’ll ask if I want to open the original NEF and re-apply the tool chain instead of just opening the JPEG.
Anyway, let me know if you find any other color management foo… Thanks!
I gave img then rawproc a tentative try last night and I must say that it loads the raw from [PlayRaw] Hawkcraig Pier very slowly (20-22s). In comparison, dcraw exports to TIF under 10s.
Hmm, I downloaded the image and opened it with the AppImage, took about 10sec on a AMD Phenom II with four cores. Here’s the CPU utilization; the left hump is the libraw open, and the right one is the conversion to the working profile. Interesting behavior on the open; it looks like the work is being swapped among processors. I struggled a bit to get libraw to compile wilth OpenMP on Windows targets, but I didn’t pay much attention to Linux, looks like I need to…
When img is working, the first step is 20-22s. The rest is negligible. In contrast, everything is done in dcraw in less than 10s. Maybe 5-8s; CMD doesn’t have a timer. I tried it on win7 with an Intel Core i3-2310M CPU @ 2.10GHz (two core) laptop. Note that when I first got it, it already felt slow even though it was an upgrade from my even lower powered systems at home. I know that not all computer parts are made equal…
First impressions:
Noticed that img is case sensitive. Maybe I am just used to insensitive CLI apps.
Noticed that img doesn’t overwrite files, at least when I use * in both input and output. Maybe offer to overwrite.
NL denoise never runs well on my computer. rawproc GUI just turns white and stalls. Maybe offer other cheaper options.
Case sensitivity is a unix thing. Or case insensitivity is a Windows thing. rawproc/img are cross-platform programs, so the unix convention overrides…
The ‘no overwrite’ is a feature. I sometimes batch process in increments, and this configuration allows me to dump the new NEFs in the same folder for the day, and when I run img, it skips the already-processed images. I am considering an overwrite switch.
Yes, my NLdenoise algorithm is the plain version, not optimized. Lately, if I need to denoise, I use the libraw/dcraw wavelet denoise, in Properties set input.raw.libraw.wavelet_denoise to 500 and open the raw file. Much faster, and good quality denoise.
Exiftool is what I was thinking about using, but I’ve never tried using exiftool this way - I’ve extracted thumbnails, but not embedded them. It would be nice to process new raw files at the command line to output resized jpegs and then embed the jpegs in the raw files.
What is this img command line program?
Hmm, my apologies, when it comes to figuring out how to use new software, I seem to be about as incapable as a person can get (GIMP, PhotoShop, and RawTherapee for some reason being exceptions to the rule - Blender? forget about it!).
I started the latest rawproc appimage. Looking at the rawproc help there are instructions for setting up color management, but I’m clueless to interpret. I’d copy-paste to show which part of the help I’m talking about, but for some reason copy-paste doesn’t seem possible.
So first question: How does one get to the “input.cms” thing?
Denoise just stalls the app right away without fair warning. I am not asking for anything to be done per se, as I know that NL denoise of any kind (even optimized) gives my laptop a heart attack. I personally don’t use wavelet denoise either; I tend to export from the raw processor and subsequently use G’MIC or GIMP.
My impression is that img is behind rawproc. Feature parity would be nice if it is.
When I wrote the gImage library, I wrote a command line program to test things, called gimg. It proved to be useful, so I copied it over to the rawproc source tree and renamed it img.
It’s part of my workflow now. On our recent vacation, I’d shoot raw all day, then batch-process them in the evening with img to produce 800x600 images for posting for the poor misfortunates at home to regard. These would upload in a minute or two over bandwidth-challenged hotel wifi.
img does a subset of rawproc’s operations, and does not yet carry icc profiles through the processing chain. But, if it only did one thing, resize has more than proved it’s worth. This command would read a set of full-sized JPEGs, resize to 640xAspect, and save to a subdirectory with an appropriate name:
img "*.jpg" resize:640,0 "album/*-800x600.jpg"
Ah, things that are obvious to me are not, really. The help file, reachable by “Help->View Help”, describes the Properties dialog and Configuration parameters, but I’ll cut to the chase here.
The Properties dialog, reached by “Edit->Properties…” is patterned after the about:config page in Firefox, a property for every whim. All the properties are described in the Help file in the Configuration page. You change them by clicking in the value column and editing with the keyboard. You can add or delete properties with the Add and Delete buttons. You can filter the properties by typing a search string into the Filter box, e.g., type ‘cms’ and all the color management properties are selected. Some property changes take place immediately, some with the next display update, some when a new image is opened, and some when the application is restart.
For color management, you need to do the following:
Set up a profiles directory. This is not required, but if you don’t, you’ll need to use the full path to specify profiles.
Set cms.profilepath to this directory. Use forward slashes for both Linux and Windows.
Set input.cms to 1. This enables color managment. Changing this parameter takes effect with the next image open.
Set display.cms.displayprofile to your display’s calibrated icc filename. rawproc will complain if you don’t set one, but you can turn that off with display.cms.requireprofile=0
If you’re going to ouput to JPEG, set output.jpeg.cms.profile to an appropriate profile. I use @elle’s sRGB-elle-V2-g22.icc.
Now, if you want to try things quick-n-dirty, you can open a raw file with one of the dcraw output profiles by setting input.raw.libraw.colorspace and input.raw.libraw.gamma. For instance, to use the dcraw prophoto output profile, set both of these to ‘prophoto’; for gamma, ‘prophoto’ is the alias for 1.8
If #6 leaves you wanting, you can use a calibrated camera profile and working profile by setting input.raw.libraw.cameraprofile to the camera profile icc filename, then after the image is opened, convert to an appropriate working profile by first adding a colorspace tool and specifying the appropriate file. If you do the same for every raw image, you can automate that with input.raw.default=colorspace:file.icc,convert,relative_colorimetric
Sooo… Steps 1-5 and 7 give you a fully color-managed workflow from open to save. I realize the Properties approach is a little tedious, but it makes implementing new properties quick and easy.
Hmm, clicking on Edit/Properties" brings up “No configuration file found.” Maybe there could be a default configuration file? I know I had a rawproc config file at some point but it seems to have disappeared. Can it be blank? Where should the config file go, using what file name?
@afre, I compared the run times of the 0.6.2 -w64 executable with a native-compiled version on a 2-core i5 laptop, both open the same D7000 NEF (16MP) in about 9 seconds. About the only thing I can think of would be two-fold: 1) are you using the same raw file in the same location? and 2) are you opening the raw file in rawproc from something like a SD card? I’ve found that to be the bottleneck on my Surface 3; I use a microSD for my Windows Libraries (Documents, Pictures, etc), the first one I had was a pig, and it showed its colors at file open/save.
I have been using the raw from [PlayRaw] Hawkcraig Pier, downloaded it into a new folder, the one that I processed with img, dcraw and PhotoFlow, respectively. (PhotoFlow is slow too but the version that I am using is busy pre-calculating CA at loading.)
Do you mean:
file from the same path (yes), or
accessing the file at the same time? (unsure)
No, it is on my main drive, an internal SSD (upgrade).
Is it installable to Ubuntu Xenial? I run the long term support version here, and the compiler tells me WxWidgets is not new enough - I have 3.1.0 and 3.2 is demanded. I’m a little wary of going down a trail of updating libraries to testing versions (although I am running Gimp 2.9)
Hmm, in the configure script the wxWidgets version tested for is 3.1. Now, I compile against a wxWidgets build I compiled and link statically, so there might be something else going on in compiling against the Ubuntu package. If you could paste a copy of the configure error here it might give me some clues where to look.
No problem here. I started with 3.0.2, but moved to 3.1 for some of the new standard directory functions. I’m trying hard to keep it transparently consistent between Linux variants and Windows, and that was one of the challenging things, where do files get put.
There’s an AppImage for 0.6.2 on the release page. I built it with Ubuntu, so it should work for you, just download and run (oh, you may have to change its permissions to allow execution).
I’m also about ready to start the home stretch to version 0.7, which has a bunch of fixes as well as a few new things like lens correction. It’s also the first version that implements a fully floating point color-managed workflow for my taste, which is to batch-convert a set of raws to 800x600 JPEGs, then edit selected ones in rawproc with the applied batch tool chain as a starting point. I’m pretty sure you can do the equivalent in either RT or darktable, but I can do it on a Microsoft Surface 3 with rawproc, a rather low-powered device.
That said, it’s definitely a “manual transmission” vehicle. You need to do the shifting yourself, (double-clutch, at that) and grinding gears is all-to-easy to do.
