@Elle I am a fan of your website http://ninedegreesbelow.com/, though I have trouble grasping the more advanced concepts 
. In terms of dcraw-float, I really miss the -w option. I was wondering how I could modify the code to turn this back on (and whether doing so is advisable). This would save the step of having to query the multipliers using another program.
Hi @afre ,
Without going back and looking at the dcraw-float code, I think adding the -w option probably would be relatively easy. I say ârelativelyâ because Iâm not very good at writing c code, by which I mean I can write reasonably good c code but doing so takes me a very long time. Hopefully you are better at writing c code than I am.
I used to use dcraw-float all the time for pretty much all of my raw processing. Most of the time I prefer to process raw files at the command line because itâs easier and faster than opening a full-featured UI raw processor like darktable, RawTherapee, or PhotoFlow. UFRaw is fast when opening a raw file, and provides instant access to the RGB white balance multipliers, but last I checked it still doesnât have true raw exposure compensation and doesnât provide the AMAZE algorithm.
These days when I want to process raw files at the command line - which is almost always because currently everything I photograph is shot from a tripod making ev-bracketed exposures - I use darktable or the linear gamma branch of PhotoFlow, making use of their âcommand lineâ options. Both of these raw processors can output 32-bit floating point files working strictly from the command line. RawTherapee also can be used from the command line, though currently output is limited to 16i (same as dcraw-float output).
Before diving in and trying to update and modify dcraw-float, have you considered using these other raw processors in their âcommand lineâ mode? If you are interested, I can provide sample commands for darktable and PhotoFlow, and Iâm sure RT users can help with using RT at the command line. All of these raw processors already provide for auto white balance.
Yes please, it would be great to learn from your current preferred workflow from raw processor to first steps in image editor.
Hmm, my workflow is a bit different from the norm, and might not be at all suitable to what you want to accomplish. Most of the time I use a raw processor strictly for outputting a scene-referred image, and then do all my post-processing using a high bit depth image editor that supports masks and layers.
For a long time I used regular dcraw for raw processing, and then my floating point dcraw, and then switched to RawTherapee because of its excellent noise removal algorithms, and to darktable because darktable allows for 32-bit floating point output.
Testing that I did several years ago convinced me that RT and dt both output the same high quality image as my floating point dcraw, and at the time I was processing images that needed some TLC in terms of white balancing and noise removal. For noise removal during raw processing, itâs better to work using the UI version of the raw processor so you can see the effect of the various noise-removal algorithms.
These days Iâm starting to also use the PhotoFlow raw processor, specifically the linear gamma branch, which is where the really interesting PhotoFlow development (from my point of view) is happening, including the ability to use a script to process ev-bracketed raw files.
In other words, free/libre image editing has an absolute wealth of raw processors to choose from. None of them are bad and the ones Iâve mentioned are all excellent. All of them (including a few that I havenât mentioned but have worked with in the past) have outstanding features that differentiate them from one another.
So before diving in and giving details about a workflow that might not be of any interest whatsoever to you, I thought Iâd ask what are your specific photographic and editing goals that have prompted an interest using the command line for raw processing?
Also, you asked about the advisability of using automatic white balance. A lot of people pretty much always use AWB and manage to make really nice final images. Personally I almost never use AWB for reasons nicely summarized and illustrated in this article: http://www.natureandphotography.com/the-urban-myth-of-auto-white-balance-and-raw/
Methods that I use for white balancing include:
-
Use a preset. For example, if Iâm shooting outdoors and want to capture the ambiance of whatever the weather is, I use the daylight preset.
-
Make a white balancing shot of a neutral object. Being inclined to not spend money when something DIY will do, usually this neutral object is a bit of white PVC plastic that I carry in my pocket or camera bag.
-
Put a diffuser (aka the end of a white styrofoam coffee cup that just happens to fit exactly over my lens) over the lens and shoot into the light source (but never into the sun! anyone reading this - please donât ever aim your camera at the sun! if the sun is the light source and you donât have a neutral object to photograph, just use your cameraâs daylight preset!).
-
Use a hopefully neutral object in the scene to set the white balance.
-
If all else fails, AWB is a last resort. But at this point very likely the white balance will still need to be tweaked by hand.
Background
Most of my shots are of youth and volunteers of all ages. In this context, I find it difficult to frame shots and get things like white balance under control, since the subjects, locations, angles and mixed lighting are variable at any given time. With a prime lens (1.6 x 50 mm = 80 mm; all I have is inherited gear), it is challenging to capture the scene well, especially in small and/or overcrowded spaces, often with trigger happy smart phone users who step in close. In many cases, I end up missing the moment or otherwise compromising the composition.
Editing Goals
I realize that I probably cannot or should not aim for an 1:1 interpretation to due artistic and technical reasons, but I do hope that I can improve upon the captured image to bring it closer to the ground truth than what the in-camera JEPG or default post-processor settings would yield. At the moment, I donât have the skills to take advantage of RAW but at least I want to learn how to interpolate the image and set up the environment so that my options as open as possible.
At the same time, I tend to be overwhelmed by the in-depth post-processing discussed on this forum and elsewhere maybe because of how my mind and eyes work. I tend to be abstract and indecisive in nature, and have trouble with colors. Itâs weird with colors: I have trouble identifying and making sense of them even though I can tell them apart. I am guessing that I am more perceptive to luminance than color. Or perhaps I cannot handle visual complexity as well as other people or single out elements as easilyâŚ
White Balancing
I tend to use AWB out of avoidance of WB management 
. Having to decide when to set WB and remembering that it is on custom, is kind of annoying, especially when you are in a chaotic environment. I guess it comes with practice. I have tried the foam cup method and found it useful, but again I might forget that I made the setting. I havenât tried PVC yet, but the advantage of that method would seem to be that I could plant and snap the neutral object after I capture the scene and not have to remember to change the WB setting as much.
The article made a good point about keeping a consistent WB for a set of photos. It would definitely be useful for any command line or batch related processing, not just to make discrepancies easier to detect.
Hi @afre
Thanks! for the overview of your shooting goals and equipment, this will help keep advice from being totally inapplicable to your actual goals. But before changing topics to goals/etc Iâd like to continue the discussion of setting custom white balances:
Well, you are right, it is very annoying and very easy to forget that you left the camera set to a no-longer-valid custom white balance setting. This is something that I intensely dislike about shooting âin cameraâ jpegs as when shooting jpegs you really do need to remember to keep changing the camera-set white balance as the lighting changes.
If you are shooting raw, itâs important to make new white balancing shots as needed for the various lighting conditions. But you donât actually ever need to change the in-camera white balance. At least so far most (and hopefully all) cameras do not use the in-camera-set white balance when saving the raw file, except as a setting in the metadata to let the photographer know what the white balance was set to at the time the raw file was captured.
The way I handle the in-camera white balance is very simple: I only shoot raw and I leave the in-camera white balance set permanently to UniWB, or as near as my camera can get.
Then during the shoot I make whatever white balancing shots (of a neutral object like white PVC or using white styrofoam as a diffuser) seem appropriate, as one of the first shots that I take, and also as the lighting changes:
-
If Iâm shooting outside under natural light, usually I donât bother making any white balancing shots as the âdaylightâ preset serves to capture ambiance.
-
If I donât want ambiance (for example I want to eliminate the âblueâ light of a rainy day or the âgreenâ light created by being under deep green foliage, or the very blue light in deep shade on a sunny day), then even outside under natural light I make a white balancing shot.
-
If Iâm shooting indoors under artificial light, I try to remember to make a new white balancing shot whenever the indoor lighting changes. Iâd probably try to do the same thing if I were shooting an outdoor event with artificial lighting, but this is a type of shooting Iâve never done.
But I donât use these white balancing shots to create custom âin cameraâ white balances for each series of shots. Instead I leave the in-camera white balance set to âUniWBâ. Reasonable alternatives to âUniWBâ for the âset it and forget itâ in-camera white balance when shooting raw would be the cameraâs daylight preset or else AWB.
Then I use the âper sessionâ white balancing shots (or presets as applicable) to set the actual white balance as part of the raw interpolation process for each shot.
The only hard part about not setting the white balance âin cameraâ as the shooting conditions change is that you have to keep track of when to use which white balancing shot for which frames. But personally I find this a lot easier to manage than trying to remember while shooting to keep setting a new âin camera custom white balanceâ as the lighting changes.
Yes, consistency of the white balance often is important when making a series of shots to be presented as a whole. And when interpolating a bunch of raw files at the command line, itâs critically important to be able to feed the correct white balancing information to the raw processor because the white balance needs to be set before the interpolated file is converted from the camera input profile to your chosen output profile.
As an added bit of information, if you have your own custom camera input profile (or if you use PhotoFlow to output an interpolated raw file still in the dcraw camera standard input profile color space, and then save the embedded profile to disk), then you can ask whatever raw processor you are using to output the image as âraw colorâ, and then you can do the white balancing as the first step in âpostâ. But this is an advanced topic that has its own set of complications, so perhaps should be deferred for another discussion.
@patdavid - maybe the title of this forum thread could be changed to something like âAdvice for event shooting and processingâ?
It would be nice to hear from some people experienced at event shooting (something Iâve never done). But personally Iâd suggest that if you are shooting raw, then set the in-camera white balance permanently to AWB or else the Daylight preset, which will allow you to shoot âonly rawâ or else âjpeg+rawâ if you choose. (UniWB is the other option, but shooting âjpeg+rawâ is pointless when using UniWB).
And then the only thing you need to worry about (besides getting past the trigger happy smart phone users to get your shot) is to also take some appropriate white balance shots for the various lighting conditions, to be used to set the white balance during raw processing.
As far as your equipment goes, having less available equipment (one camera, one 50mm prime lens) is an advantage precisely because it limits your options. You arenât fiddling with changing lenses. You arenât fiddling with setting the exactly right zoom on a zoom lens (something I intensely dislike about using zooms). So as far as framing your shots goes, the only things you have left to worry about are where you put yourself, where you aim the camera, and when you press the shutter. And thatâs quite enough to worry about!
I think everyone develops a preference for a given lens focal length. I have a 28mm prime and also a 50mm prime that I âinheritedâ from my husbandâs film camera. For anything other than street-shooting (and sometimes even then), every time I put the 28mm lens on my camera, about two shots later I put the 50mm lens back on the camera.
For event shooting and street shooting, many people will prefer a 28mm lens. But the âshootingâ price you pay for a wider angle lens is that the 28mm lens âseesâ from a different framing angle compared to what you see with your own eyes (wide angle distortion). I like the fact that when I look through the 50mm lens, what I see is pretty much what I see when I lower the camera. But this is a personal preference.
Eventually you might want to borrow or rent a 28mm lens and see if you like it better than the 50mm lens. But before trying new equipment, itâs better to master your current equipment. A 50mm lens on a 1.6 format camera is already a pretty nice combination - the smaller sensor size means a greater depth of field for any given f-stop, and for fast-paced event shooting I suspect this is an advantage.
For my own style of shooting, I put my camera on full manual mode, and my lenses are âmanual focus onlyâ. For street shooting, I do use my 28mm lens (the only time I use this lens) and the classic âf8 and be thereâ to avoid having to think about the fstop. Then only thing left to think about is the shutter speed and setting the lens point of focus, which I usually leave set to the hyperfocal sweet spot where as much of the depth of field as possible is in focus. But this style of shooting does presume conditions with plenty of light.
http://www.adorama.com/alc/0013109/article/f8-And-Be-There
I suspect many event shooters today rely on various automatic options such as auto-focus and various automatic shooting modes. Personally I never did figure out a good use for automatic shooting modes (but I know there are styles of shooting that required using these modes). And with my first digital camera (a point and shoot with a zoom lens), I came to absolutely loathe the whine of the autofocus lens seeking its point of focus, which was never where I, the photographer behind the camera, actually wanted it to be.
1 Like
Should we split the event-shooting specific stuff to a new topic? (Also, not sure but you might be able to change topic titles as a mod).
Hmm, you are right, I was able to change the forum thread title.
Most of the thread pertains to event shooting: setting up the camera, including whether to use AWB or to make white balance shots as lighting conditions change, equipment to use, etc.
The rest of the thread pertains to processing raw files at the command line. Would it be possible to split the âprocessing raw files at the command lineâ portions off to a separate thread, and maybe even move the âevent shootingâ portion over to the Capture category?
I would consider carefully if a topic title needs to be changed (hint: rarely).
Instead, we can split topics that have veered OT into new threads of their own (or move certain posts to other existing threads). You should be able to do this also.
Hmm, well, guidance is very welcome on what to do! The original question was about my floating point dcraw, but my floating point dcraw was immediately left behind as not relevant to solving the practical problems at hand. Then the thread pretty much clearly split into two parts:
-
processing raw files from the command line - not really yet covered in the thread, so maybe start a new thread and move the relevants portions of the existing thread? You say I should be able to do this, but I donât know how (and Iâm a bit afraid of doing something altogether wrong).
-
advice for shooting events, including advice for when the goal is to shoot raw files (the major portion of the current thread)
FWIW, I donât anticipate wanting to change topic titles very often, maybe not even ever again. My reason for changing the topic title for this particular thread is because the original title didnât seem likely to attract the attention of people with practical advice to offer on event shooting. If I erred, my apologies to you and also to @afre .
Sorry, I donât have much online experience so I donât know proper forum etiquette. IRL I tend to have the habit of addressing multiple threads of thought at once, which makes it very confusing for the listener. Also, I am not certain of what the tipping point is for creating a new thread or changing the title. I wouldnât want to change things all of the time or have dozens of threads. That would be obnoxious.
I guess I often just need a gentle reminder like â@afre, please split the topicâ, and I will create a new one that links to the current one. Anyway, thanks for being so considerate about it.
Donât worry about it @afre, only an admin can split a topic. We try to keep one topic per thread, but meandering is permitted. If someone feels the topic has meandered far enough, theyâll split the topic. No big deal, we just try to keep everyone on the same page.
1 Like
@afre - I have the same habits as you, addressing multiple threads at once. So if itâs OK with you and @patdavid , letâs keep everything in this one thread, renamed as it is (and again my apologies if I shouldnât have renamed it).[quote=âafre, post:5, topic:3436â]
I realize that I probably cannot or should not aim for an 1:1 interpretation to due artistic and technical reasons, but I do hope that I can improve upon the captured image to bring it closer to the ground truth than what the in-camera JEPG or default post-processor settings would yield. At the moment, I donât have the skills to take advantage of RAW but at least I want to learn how to interpolate the image and set up the environment so that my options as open as possible.
[/quote]
Could you explain a bit what it is about the in-camera-created jpeg and the usual default post-processor settings that you arenât happy with? I also donât like camera jpegs and the results of the usal post-processor settings in the various raw processors. So my temptation is to explain how to get scene-referred output that you can then modify to suit. But it would be better to first have a more clear idea of what you donât like about âthe usual wayâ and what you want to accomplish instead.
1 Like
You didnât err, of course. I was just suggesting a light touch, but I trust everyone on the staff and their decisions. 
(We havenât found anyone completely nuts yet - except maybe @houz). If you need a hand with splitting topics or anything just let me know and Iâll do whatever I can to help!
@afre youâre doing great, donât worry! 
I still have some preprocessing items to explore.
More Background
I shoot with the optical viewfinder in RAW and manual mode. I use autofocus and spot metering, though coordinating the two can be awkward (manual focus might help). I find that there is almost always more dynamic range than my camera can handle, leading to the improper exposure of certain parts of the resultant image and in turn undermining the detail and color (flash might help but mine is broken).
More Questions
-
Does WB affect the metering in the viewfinder?
-
The foam cup method doesnât require placement but WB targets do. In an uncontrolled environment, where there is movement and mixed lighting, how do I make good use of them?
-
Teflon is mentioned in one of your articles. Is there a reason that you use PVC instead?
Shooting Approach
A 35mm would be nice. Indoors, I often find myself contorted against the wall. Same with group photos, I have to stand uncomfortably far, where details begin to lose resolution. I also have trouble âbeing thereâ as I am stuck between finding a shutter speed that compensates for my unsteady hands and not getting enough light for a proper exposure. I would increase the ISO but my camera is on the noisy side.
WB doesnât affect metering in the viewfinder of an SLR. It probably does in a mirrorless, depending on what colors make up the scene.
Using a foam cup to set white balance isnât strongly affected by where you point the camera, but it is sensitive to where the camera is. Ideally you set the WB from near where the subject is. In an uncontrolled situation, just change stuff in post until it looks good.
Why not use shutter-priority mode when youâre in a situation that requires a minimum shutter speed, such as a group even or party? That way you can at least take a photo without a lot of motion blur. Iâd also say that raising your ISO is much preferable to missing the exposure. You just need to explore some noise reduction techniques.
Got a monopod? I find it makes a difference in what shutter speed I can get away with versus pure hand-held.
I think in your shooting situation the styrofoam cup or similar diffuser over the lens is the better option:
Also videographers sometimes use the diffuser approach: https://www.youtube.com/watch?v=Ak3wRrv1ZbI
When white balancing to take a close-up of a single person, if the person will cooperate, put them in the most full-spectrum lighting available and then have them hold a WB target close to their face. Likewise if you are moving in close to photograph some stationary object, you can make two shots, one with and one without the PVC or other white balance target in the predominant light source illuminating the object to be photographed. But otherwise, I donât see any practical way to use a target in an âeventâ type mixed lighting situation.
Regarding the teflon, thatâs in my article about making a target shot for making a custom camera input profile. I included the option to use teflon to white balance the target shot based on posts made to the ArgyllCMS mailing list, but since then the list member who suggested using teflon decided it wasnât such a good idea. My own effort to use the teflon tape also didnât really work. I need to revise that article and remove the reference to the teflon tape. Itâs not the easiest stuff to even get flat when attached to a back support.