Advice for event shooting and processing the raw files


#1

@Elle I am a fan of your website http://ninedegreesbelow.com/, though I have trouble grasping the more advanced concepts :sweat_smile:. In terms of dcraw-float, I really miss the -w option. I was wondering how I could modify the code to turn this back on (and whether doing so is advisable). This would save the step of having to query the multipliers using another program.


(Elle Stone) #2

Hi @afre ,

Without going back and looking at the dcraw-float code, I think adding the -w option probably would be relatively easy. I say “relatively” because I’m not very good at writing c code, by which I mean I can write reasonably good c code but doing so takes me a very long time. Hopefully you are better at writing c code than I am.

I used to use dcraw-float all the time for pretty much all of my raw processing. Most of the time I prefer to process raw files at the command line because it’s easier and faster than opening a full-featured UI raw processor like darktable, RawTherapee, or PhotoFlow. UFRaw is fast when opening a raw file, and provides instant access to the RGB white balance multipliers, but last I checked it still doesn’t have true raw exposure compensation and doesn’t provide the AMAZE algorithm.

These days when I want to process raw files at the command line - which is almost always because currently everything I photograph is shot from a tripod making ev-bracketed exposures - I use darktable or the linear gamma branch of PhotoFlow, making use of their “command line” options. Both of these raw processors can output 32-bit floating point files working strictly from the command line. RawTherapee also can be used from the command line, though currently output is limited to 16i (same as dcraw-float output).

Before diving in and trying to update and modify dcraw-float, have you considered using these other raw processors in their “command line” mode? If you are interested, I can provide sample commands for darktable and PhotoFlow, and I’m sure RT users can help with using RT at the command line. All of these raw processors already provide for auto white balance.


#3

Yes please, it would be great to learn from your current preferred workflow from raw processor to first steps in image editor.


(Elle Stone) #4

Hmm, my workflow is a bit different from the norm, and might not be at all suitable to what you want to accomplish. Most of the time I use a raw processor strictly for outputting a scene-referred image, and then do all my post-processing using a high bit depth image editor that supports masks and layers.

For a long time I used regular dcraw for raw processing, and then my floating point dcraw, and then switched to RawTherapee because of its excellent noise removal algorithms, and to darktable because darktable allows for 32-bit floating point output.

Testing that I did several years ago convinced me that RT and dt both output the same high quality image as my floating point dcraw, and at the time I was processing images that needed some TLC in terms of white balancing and noise removal. For noise removal during raw processing, it’s better to work using the UI version of the raw processor so you can see the effect of the various noise-removal algorithms.

These days I’m starting to also use the PhotoFlow raw processor, specifically the linear gamma branch, which is where the really interesting PhotoFlow development (from my point of view) is happening, including the ability to use a script to process ev-bracketed raw files.

In other words, free/libre image editing has an absolute wealth of raw processors to choose from. None of them are bad and the ones I’ve mentioned are all excellent. All of them (including a few that I haven’t mentioned but have worked with in the past) have outstanding features that differentiate them from one another.

So before diving in and giving details about a workflow that might not be of any interest whatsoever to you, I thought I’d ask what are your specific photographic and editing goals that have prompted an interest using the command line for raw processing?

Also, you asked about the advisability of using automatic white balance. A lot of people pretty much always use AWB and manage to make really nice final images. Personally I almost never use AWB for reasons nicely summarized and illustrated in this article: http://www.natureandphotography.com/the-urban-myth-of-auto-white-balance-and-raw/

Methods that I use for white balancing include:

  • Use a preset. For example, if I’m shooting outdoors and want to capture the ambiance of whatever the weather is, I use the daylight preset.

  • Make a white balancing shot of a neutral object. Being inclined to not spend money when something DIY will do, usually this neutral object is a bit of white PVC plastic that I carry in my pocket or camera bag.

  • Put a diffuser (aka the end of a white styrofoam coffee cup that just happens to fit exactly over my lens) over the lens and shoot into the light source (but never into the sun! anyone reading this - please don’t ever aim your camera at the sun! if the sun is the light source and you don’t have a neutral object to photograph, just use your camera’s daylight preset!).

  • Use a hopefully neutral object in the scene to set the white balance.

  • If all else fails, AWB is a last resort. But at this point very likely the white balance will still need to be tweaked by hand.


#5

Background

Most of my shots are of youth and volunteers of all ages. In this context, I find it difficult to frame shots and get things like white balance under control, since the subjects, locations, angles and mixed lighting are variable at any given time. With a prime lens (1.6 x 50 mm = 80 mm; all I have is inherited gear), it is challenging to capture the scene well, especially in small and/or overcrowded spaces, often with trigger happy smart phone users who step in close. In many cases, I end up missing the moment or otherwise compromising the composition.

Editing Goals

I realize that I probably cannot or should not aim for an 1:1 interpretation to due artistic and technical reasons, but I do hope that I can improve upon the captured image to bring it closer to the ground truth than what the in-camera JEPG or default post-processor settings would yield. At the moment, I don’t have the skills to take advantage of RAW but at least I want to learn how to interpolate the image and set up the environment so that my options as open as possible.

At the same time, I tend to be overwhelmed by the in-depth post-processing discussed on this forum and elsewhere maybe because of how my mind and eyes work. I tend to be abstract and indecisive in nature, and have trouble with colors. It’s weird with colors: I have trouble identifying and making sense of them even though I can tell them apart. I am guessing that I am more perceptive to luminance than color. Or perhaps I cannot handle visual complexity as well as other people or single out elements as easily…

White Balancing

I tend to use AWB out of avoidance of WB management :blush:. Having to decide when to set WB and remembering that it is on custom, is kind of annoying, especially when you are in a chaotic environment. I guess it comes with practice. I have tried the foam cup method and found it useful, but again I might forget that I made the setting. I haven’t tried PVC yet, but the advantage of that method would seem to be that I could plant and snap the neutral object after I capture the scene and not have to remember to change the WB setting as much.

The article made a good point about keeping a consistent WB for a set of photos. It would definitely be useful for any command line or batch related processing, not just to make discrepancies easier to detect.


(Elle Stone) #6

Hi @afre

Thanks! for the overview of your shooting goals and equipment, this will help keep advice from being totally inapplicable to your actual goals. But before changing topics to goals/etc I’d like to continue the discussion of setting custom white balances:

Well, you are right, it is very annoying and very easy to forget that you left the camera set to a no-longer-valid custom white balance setting. This is something that I intensely dislike about shooting “in camera” jpegs as when shooting jpegs you really do need to remember to keep changing the camera-set white balance as the lighting changes.

If you are shooting raw, it’s important to make new white balancing shots as needed for the various lighting conditions. But you don’t actually ever need to change the in-camera white balance. At least so far most (and hopefully all) cameras do not use the in-camera-set white balance when saving the raw file, except as a setting in the metadata to let the photographer know what the white balance was set to at the time the raw file was captured.

The way I handle the in-camera white balance is very simple: I only shoot raw and I leave the in-camera white balance set permanently to UniWB, or as near as my camera can get.

Then during the shoot I make whatever white balancing shots (of a neutral object like white PVC or using white styrofoam as a diffuser) seem appropriate, as one of the first shots that I take, and also as the lighting changes:

  • If I’m shooting outside under natural light, usually I don’t bother making any white balancing shots as the “daylight” preset serves to capture ambiance.

  • If I don’t want ambiance (for example I want to eliminate the “blue” light of a rainy day or the “green” light created by being under deep green foliage, or the very blue light in deep shade on a sunny day), then even outside under natural light I make a white balancing shot.

  • If I’m shooting indoors under artificial light, I try to remember to make a new white balancing shot whenever the indoor lighting changes. I’d probably try to do the same thing if I were shooting an outdoor event with artificial lighting, but this is a type of shooting I’ve never done.

But I don’t use these white balancing shots to create custom “in camera” white balances for each series of shots. Instead I leave the in-camera white balance set to “UniWB”. Reasonable alternatives to “UniWB” for the “set it and forget it” in-camera white balance when shooting raw would be the camera’s daylight preset or else AWB.

Then I use the “per session” white balancing shots (or presets as applicable) to set the actual white balance as part of the raw interpolation process for each shot.

The only hard part about not setting the white balance “in camera” as the shooting conditions change is that you have to keep track of when to use which white balancing shot for which frames. But personally I find this a lot easier to manage than trying to remember while shooting to keep setting a new “in camera custom white balance” as the lighting changes.

Yes, consistency of the white balance often is important when making a series of shots to be presented as a whole. And when interpolating a bunch of raw files at the command line, it’s critically important to be able to feed the correct white balancing information to the raw processor because the white balance needs to be set before the interpolated file is converted from the camera input profile to your chosen output profile.

As an added bit of information, if you have your own custom camera input profile (or if you use PhotoFlow to output an interpolated raw file still in the dcraw camera standard input profile color space, and then save the embedded profile to disk), then you can ask whatever raw processor you are using to output the image as “raw color”, and then you can do the white balancing as the first step in “post”. But this is an advanced topic that has its own set of complications, so perhaps should be deferred for another discussion.


(Elle Stone) #7

@patdavid - maybe the title of this forum thread could be changed to something like “Advice for event shooting and processing”?

It would be nice to hear from some people experienced at event shooting (something I’ve never done). But personally I’d suggest that if you are shooting raw, then set the in-camera white balance permanently to AWB or else the Daylight preset, which will allow you to shoot “only raw” or else “jpeg+raw” if you choose. (UniWB is the other option, but shooting “jpeg+raw” is pointless when using UniWB).

And then the only thing you need to worry about (besides getting past the trigger happy smart phone users to get your shot) is to also take some appropriate white balance shots for the various lighting conditions, to be used to set the white balance during raw processing.

As far as your equipment goes, having less available equipment (one camera, one 50mm prime lens) is an advantage precisely because it limits your options. You aren’t fiddling with changing lenses. You aren’t fiddling with setting the exactly right zoom on a zoom lens (something I intensely dislike about using zooms). So as far as framing your shots goes, the only things you have left to worry about are where you put yourself, where you aim the camera, and when you press the shutter. And that’s quite enough to worry about!

I think everyone develops a preference for a given lens focal length. I have a 28mm prime and also a 50mm prime that I “inherited” from my husband’s film camera. For anything other than street-shooting (and sometimes even then), every time I put the 28mm lens on my camera, about two shots later I put the 50mm lens back on the camera.

For event shooting and street shooting, many people will prefer a 28mm lens. But the “shooting” price you pay for a wider angle lens is that the 28mm lens “sees” from a different framing angle compared to what you see with your own eyes (wide angle distortion). I like the fact that when I look through the 50mm lens, what I see is pretty much what I see when I lower the camera. But this is a personal preference.

Eventually you might want to borrow or rent a 28mm lens and see if you like it better than the 50mm lens. But before trying new equipment, it’s better to master your current equipment. A 50mm lens on a 1.6 format camera is already a pretty nice combination - the smaller sensor size means a greater depth of field for any given f-stop, and for fast-paced event shooting I suspect this is an advantage.

For my own style of shooting, I put my camera on full manual mode, and my lenses are “manual focus only”. For street shooting, I do use my 28mm lens (the only time I use this lens) and the classic “f8 and be there” to avoid having to think about the fstop. Then only thing left to think about is the shutter speed and setting the lens point of focus, which I usually leave set to the hyperfocal sweet spot where as much of the depth of field as possible is in focus. But this style of shooting does presume conditions with plenty of light.

http://www.huffingtonpost.com/joe-newman/f8-and-be-there----as-tru_b_5486730.html

I suspect many event shooters today rely on various automatic options such as auto-focus and various automatic shooting modes. Personally I never did figure out a good use for automatic shooting modes (but I know there are styles of shooting that required using these modes). And with my first digital camera (a point and shoot with a zoom lens), I came to absolutely loathe the whine of the autofocus lens seeking its point of focus, which was never where I, the photographer behind the camera, actually wanted it to be.


(Pat David) #8

Should we split the event-shooting specific stuff to a new topic? (Also, not sure but you might be able to change topic titles as a mod).


(Elle Stone) #9

Hmm, you are right, I was able to change the forum thread title.

Most of the thread pertains to event shooting: setting up the camera, including whether to use AWB or to make white balance shots as lighting conditions change, equipment to use, etc.

The rest of the thread pertains to processing raw files at the command line. Would it be possible to split the “processing raw files at the command line” portions off to a separate thread, and maybe even move the “event shooting” portion over to the Capture category?


(Pat David) #10

I would consider carefully if a topic title needs to be changed (hint: rarely).

Instead, we can split topics that have veered OT into new threads of their own (or move certain posts to other existing threads). You should be able to do this also.


(Elle Stone) #11

Hmm, well, guidance is very welcome on what to do! The original question was about my floating point dcraw, but my floating point dcraw was immediately left behind as not relevant to solving the practical problems at hand. Then the thread pretty much clearly split into two parts:

  1. processing raw files from the command line - not really yet covered in the thread, so maybe start a new thread and move the relevants portions of the existing thread? You say I should be able to do this, but I don’t know how (and I’m a bit afraid of doing something altogether wrong).

  2. advice for shooting events, including advice for when the goal is to shoot raw files (the major portion of the current thread)

FWIW, I don’t anticipate wanting to change topic titles very often, maybe not even ever again. My reason for changing the topic title for this particular thread is because the original title didn’t seem likely to attract the attention of people with practical advice to offer on event shooting. If I erred, my apologies to you and also to @afre .


#12

@Elle @patdavid

Sorry, I don’t have much online experience so I don’t know proper forum etiquette. IRL I tend to have the habit of addressing multiple threads of thought at once, which makes it very confusing for the listener. Also, I am not certain of what the tipping point is for creating a new thread or changing the title. I wouldn’t want to change things all of the time or have dozens of threads. That would be obnoxious.

I guess I often just need a gentle reminder like “@afre, please split the topic”, and I will create a new one that links to the current one. Anyway, thanks for being so considerate about it.


(Mica) #13

Don’t worry about it @afre, only an admin can split a topic. We try to keep one topic per thread, but meandering is permitted. If someone feels the topic has meandered far enough, they’ll split the topic. No big deal, we just try to keep everyone on the same page.


(Elle Stone) #14

@afre - I have the same habits as you, addressing multiple threads at once. So if it’s OK with you and @patdavid , let’s keep everything in this one thread, renamed as it is (and again my apologies if I shouldn’t have renamed it).[quote=“afre, post:5, topic:3436”]
I realize that I probably cannot or should not aim for an 1:1 interpretation to due artistic and technical reasons, but I do hope that I can improve upon the captured image to bring it closer to the ground truth than what the in-camera JEPG or default post-processor settings would yield. At the moment, I don’t have the skills to take advantage of RAW but at least I want to learn how to interpolate the image and set up the environment so that my options as open as possible.
[/quote]

Could you explain a bit what it is about the in-camera-created jpeg and the usual default post-processor settings that you aren’t happy with? I also don’t like camera jpegs and the results of the usal post-processor settings in the various raw processors. So my temptation is to explain how to get scene-referred output that you can then modify to suit. But it would be better to first have a more clear idea of what you don’t like about “the usual way” and what you want to accomplish instead.


(Pat David) #15

You didn’t err, of course. I was just suggesting a light touch, but I trust everyone on the staff and their decisions. :smiley: (We haven’t found anyone completely nuts yet - except maybe @houz). If you need a hand with splitting topics or anything just let me know and I’ll do whatever I can to help!

@afre you’re doing great, don’t worry! :wink:


#16

I still have some preprocessing items to explore.

More Background

I shoot with the optical viewfinder in RAW and manual mode. I use autofocus and spot metering, though coordinating the two can be awkward (manual focus might help). I find that there is almost always more dynamic range than my camera can handle, leading to the improper exposure of certain parts of the resultant image and in turn undermining the detail and color (flash might help but mine is broken).

More Questions

  • Does WB affect the metering in the viewfinder?

  • The foam cup method doesn’t require placement but WB targets do. In an uncontrolled environment, where there is movement and mixed lighting, how do I make good use of them?

  • Teflon is mentioned in one of your articles. Is there a reason that you use PVC instead?

Shooting Approach

A 35mm would be nice. Indoors, I often find myself contorted against the wall. Same with group photos, I have to stand uncomfortably far, where details begin to lose resolution. I also have trouble “being there” as I am stuck between finding a shutter speed that compensates for my unsteady hands and not getting enough light for a proper exposure. I would increase the ISO but my camera is on the noisy side.


#17

WB doesn’t affect metering in the viewfinder of an SLR. It probably does in a mirrorless, depending on what colors make up the scene.

Using a foam cup to set white balance isn’t strongly affected by where you point the camera, but it is sensitive to where the camera is. Ideally you set the WB from near where the subject is. In an uncontrolled situation, just change stuff in post until it looks good.


(Mica) #18

Why not use shutter-priority mode when you’re in a situation that requires a minimum shutter speed, such as a group even or party? That way you can at least take a photo without a lot of motion blur. I’d also say that raising your ISO is much preferable to missing the exposure. You just need to explore some noise reduction techniques.


(Gord) #19

Got a monopod? I find it makes a difference in what shutter speed I can get away with versus pure hand-held.


(Elle Stone) #20

I think in your shooting situation the styrofoam cup or similar diffuser over the lens is the better option:


Also videographers sometimes use the diffuser approach: https://www.youtube.com/watch?v=Ak3wRrv1ZbI

When white balancing to take a close-up of a single person, if the person will cooperate, put them in the most full-spectrum lighting available and then have them hold a WB target close to their face. Likewise if you are moving in close to photograph some stationary object, you can make two shots, one with and one without the PVC or other white balance target in the predominant light source illuminating the object to be photographed. But otherwise, I don’t see any practical way to use a target in an “event” type mixed lighting situation.

Regarding the teflon, that’s in my article about making a target shot for making a custom camera input profile. I included the option to use teflon to white balance the target shot based on posts made to the ArgyllCMS mailing list, but since then the list member who suggested using teflon decided it wasn’t such a good idea. My own effort to use the teflon tape also didn’t really work. I need to revise that article and remove the reference to the teflon tape. It’s not the easiest stuff to even get flat when attached to a back support.