It has been years since I shot RAW images, something I’m taking up again now. My recollection is that the RAF files I processed in the past appeared dull and lifeless when initially processed and had to be edited, moving Exposure, Contrast, and Saturation sliders around to get a decent image. (Can’t recall what converter I used at the time.)
Now, I’ve ventured back into RAW processing starting with downloaded darkroom styles applied to DNG files created on Pentax Q-S1. (Yes, I know shooting RAW with a sensor that small may be a waste of time, but that’s another issue.) The styles work exactly as I hoped, Velvia in particular brightens and saturates the images exactly as anticipated (too much at times, but choosing and adjusting is part of the fun).
What confuses me a bit is the following: Even before applying a style, or any edit, the DNG images displayed already look great, not always exactly what I want, and so the styles are often useful, but not dull and lifeless either. It seems to me that, by default, darkroom is processing the DNG files from the camera just as the camera itself would if asked to create a jpeg, with punched up colors, etc. This is not a problem of course but I’m surprised that the DNG files contain such default processing information, if they do. And so I’m wondering if this is what is happening or whether there is some other explanation for the (good) look of the DNG files in darktable before I do a thing to them.
DNG is a container format, which means it can hold a bunch of different types of information inside it. If you’re using darktable and the initial opening of the file doesn’t look dull and flat, there is a decent chance your DNGs do not actually contain raw data, but rather a tiff or jpeg or simlar.
As @paperdigits says, DNGs do vary quite a bit.
AFAIK, where default processing is concerned, except things like lens type for the lens correction module, white balance, and ISO/camera model for noise reduction, darktable doesn’t recognize any processing info from the camera.
I stand to be corrected on this, but I’m fairly confident that’s the case normally.
Yep. That still stands, in most cases, not withstanding the possibility of your (very nice!) little Pentax producing DNGs that may not be truly “raw”.
Darktable does however (by default) auto apply some stuff…
In the darktable preferences, under the processing tab, you can choose your default workflow:
Both the sigmoid and legacy (base curve) options will produce a fairly colourful and punchy image on initial loading. The filmic option not so much, although it still adds some contrast, while the “none” option will give you an unaltered image, except for the “housekeeping” stuff like demosaicing, basic white balance and input profile.
To see what’s applied, you can go to the active modules tab in the darkroom.
This screenshot shows everything that’s applied when I load an image from my Pentax K10d, with darktable set to the scene-referred sigmoid workflow.
I could write a lot more about all this but don’t want to cause info overload.
If you’d like to, I’d love a Play Raw post with a DNG from your camera.
[edit:]
Waste of time? Never! I shoot DNGs on my phone (which are an interesting example of what I would call partially processed DNGs) but there’s so much more latitude for adjustment than with the slightly overdone JPEGs.
This is all really interesting, and informative. So, I’ll play around with the workflow defaults and see what I discover. One thing I’ll add here, though, is that I took a number of shots with a Pentax Q wide-angle lens to see whether darktable recognized the lens and corrected what must be enormous distortion. (Again, the sensor is tiny.) When I saw no distortion and saw darktable correctly identify the lens and focal length in the info field, I assumed that darktable did indeed know how to adjust the RAW images for the distortion. But now I’m thinking that none of this is true and the DNG file for this camera is just a Pentax gimmick that stores JPEGs.
Anyway, since you asked, here is a link to the test images I’ve taken on the Pentax Q-S1, including both DNGs from the camera and JPEGs created by darktable, some using the default setting some a Velvia style I downloaded: Test Images. (No works of art here, just test images to play with in darktable.)
Anything anyone can discover about these images would be something I’d be curious to know… As always, thanks in advance.
Never a waste of time. I shoot a lot of travel images with an Olympus TG6 because it is tough, waterproof, mudproof etc. The RAW files give pictures way better than the JPG images.
BTW, I own a Pentax D-SLR which can store images as either RAW or DNG. I have chosen to use the Pentax raw file format PEF so I know which camera took it and that it truly is a RAW. A DNG appears like a RAW but seems to have various degrees of processing done to it. Maybe that is beneficial to you is the result look good from the start, but I suspect there is more to the story with your DNGs. If you could post an image maybe we can delve deeper into it. Maybe also take a picture in RAW and DNG and compare the results. It would be interesting to know (at least to me).
Thanks. As I mentioned above: Here is a link to the test images I’ve taken on the Pentax Q-S1, including both DNGs from the camera and JPEGs created by darktable, some using the default setting some a Velvia style I downloaded: Test Images . (No works of art here, just test images to play with in darktable.)
Well, I’ve been a little short on time since you posted the link (thank you!) but I’ve had a bit of poke around now.
The DNGs from the Q-S1 actually have similarities with those from my phone.
That is;
they are un-demosaiced
they are not white balanced (if I switch off the color calibration and white balance modules the image turns bright green)
there’s some noise in the shadows
there’s a decent dynamic range evident when doing exposure adjustments
they do rather appear to be distortion-corrected.
All those points except the last say, “RAW!”. But the last point is not consistent with what I think of as a “normal” raw file.
As to the colour saturation, they are on the vivid side for an unaltered, normal RAW file, but not unduly so - and they are quite low contrast, with sigmoid or filmic turned off, which is normal.
I guess my takeaway, is that for most intents and purposes, they are genuine RAW files. I’ll try to see if I can find any info on the lens correction thing. I’m curious now…
I should have elaborated my possible faulty thinking there… sorry. I’m just going off the fact that the images appear remarkably distortion free, despite being shot with a wide angle zoom. (17.5-28mm equivalent)
All the compact cameras I’ve known have noticeable barrel distortion corrected in software.
Of course the Pentax Q series isn’t exactly an average compact…
Edit: I’m getting more and more doubtful about my reasoning… can’t find much online except that lensfun (as used in darktable) does support the 01 prime lens.
Which kind of suggests that it does need correcting…
Look like those DNGs might just be straightforward raws and the 08 Wide lens is just an awesome optic!
Umm. After more browsing through @Lobalobo’s images, I notice that some, like IMGP2319 do have barrel distortion. No correction there. Thanks @nosle for making me think about my daft assumptions.
Thanks so much, Steven. The 08 lens is, in fact, by far, the most expensive part of the kit, so I wouldn’t be surprised that it corrected a good deal on its own, but I never imagined that it could correct as much as it did; I expected the RAW image to be fisheye shaped.
Sounds as if, then, that darktable simply has defaults that punch up the images more than I expected (and I’m not experienced enough with darktable yet to identify which modules are providing those default corrections).
The one remaining issue I have is about how to correct the remaining distortion you’ve noticed. When I go into the lens correction module on darktable the lens isn’t recognized, this even though on the info tab in the software identifies the lens as well as where it was zoomed to on capture. This combined with the fact that Lensfun lists only the 01 Prime (and you say that darktable uses lensfun) suggests that darktable does not in fact have lens corrections for the other Q lenses.
There are four Q lenses worth correcting: the 01 Prime as well as the Wide, Standard, and Telephoto Zooms.) If Lensfun or darktable accepts contributions of lens images, I’ll add them next time I have the camera with me, which I don’t now.
It could also simply mean that the gamut captured by the colour filters on your “older” camera is smaller and closer to your monitor and sRGB gamut so it already appears vivid enough.
That’s interesting. I hadn’t thought of that. Makes sense, though. (even to me!) It could explain why my even older K10d and Sony NEX5 do have raw files noticeably more ‘colourful’ than my newer Nikon.
It will be either the sigmoid, filmic or base curve module. If your preferences are standard, it should be filmic I think. You can, by the way, switch modules off and on with the little power button at left
As far as I know, the only way to get corrections is to get them added to lensfun - see Lensfun
You can supply the images used for the lens profiles via the link on that page.
I must confess I don’t know what the timeline is for this kind of thing to work it’s way through the system.
I’ve sometimes thought it would be nice if darktable had a generic kind of manual distortion correction module - but it doesn’t.
It’s possible for DNG files to contain lens correction embedded in the metadata in a standard format, and the next release of darktable will support it, but the Pentax files posted here do not contain those tags.
IMGP2329.DNG.xmp (14.5 KB)
Hi Barry,
I just download one of the supplied test shots. I can’t add to the extensive comments made here, but the image I downloaded was not great out of the box and need typical processing that I would apply to RAW files. I feel it would be of benefit for you to take two identical shots with one as DNG and other as PEF (Pentax Raw format).
