Band of slightly darker pixels along one border of raw image

Processing
1

Hello,

The raw images of my camera, an Olympus E-M5 III, show a band of darker pixels at the extreme right - and only there. This area is cropped away on out-of-camera JPEGs, but other than for the somewhat darker tone, the area looks perfectly usable.

Strangely, the dark band is not always visible. So far I have no understanding under which conditions it appears. (But it appears often enough.)

Any ideas? I’d be grateful for suggestions about the origin of the phenomenon and how to best deal with it. (I think I could define a crop preset, but then I’d have to manually check the border of each image. Alternatively, I could try to configure darktable to get rid of the border even before the pipeline, but then I will be throwing perfectly good pixels away.)

Thanks!

Here is a crop from an example:

darker_border
darker_border352×1003 107 KB

Here is the full darktable edit, as well as the raw file:

PA060595
PA0605954756×3228 3.31 MB

PA060595.ORF (17.4 MB)
PA060595.ORF.xmp (11.8 KB)

2

I get this with some cameras. It is not really a fault in my view. It is just DT is not cropping the edge pixels that the camera crops. It was so common with my Canon G16 that I made a crop preset that I could apply to the image to remove the few pixels. Others may have a more informative answer for you.

3

Probably of no help, but some makers if not all use sensor border pixels for special purposes. For example, my raw viewer can include or exclude masked pixels from the review image.

I can only speculate that those dark pixels could be used to detect the onset of over-exposure. Such pixels are included all over the Foveon F23 sensor, not in the borders.

1 Like
4

A common issue with Darktable camera database. Some time ago I actually submitted a pull request to dump these perfectly good pixels in raw files from my G9 because the predefined crops were unreliable respectively to the various camera modes and their combinations.

5

Thanks for the replies so far!

Having a closer look at the issue, the darker band appears also in an image converted using the dcraw tool. That image has the resolution of 5240 * 3912 which, according to dxomark, is the (full) resolution of the sensor.

The width of the band is exactly 32 pixels throughout.

Is there a way to trim these pixels early on (before lens correction module), without affecting the work of the lens correction module? The crop module operates on the lens-corrected image, so it’s not ideal.

6

There is indeed. Search the forum for the “allow_editing_crop” workaround, and you will then get the expert crop controls in the rawprepare (raw black/white point) module.

7

Perfect, thanks a lot, I didn’t find this myself. Now I created an auto preset.

8

Hello again,

So I’ve been using the solution suggested by @kmilos (=using the raw black/white point module to get rid of the column of darker pixels) for some time now, and it works well for this particular problem. However, it introduces another problem: it prevents the raw black/white point module from honoring the BlackLevel EXIF MakerNote tags. This introduces a color cast in (heavily) lifted shadows.

The problem is that on my camera (Olympus E-M5 III) the EXIF BlackLevel varies from image to image. It seems to depend not only on ISO, but also on things like shutter mode (mechanic vs electronic) and probably yet other parameters.

Is there a way in darktable to create a preset that only modifies some values of a module? So far I was not able to find one.

9

Looks like I’ll have to learn how to write Lua scripts for darktable:

10

It does not seem possible to dynamically adjust module defaults upon import using Lua. (See here for details.)

In the end, I defined a shortcut that sets the right crop to 32. This is possible directly within the GUI. Now I can apply the crop whenever necessary (curiously, the band of pixels is not always present). This is not a perfect solution, but it will do.

1 Like
11

https://www.dpreview.com/forums/thread/4815412

"OK, mystery solved! The discontinuity/circular banding, and also the dark band along the right border are all baked into the RAW file by Olympus’ vignetting correction called “shading compensation”.

I switched this “feature” off, and now I can even use all the 5240 x 3912 pixels without any weird bands or other such artifacts if I like."

1 Like
12

That was me over at dpreview, just with a different hat on…

I’m working on a proper fix to this problem. So far I have this in my luarc: the RAW crop is applied only when necessary:

local dt = require "darktable"
local df = require "lib/dtutils.file"

local SEP = dt.configuration.runnin_os == "windows" and "\\" or "/"

local function shading_compensation_on(image)
    local exiv2 = df.check_if_bin_exists("exiv2")
    if not exiv2 then
        dt.print(_("Unable to find exiv2 command."))
        return false
    end

    local p = io.popen(exiv2
        .." -K Exif.OlympusCs.ShadingCompensation "
        ..image.path..SEP..image.filename)
    if not p then return false end

    local line = p:read("*l")
    p:close()
    return line and line:find("On", 1, true) ~= nil
end

local function crop_right_border_on_import(event, clean, image)
    if not clean then return end
    if image.exif_maker ~= "OLYMPUS CORPORATION" or image.exif_model ~= "E-M5MarkIII" then
        return
    end
    if shading_compensation_on(image) then
        dt.gui.action("iop/rawprepare/crop right", "value", "set", 32)
    end
end

dt.register_event("em5_iii_crop", "darkroom-image-loaded", crop_right_border_on_import)

For the above to work, plugins/darkroom/rawprepare/allow_editing_crop must be set to true in darktablerc.

If people see this effect with different camera models, could they please post their names here?

1 Like
13

Here is a much improved version that

  • If “shading compensation” was enabled, crops the RAW image to remove the dark band.
  • If “shading compensation” was disabled, sets up an instance of “lens corrections” using “Lensfun” to provide a much better correction.

For me this seems to work quite well. See the comment for more information.

-- Automatically handle Olympus' "shading compensation" when an image is loaded into the darkroom.
--
-- If "Shading compensation" was active, crop a 32 pixel wide column from the right border of
-- the RAW image, since it has no effect there and this results in a dark band.
--
-- If "shading compensation" has been disabled (recommended), create a new instance of
-- "lens correction" and set it to "Lensfun" correcting "only vignetting", unless such an instance
-- already exists.  To prevent its recreation, disable the instance instead of deleting it.
--
-- For the RAW crop to work
-- plugins/darkroom/rawprepare/allow_editing_crop must be set to true
-- in darktablerc

local dt = require "darktable"
local df = require "lib/dtutils.file"

local SEP = dt.configuration.runnin_os == "windows" and "\\" or "/"

local function shading_compensation_on(image)
    local exiv2 = df.check_if_bin_exists("exiv2")
    if not exiv2 then
        dt.print(dt.gettext.gettext("Unable to find exiv2 command."))
        return nil
    end

    local p = io.popen(exiv2
        .." -K Exif.OlympusCs.ShadingCompensation "
        ..image.path..SEP..image.filename)
    if not p then return false end

    local line = p:read("*l")
    p:close()
    return line and line:find("On", 1, true) ~= nil
end

local function maybe_setup_lensfun_vignetting_correction()
    -- Search for an instance of "lens correction" that is set to "Lensfun" and "vignetting".
    -- We only search the first 5 instances - is there a better way?
    for i=1,5 do
        if dt.gui.action("iop/lens/correction method", "selection", "popup", "", i) == -2
           and dt.gui.action("iop/lens/corrections", "selection", "popup", "", i) == -8 then
            -- Assume that vignetting correction is set up already (active or inactive).
            return
        end
    end

    -- Not found: Create a new instance and set it up.
    dt.gui.action("iop/lens", "instance", "new", 1,000, 0)
    dt.gui.action("iop/lens/correction method", "selection", "item:Lensfun database", 1,000, 0)
    dt.gui.action("iop/lens/corrections", "selection", "item:only vignetting", 1,000, 0)
    dt.gui.action("iop/lens/scale", "value", "reset", 1,000, 0)
end

-- See https://discuss.pixls.us/t/working-with-modules-from-lua-scripts-in-darktable/32934/224
-- and https://discuss.pixls.us/t/need-your-help-with-a-lua-script/49259/2
local function handle_oly_shading_compensation(event, clean, image)
    if not clean then return end
    if image.exif_maker ~= "OLYMPUS CORPORATION" or image.exif_model ~= "E-M5MarkIII" then
        return
    end
    local sc_on = shading_compensation_on(image)
    if sc_on == nil then
        return
    elseif sc_on then
        dt.gui.action("iop/rawprepare/crop right", "value", "set", 32)
    else
        maybe_setup_lensfun_vignetting_correction()
    end
end

dt.register_event("em5_iii_crop", "darkroom-image-loaded", handle_oly_shading_compensation)

So far the E-M5 Mark III is hardcoded (it can be changed). The next step will be to check which other Olympus/OMS cameras (all?) are affected.

If anybody else tries this out, please let me know whether it works for you.

1 Like