[Possible bug] Raw Black/White Point module: Unexpected replication/appending of duplicate pixels to right side of image pipeline

Software darktable
1

Dear Devs,

First, congrats on a truly remarkable v3 release. I’m so impressed with the great work that has gone into this latest release of darktable.

Second, I’m seeing a replicable and extremely unusual/unexpected behavior that seems to be tied to the Raw Black/White Point module and/or potentially interaction with Lens Correction modules. I am using the official release v3.0.0 on Mac OS X.

I first noticed this unexpected behavior when copying/pasting a stack of edits from one Raw file to another.

What I observed is the edits pasted in and did the expected things, but in addition I am seeing a new and unexpected vertical slice of pixel data (about 1.5% of the image, or for my camera and image, this turns out to be a 78 pixel wide vertical slice) pasted/appended to the right side of the image. This slice of pixel data is not random - it is duplicated from another more central part of the image and appended to the right side of the image.

This comparison/culling view screen capture illustrates the behavior (please ignore the garish colors; the edit stack I copied pasted from an interior shot with completely different white balance).

Screen Shot 2020-03-05 at 8.46.21 AM
Screen Shot 2020-03-05 at 8.46.21 AM1902×1007 2.06 MB

To see if I could isolate and replicate where in the pipeline the problem was occurring, I walked through the history stack starting at original and then adding one edit at a time to see where the extra pixel data was appended. I observed that the new vertical slice of data was added on the Raw Black/White Point module step in the pipeline.

I tested this by making a new virgin duplicate image, and copying/pasting the editing stack while unchecking the Raw Black/White Point module. When I do this, I end up with a normal/expected image without the vertical stripe of extra image data.

This seems to confirm to me that the problem may be linked to the Raw Black/White Point module step.

I have saved the problematic stack as a style and exported it, and attached it here if it is useful for trying to diagnose or fix the problem.

test_append-to-right.dtstyle (9.0 KB)

I may test uninstalling and reinstalling darktable to see if somehow there’s been a corruption a config file somewhere that’s led to this problem.

If I can provide any additional assistance in diagnosing/testing/fixing this issue, please let me know.

Thank you!

2

I checked your style and while something weird does happen on my sample image I do wonder how and why. It doesn’t happen the same as your image. Have you messed around with raw point settings? it seems like those shouldn’t be messed with. Maybe the rawpoint setting mess around with modules later in the pipe, since if i reset rawpoint to defaults I don’t have problems.

1 Like
3

I’m, like @johnny-bit, also inclined to point to the raw black/white point module as a possible culprit.

It is however not sensible to apply your style to other images and make sensible conclusions based on that. The images all have a different base/starting point, not limited to but including the raw black/white point setting.

It would be helpful if you, @mmmtv , could upload one of the RAW’s that goes with the style stat was uploaded and that shows the behaviour you mention.

1 Like
4

As far as I know, I have not made any changes to the raw black/white point settings.

Perhaps i did so by accident.

But I never touch this module as part of my intentional workflow.

5

I don’t believe the problem is with any particular RAW file itself, but if the problem is in the interaction with my Panasonic GX85’s RAW file format and this module, I’ll be happy upload one or a few of my RAW images in a while to see if it can be replicated by someone else. Thanks!

Updated: Attached is a sample RAW file where I was seeing this problem.20191206_P1240098_0080.RW2 (18.9 MB)

I’m explicitly sharing this image and others I’ve posted or will post on this thread under standard GFDL permissions - no attribution needed.

6

I just had a look and did some tests with the RAW you provided:

This does seem to be related to both the raw black/white point and this specific style.

If I strip down the test_append-to-right.dtstyle you posted to the 8 basic modules and apply that to a clean image I get the strip on the right side of the image.

base.8.dtstyle (3.0 KB)

Resetting the raw black/white solves the immediate problem.

If I remove the raw b/w module from the style all is well.
base.7.min.rawbw.dtstyle (2.6 KB)

However, I am not able to reproduce this starting with a clean version of your RAW doing some edits myself (involving one or more modules you also used in your style), creating a style from that and reapplying it to a clean version.

Something in the style you posted makes it behave this way.

So I can confirm that this is happening when using this style and this RAW image.

I don’t have a reason why this happens. I’ll have another look when time permits. Maybe others recognise this issue and can provide you with a reason/solution in the meantime.

7

Had another look at this.

I downloaded 2 other DMC-GX85 RAW’s and applied the base.8.dtstyle: The issue also shows up on those. Other Panasonic type’s (as far as I tested them: 4 other types) do not show this though.

Haven’t been able to find what actually causes this.

I’m not seeing any related messages in the terminal. A simple manual reset of the raw black/white point modules solves this. You can create a style after the reset and all works as expected.

1 Like
8

Thank you for the replication and investigation, Jacques. I appreciate you providing a simple workaround for the problem. It is curious that the issue replicates for the GX-85 RAW but not other Panasonic RAW files. I also own a Panasonic FZ-300 and I tested applying the problematic style to a sample RAW file from that camera. I also did not observe/replicate the “appended pixel” problem manifest. So it definitely seems to be tied somehow to the interaction of a particular RAW format with that module.

This sounds like a very odd edge case kind of issue, so not a high priority for the dev team to tackle.

Cheers,
Mark