Demosaicing disabled as not appropriate

Any clue about this one? Everything still works and I can change demosaicing method. A restart may remove the message. I have been shooting CR2 + JPEG. When I open CR2, no error. Move to JPEG and I get the warning, of course. Move back to the same or other CR2 and now I also get a warning there.
Skärmbild från 2021-07-10 14-09-34

CC0 https://drive.google.com/file/d/17xexNmZyQmv6daGNyfhuiCB1V5YxxDJR/view?usp=sharing

demosaic is for RAW only, so that’s absolutely normal. A jpg doesn’t need demosaic

CR2 is raw.

For JPEG it looks like this and that is normal:
Skärmbild från 2021-07-10 14-52-52

On my 5DmkII, when I first got it, I shot some images with a smaller size raw file.

It seems to do some processing, including demosaicing (and some other raw features, such as setting the black point, and I think some other transforms for the highlights and such too). I’ve seen something like this in some of my own files.

The format is called “sRAW” or “mRAW” in Canon cameras. (And I think other manufacturers have something similar these days too.)

In the case of mine, it looks like this:

If you run the command “exiftool” on the file and lok for “sRAW” (or “mRAW”, depending on your settings), I bet you’d probably see something similar.

Bad news: It’s a (bad) tradeoff between file size and quality. The images are forever stuck somewhere in-between fully-baked JPEGs and “unbaked” raw files.

Good news: They’re higher quality than JPEGs and there are still a bunch of settings you can adjust that just isn’t possible with JPEGs, such as white balance. They’re still somewhat raw-ish. You can still get good results processing the files.

(The one in my screenshot was extremely underexposed and Canon cameras seem to shift underexposed, dark parts of pictures into green sometimes, so it shouldn’t be fully representative of sRAW/mRAW.)

Suggestion for the future: If you’re currently shooting with a reduced small or medium raw file format, I strongly suggest switching to full raw. It’s a bit larger in file size, but definitely worth it for the quality.

(Aside about using any format other than the full, native camera raw: I’m happy I didn’t shoot too many photos with sRAW. I also did convert too many to DNG, but that’s another story… at least converted DNGs are usually—but not always—closer to what the camera gave in its native raw format.)

As you can see in the Google drive link, these files are not mRAW or sRAW.

Other sample. Same CR2 file. Opened CR2 in darkroom, moved to JPEG and moved back to the same CR2 file.


Are you sure that they’re not sRAW?

$ for i in *CR2; echo "$i"; exiftool $i | grep -i sraw; end
IMG_0351.CR2
SRAW Quality                    : n/a
SRaw Type                       : 1
IMG_0352.CR2
SRAW Quality                    : n/a
SRaw Type                       : 1
IMG_0353.CR2
SRAW Quality                    : n/a
SRaw Type                       : 1

All the CR2 files in your zip file show that they’re sRAW type 1.

darktable really should indicate if a file is sRAW, mRAW, etc. (I haven’t found a place where it’s indicated. I guess we should file an issue.)

Compare with sRAW from 6D. https://raw.pixls.us/getfile.php/801/nice/Canon%20-%20EOS%206D%20-%20sRAW2%20(sRAW).CR2

Moving from sRAW to TIFF and back to sRAW gives the same message.

Oh, I’ve imported them here into darktable 3.6.0 and they don’t show a demosaicing error or the lack of demosaicing… they show just a normal demosiac module. This is really odd that you’re getting that error. Apologies for thinking they are sRAW.

If I do look at a later file of mine that I know isn’t sRAW, I also get a “SRAW Quality” of
“n/a” and “SRaw Type” of 1 — just like with your files.

What happens if you clear the demosaic history on one of your files with an error? (Either just for the demosaic module with the icon second on the right with the ⏼ icon or the complete history stack.) Perhaps darktable doesn’t like the demosaicing specified in the XMP file?

I know darktable 3.6 brought some changes in the demosaicing, including a new type (RCD) and also hybrid types (two merged demosaicing methods, one optimizing for the general image, another for the sharper parts and then combines them). Perhaps a bug crept in with some existing files that were already processed? Just a guess.

Update: I’m also getting the error switching between images here. I see exactly what you mean. This is a weird bug. If I switch photos using spacebar or backspace in the darkroom from JPEG to raw, then I hit this. Switching back to lighttable and selecting the raw to go to darkroom works as expected.

(And clearing the history does not help.)

This is definitely a bug in darktable and should be reported. I’ll see if I can find other cameras where I shot raw + jpeg to see how widespread this is.

Doesn’t work to reset. Only restarting darktable works.

Maybe the error is in older XMP file for JPEG.

Edit: No error if I clear XMP files.

After spending far too long looking for RAW+JPEG shots (which I apparently really don’t have much of), I finally convinced darktable to recreate this same problem when switching between a panoramic TIFF file in my photo collection and a RAF (Fuji X-T2 raw file).

At first, it didn’t have an issue when switching between the images. The demosaic module was disabled and hidden by default with the TIFF file and I could switch between the two just fine. But when I forced the TIFF to have the module by clicking the ⏼ icon (to “revert” to default settings), it turned the module on (but didn’t use it of course), even if I reset the history for the file. And then when I switched from the TIFF to the RAF, the RAF also said that it’s not applicable (when it totally is).

So I was able to reproduce this with a completely different camera (even with a different mosaic pattern to demosaic). The problem isn’t your images, isn’t your camera, and isn’t even the brand of camera or the method of demosaicing. The problem is 100% with darktable’s logic to show the warning and hide the module for non-mosaiced images when switching between photos. And I am pretty sure that changed between darktable 3.4 to 3.6.

It probably wasn’t found as it’s a bit of a corner case, as most people probably usually shoot exclusively raw, not a mix… or even if they do, they probably have demosaicing completely off and absent from JPEG, TIFF, and DNG files… so they probably didn’t hit this.

This is definitely worth opening an issue for. Hopefully the devs can fix this in a point release. (Hopefully it’s an easy fix where the module is just reinitialized per photo or something like that. It feels like the value of a variable is getting carried over between images or something like that.)

1 Like

What you see is just correct, the point is: your history for a non-raw image has the demosaicing module enabled which is absolutely incorrect and so you get a message about it. Take this as a warning that you did something not very wise and darktable gives you a hint :slight_smile:

How does that happen? You have very likely applied a style or have copy&paste part of the history from a raw file to a jpeg (or whatever non-raw file you are using).

@hannoschwalm: The issue isn’t with the non-raw image, but with a raw image.

It’s after switching from a non-raw image that incorrectly (for whatever reason, likely a previous setting in the XMP or database) has a demosaic module on (that isn’t doing anything but displaying an error).

After recreating the issue here (see my comment above), I attempted to switch between demosaic types in the raw and it worked just fine. And it was also demosaicing the image fine as well.

This seems to be a cosmetic issue only, where there’s an error with a banner at the top of the module and a warning icon in the module’s header.

I think the actual error might not even be with demosaicing itself, but darktable not resetting warning/error banners and icons for module states when switching between images. But I don’t know darktable internals, so this is just a guess.

This issue could also get triggered when someone copy/pastes settings from a full raw file to a DNG or an sRAW/mRAW file (which looks exactly like a raw file in darktable, including the filename on disk, except it doesn’t have to demosaic in darktable… otherwise it looks and acts the same as a full raw file).

And since it seems like it’s really a warning/error display issue not being reset when moving between files in the darkroom part of darktable, these false errors could show up in other modules as well. (Again: Just guessing here about other modules.)

Thanks for reporting - although the better place would be github as this got detected only by luck :slight_smile:

I know the causing dt code sections quite well and will have a deeper look.

4 Likes

@Peter, @hannoschwalm:

Filed an issue with the relevant parts of this conversation @ darkroom: warning/error banners do not properly update when switching images · Issue #9497 · darktable-org/darktable · GitHub

1 Like

That sounds like what a linear DNG is also. Some programs such as the Topaz programs and DXO PureRaw can produce them as output.

Not necessarily. sRAW is usually lower resolution and perhaps has lossy compression applied. Linear raw DNG is mostly high bit depth, original resolution, and lossless, you just don’t have control of the demosaic step.