Same phone, different raw... rgbw and rgb versions?

ggc · November 23, 2019, 7:32pm

Thank you.

I don’t have such an image (my phone is the “standard” one, and the weather here is terrible) but i have one with green grass and blue sky with white clouds, and one with a red sofa with a white puppet.OtherImages.zip (28.2 MB)

Attached.

paperdigits · November 23, 2019, 7:38pm

Have you checked their firmware versions as well? Android ships the drivers on one big installable blob.

ggc · November 23, 2019, 7:40pm

Firmware i don’t know… but software, android version and patches are the same.

afre · November 23, 2019, 7:50pm

This and, sorry, I didn’t examine the raw to find out. Also, @snibgo is much more experienced than I. Listen to him.

snibgo · November 23, 2019, 8:36pm

Using OtherImages.zip, messing around with the channels, they seem to be [green blue][red green]. This is neither the common [r g][g b] pattern nor the pattern claimed by exiftool: [b g][g r].

This is strange. When I make ordinary sRGB images with dcraw, they look okay, but I assume dcraw uses the pattern as declared by exiftool, so those sRGB images should look wrong.

EDIT: Ah, okay, the pattern is something weird. Looking at the individual [0 1][2 3] channels, they are all sometimes weird. Enlarging by 500%, one looks like this:

This suggests that this channel is sometimes used for green, but sometimes for something else (eg white). In other words, the CFA isn’t a simple 2x2 array.

ggc · November 23, 2019, 9:16pm

really nice. i think strange pixls are af points.

edit: see this old thread when i found something similar on mine White dots in raw - #7 by heckflosse

snibgo · November 23, 2019, 10:27pm

The pattern for my Nikon D800 is a simple 2x2 array:

R G0
G1 B

Many cameras follow this pattern.

The pattern for @ggc’s Honor 6A (huawei dli-l22 ) phone seems to be a 16x16 array:

G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 x0 G0 B  G0 x1 G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 
G0 B  G0 B  G0 B  G0 B  G0 x2 G0 B  G0 x3  G0 B 
R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1 R  G1

I have marked 4 pixels as “x0” to “x3”. These are instead of the Blue channel. Sometimes they are lighter than surrounding pixels, and sometimes darker. Perhaps these pixels are for auto-focus, in which case we can discard them for demosaicing, or perhaps use them for 3D.

But perhaps these pixels contain useful data. Let’s extract just the “x0” pixels:

These clearly are not 3D values. They seem to be “lightness”. I suspect they are “white” pixels (but I’m not certain).

afre · November 24, 2019, 12:23am

Did some sleuthing. Huawei does make phones with RGBW sensors, though Honor 6A isn’t one of them. 6A’s sensor is PDAF. Could these pixels be reserved for PDAF? I don’t remember how it works.

Is dcraw intelligent enough to read frame crop info or ignore rows and columns until it finds the first rggb or bggr sequence?

snibgo · November 24, 2019, 4:21am

Another point of information: when dcraw creates an ordinary sRGB output, noise is visible in the sky. Each noise area is centred on one of the x0, x1, x2 or x3 pixels. I suppose dcraw treats these pixels as Blue channel, and it shouldn’t.

I see nothing in exif metadata that says these pixels need special treatment.

My version of dcraw is quite old. It does no special processing for this make or model of camera.

I’ll hack my hack of dcraw, to spit out what it thinks the CFA is. But it’s too late tonight.

ggc · November 24, 2019, 8:49am

Meanwhile i have found out where i read that it was using an rgbw sensor:

IMX278 that as far as i understand, is of that kind?

Anyway here is a list of phones with that sensor: Smartphones with IMX278 lens model

No sample on raw.pixls.us… for mine (that is totally “normal”, not this strange version) and some other phones i have uploaded them long time ago.

afre · November 24, 2019, 6:09pm

Sorry, where does it say so? The first link doesn’t say anything about 4-colour sensor or RGBW. The second doesn’t even list Honor 6A…

Claes · November 24, 2019, 6:14pm

Yes, it does, indirectly.
The back (not front) lens is handled by an IMX278.

Have fun!
Claes in Lund, Sweden

Error corrected

afre · November 24, 2019, 6:18pm

Oh, I read the first post again and it does say IMX278. I know no Italian whatsoever but the part name is there. However, the second link still doesn’t list the 6A, and yesterday when I checked Huawei and Honor’s websites, they don’t say that the 6A has a 4-color sensor. For phones that do, they advertise it in a strong way.

Claes · November 24, 2019, 6:21pm

Oh, yes you do know Italian! The word Pizza, for instance.

Apart from that, in case you feel insecure about a foreign vernacular,
you can always use translate.google.com.

Ha det så kul!

Mvh
Claes i Lund, Sverige

ggc · November 24, 2019, 6:31pm

In the first link, i read that the image sensor is a sony-imx278 .

I don’t understand if imx278 is a rgbw or an rgb, and doesn’t matter that much anyway, what would be cool is to be able to develop the files also from that version of the phone.

And anyway, i think there has been a “first release” of the phone with a kind of sensor, than mine with a normal sensor.

snibgo · November 25, 2019, 1:42am

Aarrhhh, sorry, I’m so stupid. I was looking at looking at the image of the green tree and blue sky. But this is potrait format, so the camera was rotated, so the top-left pixel of the final image was the bottom-left pixel as recorded by the camera.

So the bottom-left 4 sensels as captured by the camera were:

b g
g r

After rotation, these become:

g b
r g

So the exif metadata is correct. The CFA really is [b g][g r]. That solves that problem.

The only problem is those four weird sensels in each 16x16 square. Ideally a raw processor would know what to do with them. If they are “white”, it could incorporate them into the demosaicing algorithm (but I don’t know how).

Next best: a raw processor would ignore the values in those sensels, treating them as having no R, G or B value, so these sensels need interpolated values for all three channels instead of just two. Again, I don’t know how.

Worst option: ignore the values in those sensels, but assume they should be Blue values, and make them the simple average of the four nearest Blue sensels. Then the image can be demosaiced in the usual way. And I can do that. The commands are for Windows BAT scripts.

Get the raw sensel values. However, dcraw re-orients to make the image “portrait”, even with the “-j” option:

set CAMERA_SRC=IMG_20191105_134321.dng

%DCRAW% -v -W -o 0 -6 -r 1 1 1 1 -g 1 0 -D -d -T -O cam_gray.tiff %CAMERA_SRC%

Make a list of options for exiftool:

(
  exiftool -args -make -model %CAMERA_SRC% 

  echo -DNGVersion=1.4.0.0
  echo -DNGBackwardVersion=1.3.0.0
  echo -EXIF:SubfileType=Full-resolution Image
  echo -PhotometricInterpretation=Color Filter Array
  echo -IFD0:CFARepeatPatternDim=2 2
  echo -IFD0:CFAPattern2=1 2 0 1
  echo -Orientation=Horizontal
  echo -BitsPerSample=16
  echo -SamplesPerPixel=1
)>exifargs.txt

Note the CFA pattern is correct for this “portrait” image.

Make a 16x16 mask that is white where the sensels are weird, otherwise black:

magick ^
  -size 16x16 xc:Black ^
  -fill White ^
  -draw "point 1,6 point 5,6 point 9,14 point 13,14" ^
  m.png

This is correct for this “portrait” image. For other orientations, it should be rotated.

Do some magick on the image: transform from the range 0-1023 to 0-QuantumRange (eg 0-65535); make every pixel the mean of the four nearest pixels of the same channel; composite that over the original, with a tiled mask, so only the weird pixels are changed.

magick ^
  cam_gray.tiff ^
  -evaluate Multiply %%[fx:QuantumRange/1023] ^
  ( +clone ^
    -define "convolve:scale=^!" ^
    -morphology Convolve 5x5:%KNL% ^
  ) ^
  -size %%[fx:w]x%%[fx:h] tile:m.png ^
  -compose Over -composite ^
  cam_gray2.tiff

Make this TIFF into a DNG:

del cam_gray.dng
exiftool -@ exifargs.txt -o cam_gray.dng cam_gray2.tiff

Finally, test the DNG by using dcraw to make a sRGB TIFF:

%DCRAW% -v -6 -T -O x.tiff cam_gray.dng

The result, x.tiff, is good, There is no visible pattern of repeating noise.

This is not a great solution. The code is klunky, and would be worse if we had to properly process camera rotations. It ignores the data that is in those 4 every 256 sensels, although this might be useful “white” data.

But it works.

ggc · November 25, 2019, 8:44am

Great work!

But i am on linux and i don’t know what is magick (i have imagemagick installed, but i can’t call it with magick).

Thank you!

Claes · November 25, 2019, 9:15am

Hi @ggc,

Presumably your imagemagick is too old? Check your version.
Mine is ImageMagick 7.0.9-2 Q16 x86_64 2019-11-06.

Have fun!
Claes in Lund, Sweden

ggc · November 25, 2019, 9:34am

I have this one, from ubuntu 18.04 packages…

imagemagick 8:6.9.7.4+dfsg-16ubuntu6.8

snibgo · November 25, 2019, 3:43pm

Yes, my commands use ImageMagick. I’m using version 7. For version 6, use “convert” instead of “magick”.

In v6, “-evaluate Multiply %%[fx:QuantumRange/1023]” won’t work. Assuming your ImageMagick is Q16 (use “convert -version” to verify), you can substitute “-evaluate Multiply 64.06158357771261”.

In v6, “-size %%[fx:w]x%%[fx:h]” won’t work. Substitute the size of you image, eg “-size 3136x4224”.

I expect this work could be done with G’MIC, but I don’t know how.

An alternative might be to declare the weird pixels to be “dead”, and use dcraw’s “-P” option. I’ve never tried this.