Raspberry Pi Camera Sony IMX2019 8MP RAW format

Okay, got me thinking about this. What would be a good start would be to have the bayer array in a TIFF, so that width and height are carried in the metadata. The only other piece of information that needs to accompany it would be the bayer pattern, which libtiff can encode in EXIF as CFAPattern.

Where I’m going with this is that rawproc, my hack software, uses the RawTherapee demosaic algorithms from librtprocess. I don’t think it’d be too much change to get it to demosaic a TIFF that contains width, height, and the CFApattern in its metadata. To get that TIFF, I’d hack my rawdata program, which is on github at GitHub - butcherg/rawdata: Simple program to make a TIFF image from the raw image, with no processing.

@volkerjaenisch, can you post a sample .bayer file?

RT may be able to accomodate a similar modification, @heckflosse?

@ggbutcher As you know, I like to tinker. :wink:

1 Like

I’ve hacked up a bayer2tiff, here it is:

#include <stdio.h>
#include <stdlib.h>
#include <tiffio.h>

int main(int argc, char * argv[])
{
	char * img;
	if (argc < 5) {
		printf("bayer2tiff: Opens a file containing raw bayer data and saves it to a\n");
                printf("16-bit integer TIFF with absolutely no processing.  The CFAPattern\n");
		printf("EXIF tag is loaded with \"RGGB\" to facilitate subsequent demosaicing.\n");
                printf("\nUsage: $ bayer2tiff width height rawfile.bayer (whatevernameyouwant).tif\n\n"); 
		return 1; 
	}

	printf("Versions:\n\ttiff=%s\n", TIFFLIB_VERSION_STR);

	//opens the bayer file and slurps it into memory
	FILE *f = fopen(argv[3], "rb");
	fseek(f, 0, SEEK_END);
	long fsize = ftell(f);
	fseek(f, 0, SEEK_SET);

	char *bayerdata = (char *) malloc(fsize);
	fread(bayerdata, 1, fsize, f);
	fclose(f);

	//raw_image is an array of 16-bit integers of size width*height:
	int width = atoi(argv[1]);
	int height = atoi(argv[2]);
	unsigned short *rawimage = (unsigned short *) bayerdata;

	//Use libtiff to write the TIFF image file:
	TIFF* tif = TIFFOpen(argv[4], "w");

	if (tif) {
		TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, width);  
		TIFFSetField(tif, TIFFTAG_IMAGELENGTH, height);    
		TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 1);   
		TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 16);

		TIFFSetField(tif, TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_UINT);
		TIFFSetField(tif, TIFFTAG_ORIENTATION, ORIENTATION_TOPLEFT);   

		TIFFSetField(tif, TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG);
		TIFFSetField(tif, TIFFTAG_PHOTOMETRIC, PHOTOMETRIC_MINISBLACK);
		TIFFSetField(tif, TIFFTAG_COMPRESSION, COMPRESSION_NONE);

		//set the relevant CFA metadata:
		static const short bayerPatternDimensions[] = { 2, 2 };
		TIFFSetField (tif, TIFFTAG_CFAREPEATPATTERNDIM, bayerPatternDimensions);
		TIFFSetField (tif, TIFFTAG_CFAPATTERN, "\00\01\01\02"); //RGGB


		//about the simplest TIFF format to write, one big strip:
		TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP, height);
		if (rawimage)
			TIFFWriteEncodedStrip(tif, 0, rawimage, width*height*2);
		else printf("bayer2tiff has a problem...\n");

		(void) TIFFClose(tif);

	}
	else {
		printf("TIFF file save failed...\n"); 
		return 1;
	}

}

All I know is that it compiles. It assumes raw.bayer contains unsigned shorts. It also assumes the CFA pattern is ‘RGGB’. It can be compiled with this command line:

g++ -o bayer2tiff bayer2tiff.cpp -ltiff

Once I can get a TIFF produced from this file with a raw.bayer input, I’ll see what modifications I need for rawproc, my hack software , to compel it to demosaic…

1 Like

You could also go via DNG in a few steps:

  1. Convert raw pixel data to grayscale PGM using Rawtopgm User Manual
  2. Convert to DNG using http://a1ex.magiclantern.fm/bleeding-edge/pgm2dng.c
  3. Set DNG tags using ExifTool (correct CFAPattern etc)
2 Likes

@kmilos Welcome to the forum!

@ggbutcher @kmilos How do your methods compare with dcraw's document mode.

If dcraw were be able to open a .bayer file, I don’t think it would populate a metadata tag with the bayer pattern that is needed for the demosaic routine to interpret the pixels correctly. Otherwise, dcraw -D would produce a TIFF with the bayer mosaic. I’ve made those from Nikon NEFs.

I’m also considering an addition to my demosaic tool that would allow one to specify a pattern. That way, a TIFF wouldn’t need to carry that information.

Thank you so much for the fast and professional help!

Here http://fotogruppe-altenstadt.de/speicher/bilderspeicher/volker/2019/imx219/test4_3280x2464-bayer/view
is an example bayer file generated with

v4l2-ctl -d /dev/video0 --set-fmt-video=width=3280,height=2464,pixelformat=V4L2_PIX_FMT_SRGGB10 --stream-mmap --stream-count=1 --stream-to=test4_3280x2464.bayer

I am not sure if V4L2_PIX_FMT_SRGGB10 is the correct input format for your cpp code.

Cheers,

Volker

The videoformat was not correct.

volker@power:~$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RG10’
Name : 10-bit Bayer RGRG/GBGB

Uploaded a new bayer file with RG10

http://fotogruppe-altenstadt.de/speicher/bilderspeicher/volker/2019/imx219/test6_3280x2464-bayer

Cheers,

Volker

@volkerjaenisch
rawtopgm should have no problem w/ the 10/12-bit format (as long as it’s not packed which is not the case here), just make sure you use -bpp 2 -littleendian

I’m in the middle of sorting out work and home stuff, so my input will be intermittent for few days.

  1. I remember now reading this some time ago, the raspi camera raw file is a bit of a mess of bit offsets that have to be shifted around to make individual unsigned shorts.

  2. I found a previous work in github that reads the raspbi JPEG+RAW jpeg file and makes a DNG of the raw:

It requires a specific libtiff version and the dcraw libtiff patch, which along with the other dcraw pages isn’t at cybercom anymore, so I’m stuck right now making it work. Looks to be the most promising way to do this, however…

I’ll be back engaged on Monday…

Found problem. The 10 bit for a bayer pixel are distributed over 5 bytes, where the last byte contains the high bits.

I will experiment with this python code. I will habe to extend the code to write the results into a valid TIFF file with the neccessary metadata.

Cheers,
Volker

Here we go!

@ggbutcher : Your C-Code example was really helpful, since I had no clue on TIFF !

I wrote a really simple python script

rp2tiff.py (1.5 KB)

thats utilizes picamraw and pylibtiff. Picamraw does the heavy lifting of bitshifting the ugly 10Bit encoding into 16Bit from the picamera “RAWJPG” and then store it into a suitable TIFF file like @ggbutcher supposes. The main obstacle was that the pylibtiff lib does not support

TIFFTAG_CFAREPEATPATTERNDIM, and TIFFTAG_CFAPATTERN

I fixed this and opened a pull request to pylibtiff.

test.raw.tif (15.4 MB)

I attached the resulting TIFF file.

But where to go now. I have a nice gray scale image. How to procede to demosaicing and colors???

Cheers,
Volker

2 Likes

The challenge is to get the raw processor to treat the TIFF as a raw file. And the essential information it needs is the CFA pattern. I’m going to change my software next week to allow one to specify the pattern in an edit field, and that’ll do the trick. RT might need other such trickery…

That is the part that interests me the most. I already know how to go from raw to Bayer using dcraw's document mode (might be pre-cropped though). I know how to do rudimentary demosaicing. However, to be able to use AMaZE and friends would be amazing! Especially, if I want to experiment with pre-demosaicing processing.

I am confused

WhenI just remane my “raw file” to *.dng. I get a color image (with heavy green tint) in rawTherapee. This means, that RT interprets the CFA-Tags in part correctly (attributing the colors to sub-pixels), but does the demosaicing wrong.

Is this crude guess corect?

Cheers,
Volker

Sounds like white balance to me… RT doesn’t have access to the ‘as-shot’ white balance multipliers, so you’ll have to do that with the RT tool.

Yes, beside white balance multipliers, there are other colour controls too such as black and white points; balance between the green filtered pixels; and impulse and line filters.

@volkerjaenisch
So the pixel formats reported by v4l2 are erroneous then as ‘RG10’ / ‘V4L2_PIX_FMT_SRGGB10’ are supposed to really be raw bits with no packing: https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/pixfmt-srggb10.html

Glad you figured it out.

Re DNG, it is based on TIFF container, so no surprise renaming works. Apart from the CFAPattern tag, there are quite a few more needed to get a sensible image like already mentioned black/white levels, white balance, color matrix… See the DNG spec for details: https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

As mentioned, you can use the exiftool to set those tags on your DNG.

1 Like

In fact the tint does come from my incorrect asumption on pixel ordering. The order is BGGR and not RGGB.

1 Like

It was long ago and I might be making this up. :clown_face: I recall there being a raw file that started with a BG row/column but in fact had a CFA configuration of RGGB. I supposed the idea was to give headroom for better demosaicing at the edges, or it was the way it was manufactured and could be discarded.