Raspberry Pi Camera Sony IMX2019 8MP RAW format

There are now Sony IMX219 Sensor 8MP cameras for Raspberry Pi. Some of them support the CS-Mount for security camera lenses, which makes them really interesting for certain projects.

It is easy to get raw RAW information of the bayer type sensor IMX219.

raspistill --raw -o rawfile.jpg

But this rawfile.jpg is the raw image with the RGGB bayer matrix attached. Ugly!

Utilizing other tools one may extract only the pure RGGB bayer matrix

v4l2-ctl -d /dev/video0 --set-fmt-video=width=3280,height=2464,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=0 --stream-to=test_3280x2464.bayer

creates a file ā€œtest_3280x2464.bayerā€ with only the pure bayer matrix.

a) Is there a plugin of RawTherapee to read pure Bayer data?
b) Is there a simple well documented RAW format to which the bayer data may be converted to be used by rawtherapee? I know of PyDNG which aims to convert RaspiCam Raw to DNG but at first its fails for the PyCam 2.1 and also does the demosaicing by some crude algorithms. I would like to use Rawtherapee for the demosaicing.
c) Are there others interested in programming an import-Filter for Rawtherapee for IMX219 RAW data?

Cheers,

Volker

@volkerjaenisch Welcome to the forum!

Not that I am aware of. It would be neat though for experimentation.

Okay, got me thinking about this. What would be a good start would be to have the bayer array in a TIFF, so that width and height are carried in the metadata. The only other piece of information that needs to accompany it would be the bayer pattern, which libtiff can encode in EXIF as CFAPattern.

Where Iā€™m going with this is that rawproc, my hack software, uses the RawTherapee demosaic algorithms from librtprocess. I donā€™t think itā€™d be too much change to get it to demosaic a TIFF that contains width, height, and the CFApattern in its metadata. To get that TIFF, Iā€™d hack my rawdata program, which is on github at GitHub - butcherg/rawdata: Simple program to make a TIFF image from the raw image, with no processing.

@volkerjaenisch, can you post a sample .bayer file?

RT may be able to accomodate a similar modification, @heckflosse?

@ggbutcher As you know, I like to tinker. :wink:

1 Like

Iā€™ve hacked up a bayer2tiff, here it is:

#include <stdio.h>
#include <stdlib.h>
#include <tiffio.h>

int main(int argc, char * argv[])
{
	char * img;
	if (argc < 5) {
		printf("bayer2tiff: Opens a file containing raw bayer data and saves it to a\n");
                printf("16-bit integer TIFF with absolutely no processing.  The CFAPattern\n");
		printf("EXIF tag is loaded with \"RGGB\" to facilitate subsequent demosaicing.\n");
                printf("\nUsage: $ bayer2tiff width height rawfile.bayer (whatevernameyouwant).tif\n\n"); 
		return 1; 
	}

	printf("Versions:\n\ttiff=%s\n", TIFFLIB_VERSION_STR);

	//opens the bayer file and slurps it into memory
	FILE *f = fopen(argv[3], "rb");
	fseek(f, 0, SEEK_END);
	long fsize = ftell(f);
	fseek(f, 0, SEEK_SET);

	char *bayerdata = (char *) malloc(fsize);
	fread(bayerdata, 1, fsize, f);
	fclose(f);

	//raw_image is an array of 16-bit integers of size width*height:
	int width = atoi(argv[1]);
	int height = atoi(argv[2]);
	unsigned short *rawimage = (unsigned short *) bayerdata;

	//Use libtiff to write the TIFF image file:
	TIFF* tif = TIFFOpen(argv[4], "w");

	if (tif) {
		TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, width);  
		TIFFSetField(tif, TIFFTAG_IMAGELENGTH, height);    
		TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 1);   
		TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 16);

		TIFFSetField(tif, TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_UINT);
		TIFFSetField(tif, TIFFTAG_ORIENTATION, ORIENTATION_TOPLEFT);   

		TIFFSetField(tif, TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG);
		TIFFSetField(tif, TIFFTAG_PHOTOMETRIC, PHOTOMETRIC_MINISBLACK);
		TIFFSetField(tif, TIFFTAG_COMPRESSION, COMPRESSION_NONE);

		//set the relevant CFA metadata:
		static const short bayerPatternDimensions[] = { 2, 2 };
		TIFFSetField (tif, TIFFTAG_CFAREPEATPATTERNDIM, bayerPatternDimensions);
		TIFFSetField (tif, TIFFTAG_CFAPATTERN, "\00\01\01\02"); //RGGB


		//about the simplest TIFF format to write, one big strip:
		TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP, height);
		if (rawimage)
			TIFFWriteEncodedStrip(tif, 0, rawimage, width*height*2);
		else printf("bayer2tiff has a problem...\n");

		(void) TIFFClose(tif);

	}
	else {
		printf("TIFF file save failed...\n"); 
		return 1;
	}

}

All I know is that it compiles. It assumes raw.bayer contains unsigned shorts. It also assumes the CFA pattern is ā€˜RGGBā€™. It can be compiled with this command line:

g++ -o bayer2tiff bayer2tiff.cpp -ltiff

Once I can get a TIFF produced from this file with a raw.bayer input, Iā€™ll see what modifications I need for rawproc, my hack software , to compel it to demosaicā€¦

1 Like

You could also go via DNG in a few steps:

  1. Convert raw pixel data to grayscale PGM using Rawtopgm User Manual
  2. Convert to DNG using http://a1ex.magiclantern.fm/bleeding-edge/pgm2dng.c
  3. Set DNG tags using ExifTool (correct CFAPattern etc)
2 Likes

@kmilos Welcome to the forum!

@ggbutcher @kmilos How do your methods compare with dcraw's document mode.

If dcraw were be able to open a .bayer file, I donā€™t think it would populate a metadata tag with the bayer pattern that is needed for the demosaic routine to interpret the pixels correctly. Otherwise, dcraw -D would produce a TIFF with the bayer mosaic. Iā€™ve made those from Nikon NEFs.

Iā€™m also considering an addition to my demosaic tool that would allow one to specify a pattern. That way, a TIFF wouldnā€™t need to carry that information.

Thank you so much for the fast and professional help!

Here http://fotogruppe-altenstadt.de/speicher/bilderspeicher/volker/2019/imx219/test4_3280x2464-bayer/view
is an example bayer file generated with

v4l2-ctl -d /dev/video0 --set-fmt-video=width=3280,height=2464,pixelformat=V4L2_PIX_FMT_SRGGB10 --stream-mmap --stream-count=1 --stream-to=test4_3280x2464.bayer

I am not sure if V4L2_PIX_FMT_SRGGB10 is the correct input format for your cpp code.

Cheers,

Volker

The videoformat was not correct.

volker@power:~$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ā€˜RG10ā€™
Name : 10-bit Bayer RGRG/GBGB

Uploaded a new bayer file with RG10

http://fotogruppe-altenstadt.de/speicher/bilderspeicher/volker/2019/imx219/test6_3280x2464-bayer

Cheers,

Volker

@volkerjaenisch
rawtopgm should have no problem w/ the 10/12-bit format (as long as itā€™s not packed which is not the case here), just make sure you use -bpp 2 -littleendian

Iā€™m in the middle of sorting out work and home stuff, so my input will be intermittent for few days.

  1. I remember now reading this some time ago, the raspi camera raw file is a bit of a mess of bit offsets that have to be shifted around to make individual unsigned shorts.

  2. I found a previous work in github that reads the raspbi JPEG+RAW jpeg file and makes a DNG of the raw:

It requires a specific libtiff version and the dcraw libtiff patch, which along with the other dcraw pages isnā€™t at cybercom anymore, so Iā€™m stuck right now making it work. Looks to be the most promising way to do this, howeverā€¦

Iā€™ll be back engaged on Mondayā€¦

Found problem. The 10 bit for a bayer pixel are distributed over 5 bytes, where the last byte contains the high bits.

I will experiment with this python code. I will habe to extend the code to write the results into a valid TIFF file with the neccessary metadata.

Cheers,
Volker

Here we go!

@ggbutcher : Your C-Code example was really helpful, since I had no clue on TIFF !

I wrote a really simple python script

rp2tiff.py (1.5 KB)

thats utilizes picamraw and pylibtiff. Picamraw does the heavy lifting of bitshifting the ugly 10Bit encoding into 16Bit from the picamera ā€œRAWJPGā€ and then store it into a suitable TIFF file like @ggbutcher supposes. The main obstacle was that the pylibtiff lib does not support

TIFFTAG_CFAREPEATPATTERNDIM, and TIFFTAG_CFAPATTERN

I fixed this and opened a pull request to pylibtiff.

test.raw.tif (15.4 MB)

I attached the resulting TIFF file.

But where to go now. I have a nice gray scale image. How to procede to demosaicing and colors???

Cheers,
Volker

2 Likes

The challenge is to get the raw processor to treat the TIFF as a raw file. And the essential information it needs is the CFA pattern. Iā€™m going to change my software next week to allow one to specify the pattern in an edit field, and thatā€™ll do the trick. RT might need other such trickeryā€¦

That is the part that interests me the most. I already know how to go from raw to Bayer using dcraw's document mode (might be pre-cropped though). I know how to do rudimentary demosaicing. However, to be able to use AMaZE and friends would be amazing! Especially, if I want to experiment with pre-demosaicing processing.

I am confused

WhenI just remane my ā€œraw fileā€ to *.dng. I get a color image (with heavy green tint) in rawTherapee. This means, that RT interprets the CFA-Tags in part correctly (attributing the colors to sub-pixels), but does the demosaicing wrong.

Is this crude guess corect?

Cheers,
Volker

Sounds like white balance to meā€¦ RT doesnā€™t have access to the ā€˜as-shotā€™ white balance multipliers, so youā€™ll have to do that with the RT tool.

Yes, beside white balance multipliers, there are other colour controls too such as black and white points; balance between the green filtered pixels; and impulse and line filters.

@volkerjaenisch
So the pixel formats reported by v4l2 are erroneous then as ā€˜RG10ā€™ / ā€˜V4L2_PIX_FMT_SRGGB10ā€™ are supposed to really be raw bits with no packing: https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/pixfmt-srggb10.html

Glad you figured it out.

Re DNG, it is based on TIFF container, so no surprise renaming works. Apart from the CFAPattern tag, there are quite a few more needed to get a sensible image like already mentioned black/white levels, white balance, color matrixā€¦ See the DNG spec for details: https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

As mentioned, you can use the exiftool to set those tags on your DNG.

1 Like