Loading .raw images (from industrial cameras)

Software RawTherapee
#1

I’m working with industrial cameras (like the Basler ace series) and I would like to debayer and process the raw images with RT. From the provided camera software, the easiest way is to set the format to “Bayer mode” and simply click “save” to store the image as 16 Bit .raw without any metadata.

In some cases these images can directly be processed with RT:

When saving the raw image data of 1920x1080 images in 16 Bit format as a .raw file, the file is shown in the RT file manager and can be opened/processed.

With cameras of different resolution, the .raw file is not shown in the RT file manager.

However, opening the larger .raw with imageJ, cropping to 1920x1080 and saving enables the loading of the images. I assume that this behavior is tied to a magic file size that tells the x/y resolution of the image.
Where does this behavior come from?

I would like to add a few more combinations just for quick image browsing and handling of typical image resolutions. I understand that the proper way is the definition of a .dng format, but the out-of-the-box Basler gui is does not have this (and I don’t have control over this). Alsio, this way will not include any colorspace information and might result in poor colors.

Thanks!

#2

I found something in libopenraw - there is a struct with filesize definitions:

static const struct {
unsigned fsize;
ushort rw, rh;
uchar lm, tm, rm, bm, lf, cf, max, flags;
char make[10], model[20];
ushort offset;
} table[] = {

and a list of cameras.

{ 4147200,1920,1080, 0, 0, 0, 0, 0,0x49,0,0,“Photron”,“BC2-HD” },

I think this could be filled with some details of recent image sensors.
For example sensors from Sony, like IMX183, IMX 226, IMX267 are used in many cameras from all industrial camera manufacturers.

Is this a good way to proceed?

#3

Please also take a look at Raspberry Pi Camera Sony IMX2019 8MP RAW format

#4

I don’t feel this is a good way to proceed, as file size is not unique enough to identify a camera. For example the same sensor can be mounted upside-down and read/scanned in the opposite order by a different camera manufacturer making the RGGB CFA pattern actually BGGR in the file of the same size, so how would you match it uniquely in the example above (requires a different cf value)?

So I think you have to carry some extra metadata with your .raw file somehow, and creating a DNG seems the best route to me if you want to process with RT (or other raw developers). It does unfortunately mean some extra work to fill in some other sensible DNG color related metadata values…

1 Like
#5

Thanks for the help - the .DNG route is certainly the way to go for our own software.

The industrial cameras also come with ready-made software. They can only save as .raw without any metadata (which is currently of no use for RT). For this situation I would like to propose a patch.

We managed to use the .raw files off from Basler cameras (as saved in the pylon viewer gui, camera set to 12 Bit format, saved as .raw) for these systems:

  • Basler acA4096 series with Sony IMX267 sensor
  • Basler acA1920 series with Sony IMX174 sensor
  • Basler acA5472 series with Sony IMX183 sensor

by adding the following lines to the file dcraw.cc camera table:

{  4147200,1920,1080, 0, 0, 0, 0, 0,0x49,0,0,"Sony","IMX174" },
{ 17760256,4096,2160, 0, 0, 0, 0, 0,0x94,0,0,"Sony","IMX267" },
{ 39923712,5472,3648, 0, 0, 0, 0, 0,0x94,0,0,"Sony","IMX183" },

Is there a chance that anybody could merge this into the main tree?
We would have these cameras on hand and could also supply test images if necessary.

#6

A better solution would be to allow adding support for custom formats through camconst.json.

#7

Alternatively, taking the raw data and converting it to a TIFF, then adding DNG tags as appropriate would work.

I’ve been doing the same to get FITS images from siril back to DNG. ImageMagick’s convert utility to get from FITS to TIFF, rename .tif to .dng, then run the following script to add DNG tags:
#!/bin/sh

exiftool -DNGVersion="1.4.0.0" -DNGBackwardVersion="1.4.0.0" -ColorMatrix1="0.5271 -0.0712 -0.0347 -0.6153 1.3653 0.2763 -0.1601 0.2366 0.7242" \
         -IFD0:BlackLevelRepeatDim="2 2" -IFD0:BlackLevel="512 512 512 512" -IFD0:WhiteLevel=16380 \
         -PhotometricInterpretation="Color Filter Array" -CalibrationIlluminant1=D65 \
         -SamplesPerPixel=1 -IFD0:CFARepeatPatternDim="2 2" -IFD0:CFAPattern2="0 1 1 2" $1

http://www.imagemagick.org/discourse-server/viewtopic.php?t=31451#p143093 might help for converting your specific raw data to TIFF. That example is for 8-bit RGB, but it should be easily adaptable to 16/32-bit monochrome. Note that the CFA patterns and color matrices/black levels above are specific to a Sony A7M3 - you’ll need to change your color matrix, CFA pattern, and such as necessary for your imager. Although the 0x94 indicates you probably have an RGGB CFA like the A7M3.

My concern with the stuff in dcraw.cc is that the location you found for adding your file support is a nasty hack/workaround that attempts to detect something based solely on file size, and the more entries you add to it, the more likely it will break. Especially for monochrome variants of the same sensors… Whether to include this there is not my decision, but it’s my personal opinion that a legacy hack for stuff that predates DNG (and was probably written before DNG existed) really shouldn’t be expanded further for stuff that can more properly be handled via a lossless conversion to DNG.

Edit: For your IMX183 example, the conversion to DNG (before tagging) would probably be:
convert -size 5472x3648 -depth 16 GRAY:in.bin TIFF:out.dng

Assuming your file is 16 bits per sample

3 Likes
Mean/Median Stack of multiple dark frames before demosaicing.
#8

In addition to what @Entropy512 says, you might want to add make and model tags to the exiftool command, eg “-make=SONY -model=IMX174” or “-make=Basler -model=acA4096” or whatever.

Software that reads the DNG files may do nothing currently with that metadata, but they might in future.

1 Like
#9

I understand that my first approach is a bad hack.

The conversion step and external tools are fine for establishing a pipeline. Thanks for the details instructions on this!
But this is not intuitive and really hard for windows-GUI people … I guess that many might fail with this. I’ve heard “it says ‘raw-converter’ - why wouldn’t this tool handle my raw files?” or “this is the most simple form a .raw file can be” and “even with ImageJ and can open the files - why can’t you?”

What about adding a dialog for (unrecognized) .raw files?
This could ask you about the resolution, CFA layout and so on - (your fault if it is wrong). I understand that most tools can open no-metadata .raw files this way.
(This might be really annoying in the file browser if many dozen windows ask for a resolution)

#10

Hmm… Keep in mind that your particular use case is an extremely rare niche. Cameras that save raw data with no metadata OR headers are extremely rare, and getting rarer in the era of DNG. (I’d need to look, but I’m pretty sure everything in that table you’re editing are fairly outdated models.)

I’ve never seen a mainstream tool that opened no-metadata RAW files like this, except for maybe Adobe DNG converter? (ImageJ is definitely not a mainstream tool… If you’re using that you could probably spit out the DNGs in a cleaner fashion than what I hacked up.)

As to the comment from @snibgo - Good point. For my own use case with siril, I copied as much EXIF as possible from an original Sony ARW file that went into the stack. And my script almost surely could use some further tuning. It’s what my coworkers call a “minimum viable product” - I got it working, coincidentally, only 2-3 evenings ago.

Again - I’m just a minor contributor to RT (although I’m hoping to contribute more going forward) and not one of the decision makers, and the decision on such a matter is not mine, this is my personal opinion. But putting all of those overrides in the GUI could get nasty in my opinion. As Morgan mentioned, a midpoint between the two might be to allow additional definitions/overrides in camconst.json

1 Like