librtprocess - quo vadis

Carmelo_DrRaw · January 28, 2019, 2:02pm

Dear all, I have just prepared a pull request that contains a small fix in the Amaze demosaicing code to make it compatible to the way PhotoFlow processes the image buffers.

By the way, let me take advantage of this to briefly explain the PhotoFlow requirements regarding the “region of interest” (ROI) processing in librtprocess.

PhotoFlow processes the image in chunks, i.e. small regions (tiles) of more or less arbitrary geometry (they could be square tiles or strips or scan-lines). When a function processes a tile, it gets pointer to the input and output ROIs. In the general case, the input region is larger than the output one, because the algorithm needs some border pixels.

Let’s use the Amaze demosaicing as an example. Amaze requires a 16 pixels border around each output ROI. This means that the “rawData” buffer corresponds to a region defined as

{left=(winx-16), top=(winy-16), width=(winw+32), height=(winh+32)}

except when the region is close to the edges of the image.

Another crucial point is how the algorithm addresses the pixels. The amaze function loops over the pixels, starting from (winx-16, winy-16). This means that the pixels are addressed relative to the origin of the image, not the origin of the ROI. This requires a bit of pointer arithmetics when the input and output buffers do not hold the entire image. For example, the first valid pixel in the input RAW data is accessed with


rawData[winy-16][winx-16]

and this must correspond to the first element of the input buffer. On the other hand, the pixel accessed with


rawData[winy-16][winx-16-1]

corresponds to one element before the beginning of the input buffer, and therefore is not valid.

The client code should take care of properly initialising the input and output float** pointers, but I will gladly provide an helper function if it can be useful.

darix · January 28, 2019, 3:54pm

I would bet that every windows dev would jump onto shared dependencies if windows had some usable package management.

Windows already prefers libraries from the local directory. so does it really make the difference if you link it statically or copy it into the output dir in the end?

ggbutcher · January 28, 2019, 4:19pm

Not really. Just a predilection I need to get over…

heckflosse · January 28, 2019, 7:34pm

I created bayerfast branch which adds the fast bayer demosaicer from RT to librtprocess. This demosaicer is not intended for final output, only for fast preview.

afre · January 28, 2019, 7:59pm

I tend to forget the details behind dynamic v static libraries, so I do a web search every time to remind myself; e.g., c++ - Difference between static and shared libraries? - Stack Overflow, c++ - Static linking vs dynamic linking - Stack Overflow and their comments. I like this answer; analogy alert!

A static library is like a bookstore, and a shared library is like… a library. With the former, you get your own copy of the book/function to take home; with the latter you and everyone else go to the library to use the same book/function. So anyone who wants to use the (shared) library needs to know where it is, because you have to “go get” the book/function. With a static library, the book/function is yours to own, and you keep it within your home/program, and once you have it you don’t care where or when you got it.

Personally, as an end user, I think I prefer the static library. If there are dlls involved, I opt for a portable installation of the app so each app has its own set of dlls.

ggbutcher · January 29, 2019, 3:44am

I wrote a couple to demonstrate in rawproc:

float ** RT_malloc(unsigned w, unsigned h)
{
	float **rawdata = (float **)malloc(h * sizeof(float *));
	rawdata[0] = (float *)malloc(w*h * sizeof(float));
	for (unsigned i=1; i<h; i++) 
		rawdata[i] = rawdata[i - 1] + w; 
	return rawdata;
}

void RT_free(float ** rawdata)
{
	free (rawdata[0]);
	free( rawdata );	
}

(well, actually I took @heckflosse’s code and wrapped it in functions) and you can see them in use here:

github.com

butcherg/rawproc/blob/master/src/gimage.cpp#L2381


      
          			arraydim = 2;
          			for (unsigned i=0; i<arraydim; i++)
          				for (unsigned j=0; j<arraydim; j++)
          					xtarray[i][j] = cfarray[i][j];
          		}
          		else if (xtranArray(xtarray)) {
          			arraydim = 6; 
          		}
          		else return;
          
          		#pragma omp parallel for 
          		for (unsigned y=0; y<h-(arraydim-1); y+=arraydim) {
          			for (unsigned x=0; x<w-(arraydim-1); x+=arraydim) {
          				for (unsigned r=0; r<arraydim; r++) {
          					for (unsigned c=0; c<arraydim; c++) {
          						int pos = (x+c) + (y+r) * w;
          						float val;
          						switch (xtarray[r][c]) {
          							case 0: //r
          								val = image[pos].r - subtractr; 
          								break;

BTW, librtprocess with use of most of the demosaics is now in the rawproc master branch. 'course, it just occurred to me that’s with the need for manual copying of jaggedarray.h to some include/ directory, so I’ll probably push a commit that replaces them with the RT_malloc/RT_free approach…

heckflosse · January 29, 2019, 10:38pm

Is there an interest in getting ppg demosaic into librtprocess? Afaik it’s the main bayer demosaicer in DT and there is also code (though currently unused) available in RT code base which could be used to make a faster version (well, at least faster than the current code in RT, don’t know about speed of DT ppg-demosaic code).

ggbutcher · February 2, 2019, 4:29pm

I added a pull request for the README.md file, added some usage instructions.

ggbutcher · February 8, 2019, 5:26am

Added a pull request for a white balance operator.

ggbutcher · February 8, 2019, 5:35pm

To expose my thought process, I’m thinking of a few additions to librtprocess to enable a command-line raw processor. Since I’m still an imaging neophyte, I look to previous work to tell me what’ s needed; in that regard, I’ve spent some time poring over @Elle’s dcraw.c annotated page:

https://ninedegreesbelow.com/files/dcraw-c-code-annotated-code.html

This tells me the following are probably essential operations pre-demosaic:

Bad pixel removal.
Dark frame subtraction.
CA correction.
WB application.

Demosaic then forms a “rubicon” to be crossed, in that it converts the image format from the mosaic to RGB. So, prior to that, one would want operations that benefit from the “really raw” data, and the float** data structure that benefits its performance. Also, keeping the operations discrete lets one consider what they’re individually about in how they affect the image.

I would build such a command line tool as a true “toolbox”, where each operation’s application is explicitly defined in the command line, no defaults. Even dcraw’s defaults just drive me nuts, indeed, it takes logical gymnastics to turn some things off in it. To my experience, dcraw has been the place where I learned what raw processing really is about, and I’d like to continue that thinking in a tool that leverages more libraries as well as provides explicit semantics.

Food for thought/discussion…

afre · February 8, 2019, 6:59pm

It is still early days but I would like to see full documentation on the various demosaicing algorithms, etc., and the other components of librtprocess.

heckflosse · February 8, 2019, 7:19pm

I can write some signature documentation (about the parameters to pass to the functions) but not about how they work. And for the signatures it’s too early as they most likely will change.

ggbutcher · February 9, 2019, 6:33pm

I have a starting implementation of the librtprocess demosaic routines in rawproc and img. Missing yet are the x-trans demosaics and CA_correct, no challenge, just haven’t gotten to them yet.

Right now, these changes are in the github master branch, not yet in a Wininstaller or AppImage. If one just wanted to compile the img command line tool and not mess with wxWidgets, the instructions to do so are in the README at the github repository.

I messed a bit with doing a librtprocess pipeline in rawdata (in another github repository of mine), but that’ll require more thought with respect to command architecture and the associated data transforms for TIFF output…

ggbutcher · February 10, 2019, 12:40am

…and now I have xtransfast_demosaic() and markesteijn_demosaic() in both rawproc and img. @Claes, they seem to work well…

Still, hard-coded params for unique things. It’ll take a bit to figure out the UI…

ggbutcher · February 21, 2019, 2:33pm

For exercising the librtprocess demosaic algorithms, I have posted a rawproc AppImage, rawproc.conf, and readme.txt at this location:

https://glenn.pulpitrock.net/rawproc/

The AppImage is a full-featured rawproc, compiled as a development snapshot, so keep that in mind when using. The rawproc.conf is specifically configured for just doing demosaics, but can be amended to taste.

The readme.txt is what I think one would need to set things up, but you can never fully comprehend what might be needed, so let me know if you have questions.

Please note this particular deployment is targeted at developers; others are more than welcome to try it out, but rawproc is like a manual transmission Ford Pinto automobile with a missing back bumper; it won’t protect you from bad processing decisions, nor help you a lot with the specifics of using the tools.

Hope this helps…

Edit: ~~Almost forgot: rawproc has the librtprocess demosaics, but the img command line program doesn’t yet. If someone wants that sooner, let me know.~~

Edit2: Gee, I just opened img.cpp to do the librtprocess logic, and it’s already there, forgot I’d already done it. Now, it doesn’t include the parameters for the algorithms that need it, so I’ll work on that this weekend…

CarVac · April 19, 2019, 8:46pm

I’ve added highlight recovery (the inpaint method from RawTherapee) to librtprocess, and it’s doing great things in Filmulator.

Could someone else test it with their applications?

It’s rather simple: you need to pre-apply the white balance multipliers in raw color space, and tell it the max actual value of each channel (chmax) and white clipping value of each channel (clmax).

It makes the difference between this (camera jpeg):

and this (filmulated, with additional white balance adjustments):

heckflosse · April 19, 2019, 10:23pm

@CarVac Great to see some progress in librtprocess . I didn’t find the time to contribute last weeks …

afre · September 1, 2019, 4:41pm

I appreciate auto-CA in RT. Good candidate for librtprocess?

heckflosse · September 1, 2019, 4:42pm

It’s already in librtprocess

afre · September 1, 2019, 4:54pm

Great! Haven’t checked since April / May. Glad it has been included.