Variable white clipping points in raw file

Software
3

Looks like the sensor saturation tops out well below the 14-bit maximum reported by libraw, ~13542 in the red channel. Here are the stats for this image:

channels:
rmin:     0.030563	rmax:     0.206375
g1min:    0.030731	g1max:    0.206161
g2min:    0.030762	g2max:    0.204697
bmin:     0.030563	bmax:     0.203796

channel means:
rmean:     0.045369
g1mean:    0.043255
g2mean:    0.034956
bmean:     0.043259

These are internal rawproc 0.0 -1.0 numbers; multiply by 65536 to get the integer number read from the raw file.

On the black side, Libraw reports a black value of 2047, or 0.031234741 in rawproc terms, and the channel mins are a bit below that.

In rawproc, I recently included a mode to set the white point to the actual data maximum from all channels, and an option to set it at the minimum of the channel maximums to obliterate the whitebalance shifting, or as I’ll call it “highlight de-construction”… :smile:

4

Have a look at the camconst.json entry in RT for this camera (especially the white part):

    { // Quality B, White Levels not properly indicated, aperture scaling..missing scaling factors are guessed
        "make_model": "Canon EOS 80D",
        "dcraw_matrix": [ 7457,-671,-937,-4849,12495,2643,-1213,2354,5492 ], // DNG_V9.5 D65
        "raw_crop": [ 264, 34, 6024, 4022 ], // full size 6288x4056, official crop 276,46,6275,4045
        "masked_areas": [ 40, 96, 4000, 260 ],
        "ranges": {
            "white": [
                { "iso": [ 100 ], "levels": 16100 }, // nominal 16383, LENR blue 16243
                { "iso": [ 125, 200, 250, 400 ], "levels": 16200 }, // nominal 16383, LENR blue 16243
                { "iso": [ 160 ], "levels": 13000 }, // nominal 13097,
                { "iso": [ 320, 640, 1250 ], "levels": 13250 }, // G1,G2 13415
                { "iso": [ 500, 800, 1000 ], "levels": 16150 }, // nominal 16383, LENR ISO3200 16150
                { "iso": [ 1600, 2000, 3200 ], "levels": 16100 }, // nominal 16383, LENR ISO3200 16150
                { "iso": [ 2500, 5000 ], "levels": 13150 }, // G1,G2 13415
                { "iso": [ 4000, 6400 ], "levels": 16050 }, // nominal 16383, LENR ISO6400 16050
                { "iso": [ 8000, 10000 ], "levels": 16000 }, // R,G1,G2 16383, B 16243, LENR B 16000
                { "iso": [ 12800, 16000, 25600 ], "levels": 15900 } // R,G1,G2 16383, B 16243, LENR BLUE 15750
            ],
            "white_max": 16383,
            "aperture_scaling": [
                // need for more data to properly fill all scale factors
                { "aperture": 1.4, "scale_factor": 1.200 }, // guessed
                { "aperture": 1.6, "scale_factor": 1.130 }, // guessed
                { "aperture": 1.8, "scale_factor": 1.090 }, // 14338/13097 = 1.094
                { "aperture": 2.0, "scale_factor": 1.060 }, // 13934/13097 = 1.063
                { "aperture": 2.2, "scale_factor": 1.025 }, // 13438/13097 = 1.026
                { "aperture": 2.5, "scale_factor": 1.005 }, // 13190/13097 = 1.007
                { "aperture": 2.8, "scale_factor": 1.000 }  // 13097/13097
            ]
        }
    },
3 Likes
5

Maybe I need to have my own camconst.json…

6

Why your own?

7

A copy for Filmulator so that it can know these sorts of things.

8

Maybe we can share it via librtprocess in future? Just a thought…

9

Didn’t @darix start putting together a database for things like this?

10

I’d rather have a library separate from librtprocess to hold camera-specific data.

librtdata for data, librtprocess for algorithms?

2 Likes
11

Sounds good to me :+1:

12

For me, the key point is to avoid collecting this kind of data in different places. We should try to find one central place for this data which is accepted by all applications which use the data, even if getting the acceptence can be hard…

4 Likes
13
14

So in there we need the data itself and probably some basic methods for accessing it on a per-camera and per-image basis.

15

In my opinion, to get acceptence for RT and dt we need to make a superset of the features supported by RT and dt. For example RT supports white-level scaling based on aperture, while dt does not support that. On the other side, iirc dt supports some denoise parameters based on camera and ISO which is not supported by RT. A Database has to support all this values even if an application does not use all of them.

5 Likes
16

Yay!! Get Lensfun profiles in there and it’ll be dope!!

(Am I saying that right?..)

17

Actually yes. :wink:

1 Like
18

Maybe also RT camera/lens/aperture dependent capture sharpening settings…

19

We’re going more and more off topic. Should we create a new issue or continue the discussion in this issue? I would prefer the latter.

20

Discussion here is fine. The general camera info is more important than just the white point.

2 Likes
21

I prefer knowing the right value. For example RT capture sharpening does not sharpen the pixels close to a clipped highlight, because the data used to apply the deconvolution is not linear if there is a clipped highlight close to a pixel. To do that, knowing the correct white level is important.

2 Likes
22

Oooh, learning… my thought with doing that was to just go to the top of the data in one operation, instead of setting the white point then applying exposure comp to scale the data to white. Does highlight reconstruction have a similar consideration?

I do have another mode to apply the metadata values, but I need to consider alternatively looking them up in camconst.json