Last call for color-target shots for inclusion in RawTherapee 5


(Mica) #21

I’m glad they worked! It was my first time taking that color target out of the packaging.

I’ve got tungsten bulbs in my amazon cart now.


(Morgan Hardwood) #22

Thank you @Pata_Kreyp
Currently we ship a dual-illuminant DCP for the D800 so we’ll keep it.
Your shot is generally great, all parameters are fine except that it’s clipped. If you shoot a D65 + StdA shot, and make sure there is no clipping, then we can update our current DCP. Please do that, as unfortunately I don’t have the raw photos we made our DCP from.


(Pata Kreyp) #23

Ok. Sorry about the clipping.

The thing is I only loaded the shot in Lightroom (I know, I know) and it didn’t considered it as clipped :frowning:
(when I look at it in RT, it indeed shows two clipped squares)

I’ll try again this noon at -0.3ev
(Not sure about the sun this noon, btw)

P.S.
I’m also ordering some tungsten lightbulbs :slight_smile: but I’m not sure when I get them (and whether they arrive unbroken)


(Pata Kreyp) #24

It seems there is a problem with either all my shots or RawTherapee.

I shot my X-Rite passport this noon with -1ev to +1ev and there is still some supposed clipping when in RawTherapee. But neither LightRoom nor the ColorChecker application find any clipping.

Even at +1ev, RT tells that the “blue” square just above the black (in lower right corner) is clipping in black:

  • in RT, it is R:0% G: 79% B: 93%
  • in LT, it is R: 83% G: 92% B: 97%

Did I do something wrong?

RT version 4.2.1513 (I upgraded to this one and even cleared all caches)
LT version 5.7


(Morgan Hardwood) #25

@Pata_Kreyp nobody can help if you don’t upload your shots.


(Pata Kreyp) #26

The problem is already visible the NEF I posted previously.

Look at the “blue-green” square above the black square in lower right corner. It should not be clipping, according to LR and the Colorchecker application (which creates a DCP from it, BTW).

Here are two “extremes”: 8009 is -0,67ev and 8047 is +1ev (with a lot of white clipping, but still some black clipping on the “blue-green” square)

Hope, there is no quota, each NEF is about 40MB :smiley:
_DSC8047.NEF (40.8 MB)
_DSC8009.NEF (44.4 MB)


(Ilias Giarimis) #27

@Pata_Kreyp … this “blue-green” patch is normally clipped (red=0) when we use very small colorspaces (sRGB etc)
http://www.babelcolor.com/index_htm_files/ColorChecker_RGB_and_spectra.xls

See the RGB tab … at all small spaces this patch is clipped …

No worries for the job intented, because we use the “reference image for profiing” where no color conversion happens …


(Ilias Giarimis) #28

It would be better if you can upload a better exposure (-0.33 or 0) …

BTW can you supply “White frames” at all ISOs for your D800 ?.

edit:
By “White Frames” I mean very overexposed frames where all pixels are white clipped. I.e for a bright homogeneous frame, use auto exposure with +5EV (a display set to display pure white, and totally out of focus at medium aperture say f/5.6).
All ISOs I mean integer and intermediate … 50-100-125-160-200- … 51200 …

Ideally we need two sets one set with normal exposures and one set with long exposures and LENR set to ON … if two sets is too much upload only the LENR exposures

Use a cloud for uploading … and preferably precompress with 7z max compression to save bandwidth/time …


(Jonas Wagner) #29

That would be a daylight color checker shot with the D810. It was shot on my balcony which in retrospective is fairly dumb because there is likely some indirect light leaking in. As there is no sun around at the moment I can’t take a fresh one. Also I don’t have a tungsten bulb at the moment. But maybe I can get one.

Anyway here it is:
DSC_0834.NEF (39.6 MB)


(Ingo Weyrich) #30

Some report from hell here :slight_smile:


(Jonas Wagner) #31

Hah, that’s one hilarious typo. :slight_smile:


(Mica) #32

Straight from the Rawpedia page: Your balcony is not a good spot - the walls will reflect light and negatively influence the light spectrum, even if you can’t see that with your bare eyes.

:wink:


(Pata Kreyp) #33

First, thanks for the info about clipping, I had totally forgotten about it.

Ok, no problem, I shot so many :wink:

Here is one at -0,3ev attached._DSC8020.NEF (43.3 MB)

Wow, I’ll need some time to do that :slight_smile:


(Morgan Hardwood) #34

@Pata_Kreyp for the white shots see this http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats

@ilias_giarimis you can use that link to avoid typing the same instructions every time. Let me know when you sign up for a RawPedia account so that I can give you edit permissions.

@Jonas_Wagner the shot works quite well and the DE values look quite good, I think we can use it. Would be great if you could get us a tungsten incandescent light bulb shot too.

https://filebin.net/pn9yiuzrqjns61kc/NIKON_D810.dcp

Before:

After:

I measured the DCP curve to look like this:

NIKON D810 ACR.rtc (123 Bytes)

@ilias_giarimis
DCP log:

dcamprof-1.0.5 $ ~/scripts/generateDCP -c "NIKON D810"



argyll-scanin daylight ----------------------------------------

Input file 'daylight.tif': w=2594, h=1729, d = 3, bpp = 16
Data input file 'data-examples/cc24_ref.cie'
Data output file 'daylight.ti3'
Chart reference file '/usr/share/argyllcms/ref/ColorChecker.cht'
Creating diagnostic tiff file 'daylight-diag.tif'
About to allocate scanrd_ object
Verbosity = 2, flags = 0x62af9
About to read input tiff file and discover groups
adivval = 1.000000
About to calculate edge lines
2613 useful edges out of 10564
About to calculate perspective correction
Perspective correction factors = -0.000003 -0.000000 1296.999808 864.500000
About to calculate rotation
Mean angle = -0.854536
Standard deviation = 5.244419
Robust mean angle = -0.254715 from 1950 lines
About to calculate feature information
About to read reference feature information
Read of chart reference file succeeded
About to match features
Checking xx
Checking yy
Checking xy
Checking yx
Checking xix
Checking yiy
Checking xiy
Checking yix
Axis matches for each possible orientation:
  0: xx  = 0.249098, yy  = 0.095301, xx.sc  = 0.122626, yy.sc  = 0.118690
 90: xiy = 0.069139, yx  = 0.235514, xiy.sc = 0.591950, yx.sc  = 0.108690
180: xix = 0.251989, yiy = 0.072048, xix.sc = 0.122643, yiy.sc = 0.243379
270: xy  = 0.069857, yix = 0.253447, xy.sc  = 0.591950, yix.sc = 0.108690
r0 = 0.258145, r90 = 0.045068, r180 = 0.132070, r270 = 0.048272
There are 1 candidate rotations:
cc = 0.258145, irot = -0.254715, xoff = -95.693493, yoff = -86.086977, xscale = 8.154878, yscale = 8.425325
About to compute final match transform
Improve match
About to setup value scanrdg boxes
About to read raster values
About to write diag file
Writing output values to file 'daylight.ti3'



make-profile daylight ----------------------------------------

Reading target...
Camera G on darkest patch(es) is 24.1% lighter compared to observer Y.
  Y dynamic range is 4.83 stops, G dynamic range is 4.52 stops, difference
  0.31 stops. A small difference is normal, while a large indicates that there
  is glare.
Generating values for the calibration illuminant D50...
Re-generating target reference XYZ values for illuminant D50...
Warning: auto-selected neutral patch (D02) is not the lightest, as the
  lightest patch is considerably off-white. That is if you later use the target
  for white balancing you should use the indicated patch instead of the
  lightest.
The most neutral patch (D02) differs 1.22 DE from actual neutral,
  transforming target reference XYZ values to match, using CAT02.
Automatic LUT relaxation weights assigned.
Making camera profile...
Finding a camera raw RGB to CIE XYZ matrix for calibration illuminant D50...
Whitest patch in target (D02) differs DE 1.22 from calibration illuminant,
  close enough to calculate whitepoint preservation.
Inverting to get ColorMatrix:
  {
    "ColorMatrix1": [
      [  1.000992, -0.308279, -0.090839 ],
      [ -0.466232,  1.260381,  0.227878 ],
      [ -0.089433,  0.177142,  0.686037 ]
    ]
  }
Matrix patch match average DE 1.68, DE LCh 0.72 0.72 1.10
                    median DE 1.63, DE LCh 0.66 0.59 0.88
                       p90 DE 2.39, DE LCh 1.53 1.53 2.29
                       max DE 4.88, DE LCh 1.92 1.97 4.70
ColorMatrix optimal white balance for target: 0.582588,1,0.657766 (m1.71648,1,1.5203)
Finding a camera raw RGB to CIE XYZ matrix for connection space illuminant D50...
  {
    "LUTMatrix1": [
      [  0.653696,  0.127580,  0.182942 ],
      [  0.279022,  0.665979,  0.054999 ],
      [  0.000187,  0.011538,  0.813476 ]
    ]
  }
LUTMatrix optimal white balance for target: 0.57772,1,0.654336 (m1.73094,1,1.52827)
Matrix patch match average DE 4.47, DE LCh 1.86 3.00 2.74
                    median DE 4.94, DE LCh 1.79 3.02 2.66
                       p90 DE 7.51, DE LCh 3.45 6.30 5.76
                       max DE 8.09, DE LCh 4.84 7.47 6.80
Finding a camera raw RGB to CIE XYZ matrix for connection space illuminant D50...
  Y row limit set to -0.2.
Applying white-balance to get ForwardMatrix:
  {
    "ForwardMatrix1": [
      [  0.658089,  0.225734,  0.080396 ],
      [  0.239747,  0.883922, -0.123669 ],
      [  0.026205, -0.223022,  1.022018 ]
    ]
  }
ForwardMatrix optimal white balance for target: 0.57772,1,0.654336 (m1.73094,1,1.52827)
Matrix patch match average DE 1.76, DE LCh 0.82 0.77 1.21
                    median DE 1.72, DE LCh 0.93 0.71 1.02
                       p90 DE 2.81, DE LCh 1.55 1.86 2.34
                       max DE 4.99, DE LCh 1.86 2.16 4.83
Making 2.5D chromaticity-addressed lookup table for XYZ correction...
4.17% of the patches was put in a chromaticity group due to nearby neighbor.
  25.00% of the patches was removed due to being nearby the whitepoint.
  Largest chromaticity group contains 2 patches. Patch count reduced from
  24 to 17. Note that patch matching cannot reach 100% when chromaticity
  groups are formed, as the LUT matches the average within a group.
Lightness axis is disabled. Since lightness affects chroma, the LUT chroma
  control points are recalculated to better match the uncorrected lightness.
  A residual error of up to about 0.2 DE is expected.
Relaxing LUT stretch with up to 3.93 DE. Iterating over 17 patches...
  Lightness correction is disabled.
Average DE for the 17 tested patches increased to 1.79 after LUT relax.
  35.29% could do without LUT correction.
Native LUT patch match average DE 1.65, DE LCh 0.82 0.81 0.88
                        median DE 1.61, DE LCh 0.93 0.73 0.84
                           p90 DE 2.18, DE LCh 1.55 1.99 1.50
                           max DE 3.22, DE LCh 1.86 3.15 1.69
5 worst patches for Overall DE:
  B06 RGB 0.362 0.373 0.091 XYZref 0.537 0.477 0.066 XYZcam 0.513 0.471 0.096 sRGB #F7A62B #EDA948 DE 3.22 DE LCh -0.29 -3.15 +0.59 (light strong orange)
  C04 RGB 0.439 0.556 0.112 XYZref 0.663 0.666 0.082 XYZcam 0.647 0.661 0.112 sRGB #FECE20 #FCCF43 DE 2.77 DE LCh -0.14 -2.43 +1.32 (light vivid yellow)
  C03 RGB 0.206 0.086 0.040 XYZref 0.242 0.140 0.042 XYZcam 0.259 0.154 0.053 sRGB #C0333A #C43A43 DE 2.18 DE LCh +1.86 -0.53 -1.02 (strong red)
  B01 RGB 0.340 0.242 0.063 XYZref 0.445 0.342 0.053 XYZcam 0.452 0.349 0.068 sRGB #EE802F #EE833E DE 2.18 DE LCh +0.48 -1.28 -1.69 (strong orange)
  B05 RGB 0.233 0.459 0.124 XYZref 0.390 0.485 0.099 XYZcam 0.388 0.477 0.120 sRGB #AFC245 #AEC055 DE 2.16 DE LCh -0.35 -1.99 +0.78 (light strong yellow-green)
5 worst patches for Lightness DE:
  C03 RGB 0.206 0.086 0.040 XYZref 0.242 0.140 0.042 XYZcam 0.259 0.154 0.053 sRGB #C0333A #C43A43 DE 2.18 DE LCh +1.86 -0.53 -1.02 (strong red)
  B04 RGB 0.064 0.078 0.081 XYZref 0.092 0.071 0.114 XYZcam 0.102 0.080 0.119 sRGB #5D3F6B #62456D DE 2.01 DE LCh +1.66 -0.93 +0.61 (dark purple)
  A03 RGB 0.104 0.243 0.219 XYZref 0.187 0.202 0.287 XYZcam 0.200 0.217 0.293 sRGB #5780A3 #5E84A4 DE 1.72 DE LCh +1.55 -0.80 -0.28 (purple-blue)
  D06 RGB 0.024 0.041 0.027 XYZref 0.034 0.035 0.030 XYZcam 0.040 0.041 0.035 sRGB #343435 #383939 DE 1.61 DE LCh +1.54 -0.21 -0.45 (gray 20%)
  A04 RGB 0.078 0.155 0.063 XYZref 0.121 0.146 0.059 XYZcam 0.131 0.157 0.067 sRGB #626F44 #667349 DE 1.53 DE LCh +1.44 -0.53 +0.12 (yellow-green)
5 worst patches for Chroma DE:
  B06 RGB 0.362 0.373 0.091 XYZref 0.537 0.477 0.066 XYZcam 0.513 0.471 0.096 sRGB #F7A62B #EDA948 DE 3.22 DE LCh -0.29 -3.15 +0.59 (light strong orange)
  C04 RGB 0.439 0.556 0.112 XYZref 0.663 0.666 0.082 XYZcam 0.647 0.661 0.112 sRGB #FECE20 #FCCF43 DE 2.77 DE LCh -0.14 -2.43 +1.32 (light vivid yellow)
  B05 RGB 0.233 0.459 0.124 XYZref 0.390 0.485 0.099 XYZcam 0.388 0.477 0.120 sRGB #AFC245 #AEC055 DE 2.16 DE LCh -0.35 -1.99 +0.78 (light strong yellow-green)
  C02 RGB 0.088 0.272 0.099 XYZref 0.165 0.252 0.086 XYZcam 0.174 0.258 0.099 sRGB #51994E #569A56 DE 1.53 DE LCh +0.51 -1.31 +0.60 (yellow-green)
  B01 RGB 0.340 0.242 0.063 XYZref 0.445 0.342 0.053 XYZcam 0.452 0.349 0.068 sRGB #EE802F #EE833E DE 2.18 DE LCh +0.48 -1.28 -1.69 (strong orange)
5 worst patches for Hue DE:
  B01 RGB 0.340 0.242 0.063 XYZref 0.445 0.342 0.053 XYZcam 0.452 0.349 0.068 sRGB #EE802F #EE833E DE 2.18 DE LCh +0.48 -1.28 -1.69 (strong orange)
  C05 RGB 0.258 0.177 0.173 XYZref 0.341 0.221 0.255 XYZcam 0.356 0.231 0.247 sRGB #C8599B #CE5B98 DE 1.88 DE LCh +1.05 -0.10 +1.55 (purple-red)
  A02 RGB 0.321 0.353 0.202 XYZref 0.434 0.387 0.214 XYZcam 0.459 0.407 0.234 sRGB #D29987 #D89C8D DE 1.88 DE LCh +1.14 -0.05 -1.50 (red)
  A01 RGB 0.091 0.096 0.048 XYZref 0.130 0.113 0.057 XYZcam 0.129 0.113 0.053 sRGB #7C5547 #7C5544 DE 1.47 DE LCh +0.00 +0.24 +1.45 (dark brown)
  C01 RGB 0.032 0.094 0.149 XYZref 0.077 0.064 0.235 XYZcam 0.079 0.069 0.229 sRGB #00499A #004A98 DE 1.31 DE LCh +0.93 -1.10 -1.45 (dark purple-blue)
5 best patches for Overall DE:
  D02 RGB 0.372 0.645 0.422 XYZref 0.622 0.645 0.532 XYZcam 0.622 0.645 0.532 sRGB #D2D2D2 #D2D2D2 DE 0.00 DE LCh +0.00 +0.00 +0.00 (gray 80%)
  B03 RGB 0.264 0.144 0.085 XYZref 0.331 0.217 0.112 XYZcam 0.343 0.220 0.113 sRGB #D25865 #D75565 DE 0.95 DE LCh +0.31 +0.85 -0.28 (red)
  D04 RGB 0.122 0.212 0.137 XYZref 0.202 0.209 0.174 XYZcam 0.204 0.212 0.172 sRGB #7D7E7E #7F7F7D DE 0.96 DE LCh +0.28 +0.17 -0.90 (gray 50%)
  D05 RGB 0.060 0.105 0.069 XYZref 0.094 0.098 0.083 XYZcam 0.100 0.104 0.086 sRGB #575859 #5A5B5A DE 1.23 DE LCh +1.01 -0.10 -0.68 (gray 40%)
  C01 RGB 0.032 0.094 0.149 XYZref 0.077 0.064 0.235 XYZcam 0.079 0.069 0.229 sRGB #00499A #004A98 DE 1.31 DE LCh +0.93 -1.10 -1.45 (dark purple-blue)
Writing output to "daylight.json"...
Complete!



make-dcp ----------------------------------------

Camera: NIKON D810
Copyright: RawTherapee CC0
Curve: acr
TRO: neutral
Gamut: none

Enter DCP filename suffix, if any: 

Generating 2.5D HueSatMap with 90x30 = 2700 entries...done!
The tone curve's contrast value is 1.30 (=> auto chroma scaling value 1.121)
Generating 3D LookTable with 90x30x30 = 81000 entries for the neutral tone reproduction operator...
  0%..10%..20%..30%..40%..50%..60%..70%..80%..90%..100%
Writing output to "NIKON D810.dcp"...
Complete!

(Ilias Giarimis) #35

Looks good :slight_smile: … but … if you have time available, can you to change the WL to R=G=B = 15700 in camconst.json and rebuild the dcp for D810 ?.

For D800 because we don’t have WL data in camconst.json we rely on Dcraw single WL (16383) which is wrong … I think it’s better to build the dcp taking reference data not straight from NEF but from it’s conversion to DNG … (waiting for D800 samples to build the camconst.json item …)


(Assaf Toledo) #36

@Morgan_Hardwood, are these OK?


(Morgan Hardwood) #37

@assaft yes, I already committed a DCP: https://github.com/Beep6581/RawTherapee/commit/12e673e6b081d7070af93db78aa3b6dcc1cf40cf


(Philip B) #38

Here is are the shoots from my D600. I live fairly far north so the sun still has some angle to it. As for the tungsten I used a 500w old school photo bulb, But didn’t have a white space to shoot. Hope it’s still of use. If you feel its not a good example I can always drag some lights outside at night and get another shot. ND0_8796.NEF (25.0 MB)
ND0_8790.NEF (25.9 MB)


#39

I had given Beep6581 a daylight shot from the Fuji X-Pro 2. Just now I’ve uploaded a light bulb shot. See this thread: https://github.com/Beep6581/RawTherapee/issues/3269


(Pata Kreyp) #40

Huh, in this article, instructions are “making sure not to exceed an exposure time of 0.5s”, but ilias_giarimis asked me to make shots with LENR on, which only works for exposure time over 1s (according to D800 menu help and confirmed by tests).

So I took the series of shots with LENR with 2s exposure time instead of 0.5s in series without LENR.

All the shots (with or without LENR) were shot at +3ev.

Link to the shots archive: https://filebin.net/iq79twxvmtyr4f53

Hope this is ok…

And I still have not found any tungsten lamp yet. The batch I ordered should be here on Monday or Tuesday, but I may not be able to get it before Saturday at Post Office.