Reverse Engineering Nikon Z series lens correction - Part 2?

I’m starting a new thread because the forum won’t let me post three consecutive posts… :stuck_out_tongue_closed_eyes:

Oookay, did some close inspection of my code and found a parentheses error in my calculation of the adobe normalization, fixed that and the dng numbers @paolod supplied correct the image just fine…

With that and some inspection of extracted values, I determined that if one uses the adobe model and feeds it the metadata coefficients in reverse order, the correction is quite close to lensfun ptlens and dng. Still need to figure how the k0 scaling parameter is calculated, and there is a bit of a curl in the extreme edge of the image, but I think this is establishes a chain of causality…

Still, wondering how the AdobeDNGConverter determines what to put in the DNG, the numbers are close but not the same…

@paperdigits, close enough? :laughing:

Close enough that I can coax someone to add it to the darktable correction model??? Seems close-ish but not quite… But who am I to judge I have no idea about any of this :rofl:

1 Like

Yeah it is still a bit off compared to the lensfun and dng numbers, but they’re of similar magnitude and in the right direction. The polynomial could be reordered to cover the coefficient order, but since addition is commutative I don’t think that’d make any difference in the result. I did a magnitude comparison of the Nikon coefficients with the equivalent DNG coefficients for the same image, each is different so there’s no scaling factor. I need to extract numbers from more images to see if there’s single-coefficient consistency.

Also needed is code to decode the tags. Right now, only newest exiftool does it.

I think the next step should be to collect a set of test images, from a few different lenses and at a few different focal lengths of a zoom lens, run all of those through Adobe DNG Converter, and make sure the same interpretation of the metadata coefficients works in all cases.

Rather than applying the correction to an actual image and seeing if it looks right, it can be easier to plot the Rsrc=f(Rdst) function for the DNG coefficients and the NEF coefficients and see if they match up. For Olympus I was able to get an exact match once I figured out the slight adjustment that was being made to the radius.

1 Like

Yep, the un-normalized radius is quite important to align the polynomial. I’d screwed up the parentheses coding the adobe normalization, resulting in radii that were twice the correct length, putting the pertinent part of the polynomial curve in the center of the image, rather than extending out to the edges…

I have the 24mm, 50mm, and 100-400mm if you want samples.

For the 100-400, do 100, 135, 200, 300, and 400mm images; that’s what’s in lensfun…

I’ll collect them if you post them here.

Edit: If you shoot a sheet of graph paper or something like that, we can see the effect of corrections at all radii.

1 Like

Do you want the focal distance all the same?

I think just fill the frame with the paper, and focus on it at whatever distance…

1 Like

_DSC1219.NEF (50.9 MB)
_DSC1220.NEF (51.2 MB)
_DSC1221.NEF (50.8 MB)
_DSC1222.NEF (49.7 MB)
_DSC1223.NEF (48.3 MB)

@ggbutcher here you go, sorry for the delay!

1 Like

So, I’ve had a chance to sift through the Nikkor 100-400mm zoom distortion correction. Here’s what exiftool spits out for the distortion coefficients:

glenn@bena:~/Photography/Lens_Correction/Nikon Z Lens Correction/2024-01-19 Nikon Z Lens Collection/NIKKOR_100-400_4.5-5.6$ for F in *.NEF; do exiftool -csv -FocalLength -*distortioncoefficient3 -*distortioncoefficient2 -*distortioncoefficient1 $F; done
SourceFile,FocalLength,RadialDistortionCoefficient3,RadialDistortionCoefficient2,RadialDistortionCoefficient1
_DSC1219.NEF,100.0 mm,0.00430,-0.00120,0.00239
_DSC1220.NEF,135.0 mm,0.01387,-0.00137,0.00286
_DSC1221.NEF,200.0 mm,0.02062,-0.00059,0.00246
_DSC1222.NEF,310.0 mm,0.02291,-0.00036,0.00210
_DSC1223.NEF,400.0 mm,0.02360,-0.00014,0.00195

(edited the program output for clarity)

Of note, these coefficients are pretty small, not a lot of correction going on. Indeed, when I convert the NEFs to DNGs and pull out the Adobe WarpRetilinear coefficients, some all of the focal lengths only have chromatic abberation correction:

glenn@bena:~/Photography/Lens_Correction/Nikon Z Lens Correction/2024-01-19 Nikon Z Lens Collection/NIKKOR_100-400_4.5-5.6$ for F in *.dng; do exiftool -b -opcodelist3 $F | ../../opcode3a ; done
Warning: [Minor] Not decoding some large array(s). Ignore minor errors to decode - _DSC1219.dng
number of opcodes: 1

opcode: 1, dngspec: 1030000, flags: 0, numbytes: 164
operation: WarpReticlinear
number of coefficient sets: 3
kr0:1.000154, kr1:-0.000159,  kr2:0.000023, kr3:-0.000005, kt0:0.000000, kt1:0.000000
kr0:1.000000, kr1:0.000000,  kr2:0.000000, kr3:0.000000, kt0:0.000000, kt1:0.000000
kr0:1.000162, kr1:0.000203,  kr2:-0.000109, kr3:0.000025, kt0:0.000000, kt1:0.000000
cx:0.501197, cy:0.505884
Warning: [Minor] Not decoding some large array(s). Ignore minor errors to decode - _DSC1220.dng
number of opcodes: 1

opcode: 1, dngspec: 1030000, flags: 0, numbytes: 164
operation: WarpReticlinear
number of coefficient sets: 3
kr0:1.000168, kr1:-0.000224,  kr2:0.000118, kr3:-0.000027, kt0:0.000000, kt1:0.000000
kr0:1.000000, kr1:0.000000,  kr2:0.000000, kr3:0.000000, kt0:0.000000, kt1:0.000000
kr0:1.000158, kr1:-0.000031,  kr2:0.000026, kr3:-0.000006, kt0:0.000000, kt1:0.000000
cx:0.501168, cy:0.505902
Warning: [Minor] Not decoding some large array(s). Ignore minor errors to decode - _DSC1221.dng
number of opcodes: 1

opcode: 1, dngspec: 1030000, flags: 0, numbytes: 164
operation: WarpReticlinear
number of coefficient sets: 3
kr0:1.000097, kr1:-0.000151,  kr2:0.000275, kr3:-0.000064, kt0:0.000000, kt1:0.000000
kr0:1.000000, kr1:0.000000,  kr2:0.000000, kr3:0.000000, kt0:0.000000, kt1:0.000000
kr0:1.000040, kr1:-0.000037,  kr2:-0.000147, kr3:0.000034, kt0:0.000000, kt1:0.000000
cx:0.501166, cy:0.505904
Warning: [Minor] Not decoding some large array(s). Ignore minor errors to decode - _DSC1222.dng
number of opcodes: 1

opcode: 1, dngspec: 1030000, flags: 0, numbytes: 164
operation: WarpReticlinear
number of coefficient sets: 3
kr0:1.000121, kr1:0.000056,  kr2:0.000086, kr3:-0.000020, kt0:0.000000, kt1:0.000000
kr0:1.000000, kr1:0.000000,  kr2:0.000000, kr3:0.000000, kt0:0.000000, kt1:0.000000
kr0:0.999774, kr1:-0.000134,  kr2:-0.000130, kr3:0.000030, kt0:0.000000, kt1:0.000000
cx:0.501135, cy:0.505907
Warning: [Minor] Not decoding some large array(s). Ignore minor errors to decode - _DSC1223.dng
number of opcodes: 1

opcode: 1, dngspec: 1030000, flags: 0, numbytes: 164
operation: WarpReticlinear
number of coefficient sets: 3
kr0:1.000066, kr1:-0.000025,  kr2:0.000181, kr3:-0.000042, kt0:0.000000, kt1:0.000000
kr0:1.000000, kr1:0.000000,  kr2:0.000000, kr3:0.000000, kt0:0.000000, kt1:0.000000
kr0:0.999701, kr1:-0.000062,  kr2:-0.000204, kr3:0.000047, kt0:0.000000, kt1:0.000000
cx:0.501138, cy:0.505884

(Note: exiftool doesn’t decode the DNG OpCodeList paramters, so I feed its binary output to a C++ program I wrote to do that) Each .DNG contains a three-plane WarpRetilinear correction, each plane corresponding to a color channel, R, G, and B. Note that none of them have a green channel (middle entry) correction, the parameters for that plane are 1,0,0,0, or no-op. Based on this and some other (lack of) relationships between NEF and DNG coefficients, I think Adobe is putting their own corrections in the DNGs, instead of transferring the NEF data.

Regarding the k0 parameter, Nikon doesn’t provide it in their metadata, at least not in what exiftool extracts. So, I’ve been messing with how to construct the parameter value from image data, a daunting task for a fellow who only took four math courses in his three university degrees :crazy_face:

You can easily do such for Lensfun’s PTLens model:

d = 1 - (a+b+c)

Where a, b, and c are the PTLens correction coefficients and d is their k0 equivalent. No such fortune for Adobe, the powers in the model function don’t (at least to me, bear-of-little-brain) offer a straightforward equation.

Thinking it through, what d and k0 do in each model is scaling of the warped image. Using that assertion, I came to the conclusion that I could determine the scale needed for a particular image and its correction coefficients by computing the normalized (0.0-1.0) correction needed for the maximum distance-from-the-center in the width and height dimensions and use that to determine the scale factor. So, I wrote a C++ function to compute that:

float PicProcessorLensDistortion::adobe_k0(float k1, float k2, float k3)
{
	int w = dib->getWidth();
	int h = dib->getHeight();

	float normR = sqrt((w/2)*(w/2) + (h/2)*(h/2));

	float wR = (w/2) / normR;
	float hR = (h/2) / normR;

	float wD = (k1*pow(wR,2) + k2*pow(wR,4) + k3*pow(wR,6)) * wR;
	float hD = (k1*pow(hR,2) + k2*pow(hR,4) + k3*pow(hR,6)) * hR;
	
	if (wD < hD)
		return 1-(wD/2);
	else
		return 1-(hD/2);
}

With the couple of coefficient sets I’ve tested, it seems to yield a k0 from the Nikon data roughly equivalent to the k0 in the corresponding DNG data.

Anyway, comments and criticisms welcome…

Edit: I must have fat-fingered something earlier, looking at the distortion function I came to the realization that the maximum normalized radius, 1, would just take the powers out of the equation and leave it with 1-(k1+k2+k3). Freaking math genius… So, I tried 1-(k1+k2+k3)/2, and it gives a decent scaling number. Need to inspect the edges closer to make sure it takes out all of the warped edge…

Edit2: Looking at the pasts posts, I need to give a shout-out to @paolod, who really put me on the correct track looking into this.

3 Likes

:heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes:

1 Like

So, started picking at the vignetting parameters. Firstoff, Adobe DNGs don’t have any; none of the DNGs I made of the NEFs for either my 24-70 or @paperdigits’ 100-400 had a vignetting opcode, so no comparison basis.

The other concerning thing are the Nikon coefficients. Here’s the exiftool extracts for each of the focal lengths of my 24-70:

$ exiftool -FileName -Aperture -FocalLength -*Distortion* -*Vignette* DSZ_9749_NIKKOR_Z_24-70mm_f4_S-24.0mm.NEF
File Name                       : DSZ_9749_NIKKOR_Z_24-70mm_f4_S-24.0mm.NEF
Aperture                        : 4.0
Focal Length                    : 24.0 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.02917
Radial Distortion Coefficient 2 : 0.03430
Radial Distortion Coefficient 3 : -0.13184
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : -5.01538
Vignette Coefficient 2          : 2.85181
Vignette Coefficient 3          : 0.21606
Vignette Control                : Normal
glenn@bena:~/Photography/Lens_Correction/Nikon Z Lens Correction/2024-01-19 Nikon Z Lens Collection/NIKKOR_24-70_4$ exiftool -FileName -Aperture -FocalLength -*Distortion* -*Vignette* *.NEF                                 ======== DSZ_9749_NIKKOR_Z_24-70mm_f4_S-24.0mm.NEF
File Name                       : DSZ_9749_NIKKOR_Z_24-70mm_f4_S-24.0mm.NEF
Aperture                        : 4.0
Focal Length                    : 24.0 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.02917
Radial Distortion Coefficient 2 : 0.03430
Radial Distortion Coefficient 3 : -0.13184
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : -5.01538
Vignette Coefficient 2          : 2.85181
Vignette Coefficient 3          : 0.21606
Vignette Control                : Normal
======== DSZ_9750_NIKKOR_Z_24-70mm_f4_S-28.0mm.NEF
File Name                       : DSZ_9750_NIKKOR_Z_24-70mm_f4_S-28.0mm.NEF
Aperture                        : 4.0
Focal Length                    : 28.0 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.03192
Radial Distortion Coefficient 2 : 0.00445
Radial Distortion Coefficient 3 : -0.07227
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : 0.06466
Vignette Coefficient 2          : 1.29070
Vignette Coefficient 3          : 0.41281
Vignette Control                : Normal
======== DSZ_9751_NIKKOR_Z_24-70mm_f4_S-34.5mm.NEF
File Name                       : DSZ_9751_NIKKOR_Z_24-70mm_f4_S-34.5mm.NEF
Aperture                        : 4.0
Focal Length                    : 34.5 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.02835
Radial Distortion Coefficient 2 : -0.01343
Radial Distortion Coefficient 3 : -0.00955
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : -3.84567
Vignette Coefficient 2          : 2.98417
Vignette Coefficient 3          : 0.20918
Vignette Control                : Normal
======== DSZ_9752_NIKKOR_Z_24-70mm_f4_S-50.0mm.NEF
File Name                       : DSZ_9752_NIKKOR_Z_24-70mm_f4_S-50.0mm.NEF
Aperture                        : 4.0
Focal Length                    : 50.0 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.01795
Radial Distortion Coefficient 2 : -0.01077
Radial Distortion Coefficient 3 : 0.03359
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : 0.01526
Vignette Coefficient 2          : -0.08041
Vignette Coefficient 3          : 1.02670
Vignette Control                : Normal
======== DSZ_9753_NIKKOR_Z_24-70mm_f4_S-70.0mm.NEF
File Name                       : DSZ_9753_NIKKOR_Z_24-70mm_f4_S-70.0mm.NEF
Aperture                        : 4.0
Focal Length                    : 70.0 mm
Distortion Correction Version   : 0100
Distortion Correction           : On (Required)
Radial Distortion Coefficient 1 : 0.01344
Radial Distortion Coefficient 2 : -0.00748
Radial Distortion Coefficient 3 : 0.04366
Auto Distortion Control         : On
Vignette Correction Version     : 0100
Vignette Coefficient 1          : -3.37203
Vignette Coefficient 2          : -0.66680
Vignette Coefficient 3          : 1.47779
Vignette Control                : Normal
    5 image files read

The vignetting coefficients above 1.0 are way out of the correction domain, at least as far as I know. I coded up the Adobe vignetting model, put the corresponding vignetting parameters into the 24mm test shot and got this:

Funny, the DSZ_1401.NEF posted earlier was shot with the same lens, same focal length, at f9, and its vignetting numbers worked better than lensfun’s correction.

I’m going to correspond with Warren, seems like an alignment issue…

2 Likes

I though green was the benchmark that the others are “bent” to follow? I really can’t back this up but that’s how I’ve heard optical designers talk about it.

The objective is to shift the red and blue channels a bit different from the green to eliminate the CA bias. If there is no distortion to correct, these are valid CA corrections.

That’s the case for correction of chromatic abberation: different wavelengths provide slightly different images on the sensor, but you have no external reference, so you take one channel (green) as reference. Green is used as it is (more or less) in the center of the spectrum, which helps minimising remaining errors.

The coefficients discussed here are for distortion correction. That affects all wavelength, and you have an external reference (straight lines in the image). So you correct all channels.

Of course, you can combine CA and distortion correction in one set of coefficients; in that case, if there is no distortion, there would be no correction for the green channel.

Yes. I misread @ggbutcher post. I thought he was trying to solve the chromatic aberration corrections after finding out there were no distortion corrections for that lens.

What I don’t get is that for that lens Nikon metadata from the NEF has distortion correction coefficients but the DNG made from that NEF does not. That, and the differences between the two in cases where there is both leads me to believe Adobe is measuring their own corrections in spite of the manufacturer data.

This is just speculation but maybe they make a distinction between optional and mandatory distortion correction. DSLR lenses can’t have too much distortion because you always see the actual image formed by the lens through the OVF. But mirrorless lenses can have strong distortion by design which is never seen by the end user because the camera body always applies it to the EVF and jpegs, and apps like DNG Convertor or Lightroom apply it unconditionally.

When I was looking into Olympus correction I found one tag that was only ever used for Micro Four Thirds mirrorless lenses. The correction the camera body applied matched this tag, and Adobe DNG Converter only ever used this tag. I used this tag for the darktable implementation, assuming that it represents the ‘mandatory’ correction.

But I also found a couple other tags which appeared to be for distortion correction. One used the same formula as the first tag, and was sometimes equal to it but was sometimes different encoding a stronger correction. Or only this tag had some correction and the first tag was null. There was also another tag that seemed to be used only for Four Thirds DSLR lenses and had a different format. I don’t know what these tags were intended for though - as far as I can tell no camera body or application I could find uses them for anything.

1 Like

Also, Adobe DNG Converter does come with a set of lcp format lens profiles, at c:/ProgramData/Adobe/CameraRaw/LensProfiles/1.0/. But there aren’t any at all for Nikon mirrorless lenses in there. There are also none for Olympus or Panasonic as it appears to always use the embedded correction.

1 Like