Sony picture profile affects raw

Hi All,

I stumbled on an interesting video.

Here is the interesting point

In all honesty - I have not tested it yet.
In essence - Gerald demonstrates that if PP (Picture Profiles) are used on a sony camera and 1 specific setting (gamma) is changed - this setting actually is going to affect the RAW file.

From what I am seeing on the video (without testing myself) it maybe unlikely for photographers to use it.
But when it does get used - it is going to affect the brightness.

1 Like

@ggbutcherā€™s rawproc would probably be the ideal tool to prove/disprove this.

2 Likes

For Nikon, I know active-D lighting explicitly changes the exposure. Other picture controls, not so muchā€¦

rawproc would be a good tool to assess this, a raw file can be opened with no processing and the image starts can be inspected. Gonna be out of pocket starting tomorrow, else Iā€™d play with thisā€¦

1 Like

Testing this with Adobe software makes no sense. Thereā€™s no way of knowing fit it ā€œaffects rawā€ or just Adobe settings.

1 Like

Yes, see this all the time, unfortunately. Like comparing the ā€œcolor scienceā€ of two different cameras, using default LR processing :roll_eyes:

Edit: Just watched the video and he did test in other software as well.

1 Like

I wonder if Rawdigger would a useful test program, at least to verify whether the settings affect the histogram.

Do you have it installed?

Maybe when you get back?

20240206_145528_04155.ARW (37.3 MB)
20240206_145620_04156.ARW (36.2 MB)
20240206_145650_04157.ARW (36.3 MB)

I checked GitHub - butcherg/rawproc: Raw camera and general image processor
My understanding is that it has .deb but my system is older (PopOS 22.04) and it is still not in the repository.

In DT they look very different


And the green color is because I tried to turn off everything that I can
image

IMO this area shows quite a bit of the difference
For the 3 - I am using the default stack and only changed exposure to .6 to make sure it calculates the previews from the processed image.

My on camera PP settings are
04155 - none
01556 - PP3 (Gamma ITU709)
01557 - PP5 (Gamma Cine1)

And these profiles also have Black Gamma (it is configurable but does not change by default).

All shots are done in a full manual mode to preserve speed, aperture and ISO, few sec. apart, manual focus and remote trigger

In my opinion the RAW changes significantly.

If somebody wants to test - feel free to grab them.

Not yet on my new computer. The old one died a sad death.

Ooookay, hereā€™s what I can tell with rawprocā€¦

Firstoff, opened each raw in a separate rawproc instance, when prompted to apply the default processing I said ā€˜Noā€™. Thereā€™s an image statistics dialog, I pulled that up and found each image had blown highlights which drove the max value to saturation, so that was no help. The channel means are available, here they are:

rmean:     0.012030
g1mean:    0.018101
g2mean:    0.010897
bmean:     0.018110

rmean:     0.009931
g1mean:    0.013674
g2mean:    0.009365
bmean:     0.013680

rmean:     0.009401
g1mean:    0.012365
g2mean:    0.008970
bmean:     0.012369

There are differences, but they need to be put in context. So, I scaled each image to the same value to stretch the histograms into visibility, and got this:

The left and center histograms are pretty close, the right one is not but thatā€™s due to a few more saturated pixels ending up in the right-hand pile of the histogram.

Frankly, I donā€™t see difference that canā€™t be attributed to some vagary of successive capturesā€¦

1 Like

Thank you for checking!
My understanding here is - differences exist but they are not too big of a deal to be concerned :slight_smile:

1 Like

Based on what @ggbutcher said about Nikon actvie-D lighting changing exposure, could something like that happen here: the exposition is adapted to the selected gamma to protect highlights.

An exposure change changes the amount of light hitting the sensor, and that of course changes the raw fileā€¦
But it looks like thatā€™s the only change (if any) for the three images in the test

2 Likes

Detecting ā€œchangesā€ is really harder than it sounds. Try this sometimes: set up a constantly-lit scene, mount your camera on a tripod, and take ten successive exposures in Manual, with fixed aperture, shutter speed and ISO settings, take the raws to RawDigger (more info than rawproc), collect selected statistics and compute their averages AND standard deviations. Iā€™ll bet the standard deviations are larger than one might expectā€¦

Selecting statistics relevant to what Picture Controls change is important. For tone, one might want to work up some sort of by-bucket diff of histograms, not just eyeballing the graphs. For color, maybe a by-bucket diff of the separate channelsā€¦

Possible; for Nikon, Iā€™d surmise thereā€™d be something about that in their camera manuals, given they felt it important to report such for Active-D. And then, Iā€™d really hope not; if one wants to ā€œSOOC JPEGā€, they need a good baseline exposure anchored on the well-understood middle gray upon which to lay a Picture Control modification, especially for the manufacture to avoid WTF questions from people trying to use the tools that wayā€¦

1 Like

I wrote a short C++ program to collect a raw histogram:

#include <stdio.h>
#include <math.h>
#include "libraw/libraw.h"

struct hist_bucket {
	int id;
	long r;
	long g1;
	long g2;
	long b;
};

int main(int argc, char **argv)
{
	if (argc < 2) {
		printf("Usage: rawhistogram <rawfile> [number_buckets, default=256]");
		exit(0);
	}

	LibRaw iProcessor;
	iProcessor.open_file(argv[1]);
	iProcessor.unpack();
	iProcessor.raw2image();  //use this to normalize the bayer pattern to RGBG (??)
	
	// set the number of possible values in the raw data range using the libraw data_maximum, default = 16384 (2^14):
	int data_range = 16384;  //2^14
	if (iProcessor.imgdata.color.data_maximum < pow(2,13)) data_range = pow(2,13);
	else if (iProcessor.imgdata.color.data_maximum < pow(2,12)) data_range = pow(2,12);
	else if (iProcessor.imgdata.color.data_maximum < pow(2,11)) data_range = pow(2,11);
	//printf("data range: %d\n", data_range);
	
	//set the number of buckets, default = 256:
	int num_buckets = 256;
	if (argc >= 3)
		num_buckets = atoi(argv[2]);
	//printf("number of buckets: %d\n", num_buckets);
	
	hist_bucket histogram[num_buckets];
	
	//initialize channel histograms:
	for (int i=0; i< num_buckets; i++) {
		histogram[i].id = i;
		histogram[i].r = 0;
		histogram[i].g1 = 0;
		histogram[i].g2 = 0;
		histogram[i].b = 0;
	}
	
	//collect the channel histograms:
	printf(",r,g1,b,g2\n");
	for(int i = 0;i < iProcessor.imgdata.sizes.iwidth *  iProcessor.imgdata.sizes.iheight; i++) {
		if (iProcessor.imgdata.image[i][0] > 0 ) 
			histogram[ int (((float) iProcessor.imgdata.image[i][0] / (float) data_range) * (float) num_buckets) ].r++;
		if (iProcessor.imgdata.image[i][1] > 0 ) 
			histogram[ int (((float) iProcessor.imgdata.image[i][1] / (float) data_range) * (float) num_buckets) ].g1++;
		if (iProcessor.imgdata.image[i][2] > 0 ) 
			histogram[ int (((float) iProcessor.imgdata.image[i][2] / (float) data_range) * (float) num_buckets) ].b++;
		if (iProcessor.imgdata.image[i][3] > 0 ) 
			histogram[ int (((float) iProcessor.imgdata.image[i][3] / (float) data_range) * (float) num_buckets) ].g2++;
	}
	
	//print the channel histograms in parallel columns:
	for (int i=0; i< num_buckets; i++) 
		printf("%d,%ld,%ld,%ld,%ld\n", i, histogram[i].r, histogram[i].g1, histogram[i].g2, histogram[i].b);
		
	iProcessor.recycle();
	
}

You just need libraw-dev (Ubuntu), or libraw (msys2) to compile it. It runs from the command line:

$ rawhistogram DSG_0001.NEF

and you get a 256-bucket histogram set of the four channels R, G1, B, G2 in csv format, ready for loading in your favorite spreadsheet. You can change the number of histogram buckets by adding the number to the command line:

$ rawhistogram DSG_0001.NEF 1024

256 seems to be a good compromise between data precision and manageability.

If anyone REALLY wants to assess the differences weā€™ve been discussing, this is the sort of tool you needā€¦

1 Like

Whatā€™s the license for that code?

Cripes, forgotā€¦

GPL 2, I think that works with librawā€¦

One more step to plot itā€¦ Then make a contact sheet with a file name and thumbnail??? :rofl::rofl::rofl:

Wrote a gnuplot script:

20240206_145528_04155
20240206_145620_04156
20240206_145650_04157

Make of it what you will, I think a little more rigorously-captured sets of baseline images followed by pic control-modded images is neededā€¦

1 Like

good to know

Those videos are BS. Sony picture profiles do not affect the RAW in any way, shape, or form. They only affect the displayed ISO because the definition of ISO is derived from the JPEG tone curveā€™s midpoint.

If you do a proper comparison, which is:
Same shutter speed
Same aperture
Lowest possible ISO setting (for example 100 for no PP and 800 for S-Log modes), or same number of stops from lowest possible ISO setting (ISO 200 for no PP, 1600 for S-Log)

You will find NO difference whatsoever between the raw images.

Sony has an EXIF tag that helps here - ā€œSony ISOā€, which can be thought of as ā€œRaw ISOā€, or even better ā€œWhat the ISO would be with no picture profile enabled.ā€

For the images @vbs provided:

(default_venv) adodd@andyslaptop:~/Pictures/sorted/Experimental/PP_ISO$ exiftool -SonyIso *.ARW
======== 20240206_145528_04155.ARW
Sony ISO                        : 200
======== 20240206_145620_04156.ARW
Sony ISO                        : 252
======== 20240206_145650_04157.ARW
Sony ISO                        : 200
    3 image files read

One of these images had the ISO setting at a different distance from base.

2 Likes

These match. Also the ISO matches. At least what DT reports.
Sony ISO - DT does not read it or display it - whatever the effect is on the image.

So - if we can control only the 3 shutter speed, aperture and ISO and these match in the program the expectation is that the pictures will match. They donā€™t (entirely) but they prove to be not that far away from one another.

It is interesting that Sony ISO exists - I did not know about it. But looks like it does get quite technical - way more than what I expected when started pondering on it. Either way - from what I am seeing - not a big gain or loss by using one vs the other.