Sony ARW looks yellow in RT

[Entropy512 (Andy Dodd)]

RT does NOT have a dual-illuminant profile for both A6000 and A6500 - only A6000:

For A6500, there is only the basic DNG color matrix.

Note that in another discussion somewhere, there was evidence that C1 is doing some really weird things with colors in their profiles:

Emulating Capture One colors

Are you sure C1 just use a base curve ? From what I see, they darken blues, desaturate greens, resaturate skin tones and shift them slightly toward red. I suspect there is an artistic LUT going on there, it does too many things at once to be a simple tone curve.

I own an A6300 and A6500, so now that things are calming down, I MIGHT be able to submit to @Morgan_Hardwood the ColorChecker shots needed to generate DCP profiles for those cameras.

MIGHT being that around here, “cloudless noon sunlight” is extremely rare in winter. We’re under a winter weather advisory so I don’t expect to see unobstructed sunlight for a few days. On the other hand, I have two days of work remaining before I hit “use it or lose it” for my vacation days, so I’ll have much more lunchtime availability for camera profiling if the sun ever comes out.

[Morgan_Hardwood]

@Entropy512 the difference between a target shot properly on an overcast winter midday vs shot on a cloudless summer midday is greatly smaller than the difference between a properly shot target under any daylight vs no DCP at all, so go ahead and shoot - we can always submit a better one if summer ever comes around again.

[ggbutcher (Glenn Butcher)]

@Morgan_Hardwood

we can always submit a better one if summer ever comes around again.

I just finished shoveling snow off my driveway for the fourth time this season. I’m not betting on it…

[pgoelz (Paul Goelz)]

@Entropy512

Note that in another discussion somewhere, there was evidence that C1 is doing some really weird things with colors in their profiles:

Emulating Capture One colors darktable

Are you sure C1 just use a base curve ? From what I see, they darken blues, desaturate greens, resaturate skin tones and shift them slightly toward red. I suspect there is an artistic LUT going on there, it does too many things at once to be a simple tone curve.

My oh my, that thread was an interesting read. I guess I now understand better why I have been unable to duplicate my C1 results in RT. I always felt there was something non-linear going on and that it wasn’t simply RGB values.

Paul

[ggbutcher (Glenn Butcher)]

I felt something like that was going on, shoulda wrote this sooner:

Most FOSS software comes with a giant collection of “camera primaries”, sets of nine (or more, for cameras with weird color spaces) numbers that describe the extent of their spectral response. Those nine numbers can be thought of as a “reddest red”, “greenest green”, and “bluest blue” of the camera’s sensor. Actually, a lot of these primary sets come from the collection of profiles Adobe releases with their DNG Converter; a lot of others, for older cameras, come from dcraw.c, a command-line raw processor that is kind of a reference implementation for a raw processor. When a new camera comes out, the lag in support by raw processors, commercial and FOSS, is the inclusion of these nine numbers for that camera. There can be other numbers supporting the camera, but these are what I’ll call “first-order essential”.

In the path to an output image file, the camera colors need to be mapped to an appropriate colorspace, as the camera space is too big for most media. sRGB, AdobeRGB, etc, have their own set of primaries, nine numbers that describe their color gamut in the same terms as the camera primaries. The conversion is essentially a mapping of the pixel colors from camera space down to the output space of your choosing. There are different options for specifying this conversion; the predominant one simply leaves “low intensity” colors alone, as they occupy both spaces, and saves the heavy lifting of figuring out where to put colors from camera space that are outside the output space. This option is called “relative colorimetric”; there are three others in the ICC profile realm.

The reason I’m describing this bit of arcanery is that, linear camera matrix profiles, as the profiles containing the nine numbers (a 3x3 matrix) are called, typically “don’t lie” about the camera’s spectral response. They don’t contain non-linear curves or discontinuous LUTs that make the tones or colors look different. I’m abstracting this a bit, but I think of linear matrix camera profiles as the base characterization of the camera; anything else is a “look” or adaptation for some purpose. For instance, one can scooch the blue primary around to accommodate extreme blues, from certain theatrical LED lights for example, but in doing so one messes with all the other colors too.

So, I’d surmise you’ve been looking at a pleasing “look” in C1 renditions, and now you’re somewhat taken aback by the truth of your camera… Now, take the red pill…

[pgoelz (Paul Goelz)]

@ggbutcher

So, I’d surmise you’ve been looking at a pleasing “look” in C1 renditions, and now you’re somewhat taken aback by the truth of your camera… Now, take the red pill…

Thanks for that, it was educational. I don’t fully understand all the nuances of color space but I’m working on it.

So what does a fella do in RT if he… likes that “pleasing look” in C1? Other than stay with C1, of course The color just looks more smooth, rich and natural to me than anything I have been able to produce in RT.

FWIW, I have now also tried ACDSee and Darktable. ACDSee seemed to come closer. Darktable did not. And neither had acceptable de-noising or ergonomics. I tried Lightroom a while back and I don’t recall any issues with color but I refuse to commit to raw development software that stops working if I ever stop paying.

Paul

[Entropy512 (Andy Dodd)]

@Morgan_Hardwood

@Entropy512 the difference between a target shot properly on an overcast winter midday vs shot on a cloudless summer midday is greatly smaller than the difference between a properly shot target under any daylight vs no DCP at all, so go ahead and shoot - we can always submit a better one if summer ever comes around again.

Ah, I was surprised that the difference between dismal grey and blue sky would be that little… I can see “minor clouds” being a minor effect, but I would have thought that if it’s Ithacating you’re hosed.

Hopefully the parking lot will be clear enough to not gunk up my colorchecker if I try to shoot it Thursday. Today is just a horrible mess of slush all over.

[ggbutcher (Glenn Butcher)]

@pgoelz

So what does a fella do in RT if he… likes that “pleasing look” in C1? Other than stay with C1, of course The color just looks more smooth, rich and natural to me than anything I have been able to produce in RT.

If C1 is using a DCP profile for your camera, just drag it over to RT, see if that scratches the itch.

Me, having taken the Red Pill, I’d treat the C1 look as a “look”, akin to film emulations, and see if there’s a LUT to be had or constructed. I’m not specifically familiar with that hobby, but there are folks here who are. RT/dt/both (not sure which/both) have the capability to apply LUTs to images.

WRT camera profiling, I cut my teeth with @Elle Stone’s articles:

Articles and tutorials on Color management

[ggbutcher (Glenn Butcher)]

These dialogues (diatribes?? ) sometimes compel me to go look…

http://rawpedia.rawtherapee.com/Film_Simulation#Make_Your_Own

The above points to the specific itch scratcher, but you need to read the whole Film Simulation page to learn about HaldCLUTs…

[agriggio (Alberto)]

Hi,

@pgoelz

So what does a fella do in RT if he… likes that “pleasing look” in C1? Other than stay with C1, of course The color just looks more smooth, rich and natural to me than anything I have been able to produce in RT.

Here are my two cents. I used the ART fork of RT, which has a slightly different processing pipeline, but here’s what I could get. I basically only used the tone curve and camera-specific DCP profiles that I generated with dcamprof using the color targets in the dpreview studio scene tool (plus sharpening and noise reduction to match what you did with C1). If anything, it’s the C1 renderings that look a bit yellow to me…

DSC04493.ARW.arp (9.5 KB)

DSC04425.ARW.arp (9.6 KB)

[pgoelz (Paul Goelz)]

@agriggio

Here are my two cents. I used the ART fork of RT, which has a slightly different processing pipeline, but here’s what I could get. I basically only used the tone curve and camera-specific DCP profiles that I generated with dcamprof using the color targets in the dpreview studio scene tool (plus sharpening and noise reduction to match what you did with C1). If anything, it’s the C1 renderings that look a bit yellow to me…

Interesting. The room image looks great! The mantis image, not so much. To my eyes it is too cold / green.

Do I need to generate my own DCP files or would yours work for my a6000 / a6500? I looked and C1 does not seem to use DCP files so I can’t borrow them. Creating my own looks to be a serious effort in learning and execution

And pardon my ignorance, but what is an ARP file extension?

I would also like to thank EVERYONE who has participated so far. This has been most interesting and educational.

Paul

[agriggio (Alberto)]

@pgoelz

To my eyes it is too cold / green.

sure, it’s definitely less yellow than c1. my point was that I don’t think there’s anything preventing you to get less yellow results with arw files from RT…

[pgoelz (Paul Goelz)]

@agriggio

sure, it’s definitely less yellow than c1. my point was that I don’t think there’s anything preventing you to get less yellow results with arw files from RT…

Well, it is certainly less yellow than the original or C1 but it doesn’t look natural or anything like it did in the viewfinder either. It was a warm sunny day and this version looks very cold. Maybe it is just a subjective thing, But I like what I can get from that RAW easily from C1. I don’t like what I can do in RT, even with a lot of effort. I can get close, just not all the way.

Your result on the room image, however, was very acceptable and better than I have ever gotten with that image in RT.

Are we kinda reaching the end of the road here? C1 is doing something to my images that RT is not, and no amount of conventional manipulation in RT can duplicate it. It is what it is? I learned a lot in the process and I know better where to look for further improvement.

Paul

[agriggio (Alberto)]

Hi,

@pgoelz

Well, it is certainly less yellow than the original or C1 but it doesn’t look natural or anything like it did in the viewfinder either. It was a warm sunny day and this version looks very cold. Maybe it is just a subjective thing, But I like what I can get from that RAW easily from C1. I don’t like what I can do in RT, even with a lot of effort. I can get close, just not all the way.

I think you are still missing the point I was trying to make, i.e. that there’s nothing in RT that makes it render files with a yellow cast.

@pgoelz

Are we kinda reaching the end of the road here? C1 is doing something to my images that RT is not, and no amount of conventional manipulation in RT can duplicate it. It is what it is?

If you want a slight yellow cast, there are many ways to do it. Here’s one (again, using ART, but with RT it wouldn’t be much different):

DSC04425.ARW.arp (10.0 KB)

And here’s a composite of the two: one half is your C1 rendering, one half is my ART one. They are certainly not identical, but are acceptably close for me. Clearly, you might disagree:

[pgoelz (Paul Goelz)]

@agriggio

I think you are still missing the point I was trying to make, i.e. that there’s nothing in RT that makes it render files with a yellow cast.

I see your point. However, as I understand it there is a lot of processing required to get a usable image from the raw sensor data and my assumption is that C1 and RT do it differently. Again, I am seeing a fundamental difference in the color rendering of the same file in C1 and RT without any user-applied color manipulation. So you are saying that what I like about the color rendering of C1 may just be a happy accident and not to expect it from RT because that is not how the file looks? I’ll have to think on that one. You may be right. But is there one and only one “correct and accurate” way to translate raw sensor data into a usable “starting point” image?

One thing I did discover today is that the color in the mantis image in RT changes (for the better) if I add some haze removal. That was quite a surprise and I’m not sure why that would be. Without haze removal, the mantis image looks cold and nothing like it does in C1 and I am unable to duplicate my C1 results. But if I add a moderate amount of haze removal, the image warms up and comes very close to how it appears in C1. I would even call the image “acceptable”. Note that I do have a small amount of “clarity” added in C1 by default which I think is doing something similar to RT’s haze removal. However, removing or modifying the “clarity” in C1 does not affect the color at all.

Haze removal does not seem to have much (if any) effect on the color in the high ISO room image, though. Not sure why it makes a difference in the mantis image and not the room image.

Paul

[paperdigits (Mica)]

@pgoelz

Again, I am seeing a fundamental difference in the color rendering of the same file in C1 and RT without any user-applied color manipulation

I think this is where the hang up is. When you apply nothing in RT, its really nothing, and we know its nothing because we can inspect the code. When you open an image in C1 with nothing applied, there are things applied… But what? We can’t really know. So “no use applied color manipulation” isn’t an apples to apples to comparison. There probably isn’t a way to do a “no.processing” apples-to-apples comparison between RT and C1.

Different tools have different trade offs. You may need more work for RT or just to find which tools you like and to get a good starting profile.

[pgoelz (Paul Goelz)]

@paperdigits

So “no use applied color manipulation” isn’t an apples to apples to comparison. There probably isn’t a way to do a “no.processing” apples-to-apples comparison between RT and C1.

Different tools have different trade offs. You may need more work for RT or just to find which tools you like and to get a good starting profile.

Agreed. I have always assumed that the path from my original file to the “un-manipulated” first look in both programs was different. The point (for me) of this dialog is trying to identify what that difference is and how to control it. I get that it might be more work in RT, and why that might be. What escapes me so far though is how to get there. I’m getting closer, but I’m not there yet. I can see and identify what I don’t like but I have not found a way to fix it yet.

Paul

[Thanatomanic (Roel)]

@pgoelz

But is there one and only one “correct and accurate” way to translate raw sensor data into a usable “starting point” image?

Well, no. There are multiple ways to interpolate the raw sensor pattern to form a full size RGB image. I don’t think you have a choice for the demosaicing algorithm in C1, but there are multiple options in RT.
However the thing that influences colour the most is the calibrated conversion taking place that maps the raw sensor data to RGB pixel values. This calibration may be different in different software.
You can always try and produce your own calibrated conversion profile, which may help to correct the colour cast you perceive. How to create DCP color profiles - RawPedia

[ilias_giarimis]

Fact is that the current RT’s dcp profile gives consistently redish-yellowish greens … see measures on dpreview’s studio scene …
(I just tuned raw white point to take the expected value on the neutral grey patch)

DSC09204.ARW.pp3 (11.9 KB)

@agriggio Alberto hi … normally you should take much better metrics on this scene as your profile was built on this

BTW … very strange that RT’s dcp has these matrices !!

[ggbutcher (Glenn Butcher)]

Interesting… maybe if the DCP for this camera were removed from RT’s search path, it would then go to the camconst.json entry, which is referenced to the Adobe DCP, and looks okay when I use it in rawproc. @pgoelz, you might try this…