Raw Files opened in DT start "way off"


#1

When I first started to learn Darktable, I had a similar problem in opening RAW files from my Sony A700. They were over-saturated, color balance way off (too warm) compared to what the camera gave me as a preview or the preview Irfanview would give me for files stored on my computer’s hard drive.

I could open these same files in Lightroom, RawTherappee, or Sony’s RAW converter absent this problem.

I have no idea what I did, but the problem seemed to go away, and I processed many a RAW file through DarkTable that were captured on my A700.

Recently, I picked up a Sony A900, love it. But, this problem with opening RAW files in DT has re-surfaced. I did not consciously make any changes that I feel might have caused this, but am at a loss as to what I may have or am doing wrong. I don’t know where to start looking.

I see that the project has RAW samples for my camera, and, so, I assume that RT detects my camera’s profile. The puzzling part is that with neither camera did the problem start right away. I was able to process a number of shots during my first couple of sessions in DT with both cameras. Then the problem appeared.

Is there some icon that I can click to set DT to its default setting?

In order to process and distribute photos I shot during our Christmas celebration, I booted back into Windows and used Lightroom and Photoshop CC. No problems there, but I would prefer to use DT and Gimp 9 via Linux instead.

With all due respect, please do not tell me that I am not accounting for the difference between the camera’s jpeg used for its LCD review display and the underlying RAW file. I am aware of this difference (Irfanview uses that same JPG to produce its preview image). The problem appears to be occurring only with DT, not with other RAW processors (as mentioned, I use RT, LR/PS CC, and I also have Lightzone). With none of these other aps do I experience this problem. Having spent many hours familiarizing myself with DT, I really do prefer it.

Suggestions very much appreciated.

Happy holidays.

Caruso


#2

What base curve setting do you use?
Could you provide an example raw and your .xmp?

As for resetting darktable: https://askubuntu.com/questions/194616/how-to-reset-darktable
Backup your /.config/darktable first.

BR
Matej


(Roman Lebedev) #3

Please contribute the full sample set to RPU, we only have one non-CC0 sample.

Please contribute the full sample set to RPU, we only have one non-CC0 sample.

The only correct way to write that is: darktable, one word, all lowercase, no space/punctuation marks.

Did you create some presets that are set to be applied by default?
Load some new image in darkroom, which modules are enabled in the right sidepanel?

With all due respect, you have correctly identified the difference by yourself. That is the underlying reason for the difference.

The proprietary stuff has some proprietary knowledge (e.g. from the camera manufacturer) on how to process the images. RT has some support for DCP/LCP adobe profiles. DT does not have that, and it won’t have that. So the default results will naturally differ.

You can reach similar or better results by editing the image in darkroom. Also, see https://www.darktable.org/2016/05/colour-manipulation-with-the-colour-checker-lut-module/


#4

I use the base curve that comes up. Where does it come from?
I will upload a sample of my RAW and .xmp.
I also was poking around with “input color profile” and I find that DT generally defaults to enhanced color matrix, but clicking on each choice, I find that “linear Rec 2020 RGB” gives me a starting point for the image that is nearest what the preview looks like (on my camera, from Irfanview, etc.). I’m not certain this is a good thing or not.

I’m going to have a look at the forum “rules” regarding uploading, and will get back to you with a sample file.

Thanks for taking the time to respond. I know I will sort this out at some point. I am really pleased with RT’s features. When it is working for me, there is very little left for me to do in GIMP unless I want to do some cloning.

Thanks again,
Caruso


#5

Didn’t really find any upload instructions. Hope I am not breaking any rules. Sample RAW and my xmp are attached.

DSC07081.ARW.xmp (2.1 KB)
DSC07081.ARW (12.5 MB)


(Mica) #6

You are not breaking any rules and you are welcome to upload raw files here, as you have done.

However, @LebedevRI has requested you upload samples to our raw file repository at https://raw.pixls.us, which has it’s own set of requirements listed on the page. Uploading a sample with ensure your camera is supported and continues to be supported.


#7

Is there anything special I have to do to grant the level of release that is requested. I’ve read about it, but it doesn’t really make much sense to me. The samples I would upload would be adequate for profiling, not much use to anyone for anything else.
Thanks.
Caruso


(Peter Cripps) #8

Hi Caruso, I downloaded your files and opened them in darktable. After applying a spot white balance correction, the results seemed pretty good. Maybe I’m not critical enough?!


(Mica) #9

To grant the CC0 license, check the box that states so when you upload.


#10

The basket is a very light tan color, no green as shown in your correction. I’ve been playing with new captures and DT most of the afternoon, and am happy to report that my results seem to be getting better, even when DT chooses my input color profile. I still do not understand the source of the base curve or why “enhanced color matrix” input profile seems to work sometimes and not others.

This forum input is a refreshing source of information, and I am committed to developing an understanding of how this application works. It is a matter of when, not if.

Thanks again for your responses.

Caruso


(Peter Cripps) #11

Yes, now you mention it, I do see a slight green tinge on the output that I posted…

I was curious to see how the same file opened in Lightroom: no green tinge, although colors are perhaps more saturated than the darktable output.


(Glenn Butcher) #12

Here’s @yertchuck’s image, with a histogram:

Note the mis-alignment of the red channel from the other two in the right peak. It’s slight, but it’s the source of your perception of “green” cast to the basket. It’s easy to fix with a per-channel curve:

The curve in this screenshot is not really “curvy”, it’s a line with the upper-right control point scooched left about 10 units. Note what it does to the right-hand red peak - it now aligns with the red and blue peaks, and I think the basket color is a bit more appropriate to your expectation. Oh, the curve channel is set to “red”, so the curve only works on the red values of the pixels.

I’m illustrating the concept with my homebrew program, but you can do this in any of the programs that have per-channel curves. You drag the upper-right point to the left to move the peak to the right along the top edge; if you want to move the peak to the left, drag the point down the right edge. Note this is not for the shadow values, but you can mess with them using the bottom-left point.

The profiles might be messing with the color casts. I personally have moved away from them, and deal with as close to what the camera captured as possible, using tools like I describe above.


#13

Here is a jpg of the same photo as converted via Irfanview. The colors are very true to life. “I peronally have moved away from them [profiles].” How have you moved away from them in DT? I would like to give it a try.

Thanks to all for your help and advice. I will keep trying.

Caruso


(Ingo Weyrich) #14

What do you mean by converted via Irfanview ? Doesn’t Irfanview simply show you the embedded jpeg?


(Peter Cripps) #15

For reference, here’s what darktable gives me, before applying the white balance correction from my previous post. I don’t see a green tinge, altough the colors are still not quite the same as the jpg Caruso has just posted.


#16

Irfanview is one handy little tool. It will not only read RAW files, but you can use the “Save As” option in the File drop down to save a RAW file as a jpg if you like. Is it using the embedded jpeg? I don’t know, nor do I really care. I needed to show you where I would like to start with an image out of my A900, and Irfanview gets the job done with a click of my mouse.

Someone wrote in an earlier thread when I first ran into this problem with files from my A700 something to the effect that, if all I wanted to do was recreate the jpg, I should stop shooting RAW. It’s not that I want to recreate the jpg, but, OTOH, I am not at all put off with the jpeg engine in either of my cameras. For me, something close to what they offer is a good place to start. What really interests me with DT is the ability to clean up things that neither camera (perhaps no camera at this point) can do. In this sample photo, for example, I could reduce the intensity of the reflections coming off the plastic in which the basket is wrapped. If this were a daytime capture, I might want to adjust the levels of the windows in the background to enhance or de-emphasize the outdoor scene visible through the panes. It is great for taking off the shine on a human subject’s nose or forehead, applying blur to certain areas of the photo, and on and on.

I have taken photos of my extended family seated at a long dining table using a single flash bounced slightly backward off the ceiling. While this generally makes for a decent soft light for those seated nearest the camera, those seated at the far end may be underexposed. DT does wonders in that situation to give those “far-away” subjects some pop in my photos.

I am not really looking to throw away the basic photo that my camera suggests because, frankly, I doubt if I am good enough at this point to come up with an alternative that is even as good as what the camera is already giving me. I do enjoy the flexibility that DT gives me to make the aforementioned adjustments when I deem them desirable.

Thanks for your reply.

Caruso


(Alberto) #17

I think this can be paraphrased as “let’s start from a blank canvas”. I agree (if that’s not what you meant, sorry for misinterpreting!), but I’d say that an input profile that tries to be as accurate as possible gives you just that.

Here’s what RawTherapee produces when using the (automatically-selected) input profile for your camera (after some tweaking to the contrast):

I’d say close, but still not quite the same as your jpg. But in my experience, Sony cameras tend to boost the reds (at least mine does), so let’s try to do the same in RT (using the HH curve in L*a*b* adjustments):

Looks closer to me. As to which of the two is more accurate, only you can judge I suppose…

Here’s my pp3 in case you are interested:
DSC07081.ARW.pp3 (13.5 KB)


(Roman Lebedev) #18

In any case, @carusoswi, please contribute the full sample set for both the Sony A700 and Sony A900 to the https://raw.pixls.us/. See Raw Samples Wanted for details.
In particular, each camera can produce two raw formats - RAW and cRAW, so two samples are needed, per camera. The camera needs to be pointed at some static scene, and in a stable position (tripod), so that the content of all the raw samples matches.


(Glenn Butcher) #19

Ah, I probably need to clarify: I use an input profile, but it’s a calibrated camera profile I made from a ColorChecker target and @Elle’s instructions. Prior to having that, I relied on the so-called “adobe_coef” numbers that are in dcraw, and probably Lightroom as well. What I was thinking of when I wrote that were the profiles that do various manipulations, like “Vivid” in the Nikon list. But your invocation of “starting with a blank canvas” is indeed a good description of my perspective.

What I set out to do was understand each thing done to the raw data captured at the sensor; I’m nowhere near having a comprehensive understanding yet, but I’ve figured out enough to make decent images “from scratch”, so to speak.

“accurate”, I’ve come to learn, is not the best adjective for describing color. The palette of colors humans have come to understand are an amalgam of interpretations of both specific wavelengths of light as well as mixes of them. “Pleasing” is probably a more relevant adjective, albeit subjective at its root. I’ve found all the above renditions of your original image to be that. I had to inspect the histogram of the “green” image to see it, the tint did not pop out to me regarding the image.

I took some pictures of my wife’s Christmas decorating. In the batch-processed images, the family room pictures were “warm”, due to the incandescent room lighting. She loved them, and would hear none of my pleas to white balance them for her. C’est la vie…

Well, in locating this picture for posting, I looked at the rendition where I did white balance; it looks very sterile. She’s usually right about most things… :smiley:


#20

What is your display profile set to in darktable? You can see it when right clicking the button in the lower toolbar of lighttable. Also, what display profile source did you select in the preferences, xatom or colord? And how many monitors do you have?