DNG support in RT?

Software RawTherapee
1

Hey,

Personally, never cared for DNGs (or needed to use them…, ever). But I’ve understood why the format is needed from the get-go (in 2004).

So, I never thought that “RAW”-Therapee supported DNGs, since, technically, they are not “raw” image files. But then, I came across, like, a post where they were discussing processing DNGs in RT, then I saw something somewhere else… It was a while ago and I can’t remember what it was all about…

I know my RT (5.8) can’t load DNGs, so, what’s the deal? Am I stupid, or wth is going on?

I have a whole bunch of photos I took with my phone, all DNGs. It would be nice to process some of them in RT. But, hey, you know…

I apologize in advance for this stupid question.

Thanks!

2

Your computer OS version? DNG’s generated by which phone/app/os?

3

I feel like I just woke up in a parallel universe… Hmm…

Anyway, it’s an ‘older’ Windows 7 desktop, and the photos originate from a Xiaomi Mi A3 (with zero processing by any software).

4

It is missing at https://raw.pixls.us/
Please upload a raw sample there.

5

It is perfectly possible to have raw (non-demosaiced) images inside a DNG container. Also, RawTherapee is perfectly capable to load DNG files, but to support your camera/phone it needs to have some additional information. An uploaded sample to the repository would help. You can also always create a support issue on GitHub.

1 Like
6

Now we’re getting somewhere…

So, do I have to capture a specific target? I can’t find anything about it on that page.

7

Since it’s a 2019 Droid, it could be some sort of Android 9/10/11 DNG standard file.

8

At raw.pixls.us:

We are looking for shots that are:

Lens mounted on the camera, cap off
Image in focus and properly exposed
Landscape orientation

We are NOT looking for:

Series of images with different ISO, aperture, shutter, wb, lighting, or different lenses
DNG files created with Adobe DNG Converter
Photographs of people, for legal reasons.

Just take a decent photo. If you download raw files from raw.pixls.us you will see how others have taken their photos.
From my phone https://raw.pixls.us/getfile.php/4128/nice/OnePlus%20-%20ONEPLUS%20A6003%20-%2016bit%20(4:3).dng

9

HIRAM:
That’d be great! So, how come I can’t open the photos in RT?
I can see my NEF files just fine but the DNGs aren’t even on the list. Am I doing something stupid?

10

Hard to tell without a sample to try. Is dng checked in the settings?

bild
bild791×610 65.8 KB

11

WOW! I knew I was doing something stupid!

All these years and I’ve never had the need for anything other than my NEF photos. That is why it was the only extension on my list!

If someone told me this morning that by lunch time I’d be editing DNG photos out of my phone in RT, I’d tell them to go see a shrink!

So, like, wow… again…
Well, I’m off to play with my phone-photos in RT (still sounds crazy)…

Since I haven’t done any ‘proper’ processing yet it’s still early to say anything but it looks like I won’t be sending in those sample photos after all…

It was such a long shot and I’m so glad I posted here to ask.

Thank you all!

12

Hahah, the file list strikes again!

Not true. The majority of DNGs are true raw image files, that was the whole intent of them. There’s a subset of DNGs sometimes referred to as “Linear DNG” which may be demosaiced but still in camera color space, and it is alternatively possible to have a mosaiced or demosaiced DNG that was derived from a multiframe stacking approach. Apple’s “ProRAW” variant is, in fact, just that - DNG 1.6 resulting from the output of their multiframe superresolution algorithm.

Android DNGs can vary from manufacturer, but they SHOULD be standards compliant. No guarantees of that, especially from Xiaomi. The most common DNG screwup is an improper color profile, so if you have problems with unusual/difficult to work with colors, you may need to invest in a ColorChecker and profile the phone yourself.

The only known issue with RT is that there are metadata issues with many DNGs - work is being done on fixing that.

13

Yeah, that “file list”… I’ve gone over the settings a couple times but I guess I just missed the “file list”, because there was no list to speak of. Apart from the NEF, there was nothing there, checked or unchecked. I probably removed all the other extensions years ago.

Thanks for the info on the DNG. I’ve been reading up on it myself, since, now, I have to deal with the format on a daily basis. It’s nice that RT supports it as much as it does, which allows me far greater flexibility than LR.

For what it is, the main camera module on my phone isn’t terrible but it’s not that great either, by today’s phone’s standards. It does produce weird colors. The palette is all over the place. I actually do see myself investing in a solution which would allow me to create custom profiles but only because I’ll be able to use it to profile my future phones (and cameras), as well.

I work in a color-managed environment, so, for now, I should be able to tweak the colors “by sight” and save those adjustments as a preset. I know, it’s not ideal but it’s better than not doing it at all.

My real problem with this camera is the atrocious vignetting. I was hoping to address it, using flat-fields in RT, however, from what I’m seeing… The jury’s still out but I might have to just wing it.

A couple of quick questions (so I won’t have to start another thread):
DNG files are supposed to contain all the “develop” settings in them, not in a sidecar file. At least that’s how it is in Lightroom. From what I’m seeing, RT does not write anything to the DNGs but rather, it saves everything in the pp3 files (please correct me if I’m wrong).
Now, what happens when a DNG image file is imported to Lightroom and some adjustments are applied to it (and saved in the file as a set of instructions), does RT simply ignore those and load the image in its “untouched” form, or does it use those “settings” as a starting point? In RT, I want my DNGs to be as close to the original capture as possible (with no stupid sharpening or noise reduction).

Please advise.

14

Lightroom’s development settings will not apply to other photo editors.

15

Thanks, but how about other, “more native” settings? I actually have no idea how my phone saves those DNGs. Like, does it apply any “corrections” at all? As I’ve said, I want to bypass all of that and see the DNG image “as it was captured” (or as close to it as I can get).

16

You’re talking about OpCodes embedded in the DNG… Does your file have any and, if so, what are they? ExifTool is your friend :wink:

Apart from those standard ones that 3rd party SW should be able to deal with, there might indeed be some limited “corrections” going on before saving to DNG, done either by the sensor directly, or the phone:

  • defect pixel removal/interpolation
  • phase detect pixel removal/interpolation
  • Quadra/Tetracell/4-cell remosaic to Bayer

This is because there would be no way 3rd party SW would know how to deal with those (easily)…

1 Like
17

If you set the neutral profile i RT you get a very plain file no tweaks. In RT you can turn everything off.

18

Don’t know if this has been asked yet, but ehhh?
What do you call a ‘raw’ file? Something straight from a camera? Most phones do and some Sigma cameras AFAIK. Do you call it raw when it contains the original Bayer data? Since that is what it is (ahem, can be).

1 Like
19

kmilos:

You know exactly what I’m talking about. I have long suspected that there was something shady going on behind the scenes in DNG files. And that’s with zero reserach. I was able to determine that just by looking at the same photos in various apps on my phone, then Lightroom, ACDSee and now, RT.

Upon some digging, I’ve learned, confirming what I’ve been suspecting, that with photos that come out of various devices as DNGs, there are almost always ‘hard-coded’ instructions which make the picture look prettier than it actually is: i.e. • Crop out nasty, vignetted corners, • Correct lens distortions, CA • Apply vignetting correction, • Apply sharpening, noise filter, • Correct for nasty tones, etc… All of the above happens BEFORE you are presented with a “clean” image in CameraRaw or Lightroom, and I’m beginning to suspect that even RawTherapee hands you the image AFTER, at least some of the above has already been done to it.

Yeah, man… those OpCodes… I knew they were there, just didn’t know what they’re called.

I’m (relatively) new to DNG and, if not for taking pictures with my phone I wouldn’t have the need for it, cuz it is UUUG-LY! These should be a nice little read for y’all:

https://exiftool.org/forum/index.php?topic=10763.0

If you need 100% control, DNG is a rabid rabbit hole.

20

That doesn’t make it not a raw file since those opcodes are just metadata.

A GOOD raw processor should notify you of every opcode that is in play and give you the option of whether to use it or not. RT supports fewer opcodes currently than many other raw processors, but supporting the opcodes (WITH the ability to turn their handling on/off) is definitely something many of the developers would like, but is lower on the priority list than other things (since many of the opcodes can be replaced with enduser profiling steps such as lensfun calibration, and because many DNGs have broken metadata.)

For example, as far as color management, RT gives you multiple options from “no color matrix” to “separate DCP profile”. For devices with broken embedded color metadata (such as the Xiaomi Mi Sphere - yup, another Xiaomi product!), you really need a ColorChecker to make a DCP profile. What you describe hints at your device having a broken/improper ColorMatrix tag. You can kinda sorta work around this with manual color adjustments, but given that the actual color correction with a proper profile is done with a combination of a matrix AND a lookup table, you really want to invest in something like a ColorChecker Passport. (I use the Passport myself, and it was critical to making my Mi Sphere usable at all.)