Replicating Fujifilm JPG colors from raw files?

There is a global color mapping module…its older so nobody ever talks about it… Maybe it could be updated to incorporate the code from CC module that actually does mapping to a target… This module was designed to transfer the color “look” of one image to another…

https://darktable-org.github.io/dtdocs/en/module-reference/processing-modules/color-mapping/

Or maybe if we were targeting a single color it could be included in the Color Lookup as an additional function…

I think an easy and quick way to do it globally is the grab the look as a lut in Davinci.Resolve… or you can also use darktable chart and try to match to the JPG…those would be global approaches rather than a single color… So there is a sort of pick a color and remap that or there is the global need to grab a color grade… and the tools might be different in each case…

Yeah, absolutely. I use this module from time to time, but it really doesn’t work very well, probably because it’s so old and needs some TLC. It also could do with a way for the user to have more control, rather than its current automated approach.

My initial thought about adapting “area colour mapping” from the Colour Calibration module and possibly porting it to something like Colour Balance RGB was because they are both Aurélien modules, and I wondered if he was using similar code for them. I can’t imagine his code would work well with older modules like Color Lookup Table and the Color Mapping modules. But, again, it’s not my area of expertise.

1 Like

There‘s already a feature request to match colors with embedded jpg.

And in quite old days a request to match exposure FR: adapt initial exposure to embedded jpg · Issue #11346 · darktable-org/darktable · GitHub

unfortunately that’s not that easy because there are several steps in the pixel pipe so at the position of exposure or color calibration (both modules that supports matching) that affects tone values and colors.

Up to now there wasn’t a well defined specification on how to achieve the same result using darktable module algorithms…

2 Likes

Thanks @MStraeten . It doesn’t look like much will happen with that FR until someone can come up with the math. Wish I could help, but sadly I can’t.

This is not a feature I feel like I urgently need. I’m mainly happy just keeping the JPEG when it’s good, and then developing the RAW when I want a different look, although there are times when you just want to “copy” a few colours over, yet keep all the tonal latitude that RAW files provide. So, I was just looking for possible low-hanging fruit, and I wondered if an expanded version of the spot colour mapping feature of Colour Calibration might be it.

Any idea why the Colour Mapping module doesn’t work very well? Is it because it was originally developed for earlier versions of Darktable and is not now very compatible? Or did it never work very well? I don’t think I discovered it until about 2 years ago, so I don’t know if it worked well in the early days. I’m wondering how much work would be needed to get that one working better with modern Darktable…

The camera jpeg engine is much more complicated than being just a mapping or lut. If it was a mapping or lut, then we’d be able to replicate them by shooting a color target and measuring the difference. But that doesn’t work. And its a different formula for every jpeg style from every manufacturer.

3 Likes

I think that’s incorrect. The camera manufacturers themselves provide LUTs for transforming their Log-Video to their film simulations, and LUTs for transforming their raw video to Log. So a LUT implementation is clearly possible. Lightroom and Capture One picture profiles are also known to use LUTs internally.

Furthermore, camera JPEGs have been produced for many years, even on woefully outdated camera processors. I don’t think they were able to do much more than apply a LUT (and denoising and sharpening).

Some modern cameras include features like “clarity”, which can’t be replicated by a LUT, but the Fuji cameras that do this take a sweet second to apply the effect (if enabled), so clearly, this must be thought of as a “complex post-processing” effect, and is not part of the regular JPEG engine.

1 Like

Then it should be trivial to generate them, no? Where are they? You’ve gone to great lengths to try and generate LUTs and yet they don’t quite seem to match up.

1 Like

Actually, I find they work just fine if you replace sigmoid/filmic. Stuart Sowerby’s LUTs are a great example for that.

What I found much harder is bringing the output of filmic/sigmoid in line with the intended colors. That is, how to preserve the intention of the film simulations after having done some edits. This requires decomposing the LUT into various tonal and hue components, such that they can be applied independently. That’s not so straight-forward.

Hey there! I realize I’m a little late for the party and I am not 100% sure, if this tip didn’t come up before, although I at least skimmed all the replies up to now.

A while ago I found a very extensive article about the color science behind the Fujifilm film simulations.

I realize, that recreating these colors with darktable on the basis of said article would be a very tedious and cumbersome process (because I tried). But most of the tweaking could be possible with the current modules and a proficient user. And, if successful, this might be the closest you could get to the original.

See here:

https://www.imaging-resource.com/news/2020/08/18/fujifilm-film-simulations-definitive-guide

4 Likes

Nice article. I’d love to see the article updated to include the newer film simulations.
Interesting that the conclusion is that it’s basically impossible to recreate the looks exactly in post-processing.

The article actually states that it’s impossible to recreate the look of film in post processing. And then it conflates this true statement with the false conjecture that therefore, film simulations can not be achieved either.

Film has color-sensitive chemicals whose sensitivity varies with wavelength. This is roughly analogous to the color filter array on digital sensors. But the sensitivity curves differ, and some films have more than three. During development, the exposed chemicals are bonded with dyes, and various inhibitors and strengtheners that modify each dye’s response, depending on neighboring dyes. This process is both tonal and spacial. There can be many more than three dyes. This whole process is astonishingly complex, and indeed can not be fully recreated from a digital sensor. See the amazing book Making Kodak Film, by Shanebrook for more detail.

However, Fujifilm Film Simulations are just color profiles. They are routinely emulated very accurately by many raw developer program. Fujifilm themselves provide LUTs that apply the Film Simulations to their log video. Lightroom and Capture One and DxO Photo Labs and Silkypix have built-in Film Simulation profiles.

The only caveat is that you need to use a similar processing model as Fujifilm does if you want to render highlights correctly. The Base Curve and some Filmic variants in darktable are much closer to Fuji’s JPEGs than Sigmoid. Silkypix and Capture One similarly have discolored, saturated highlights, which is closer to Fuji JPEGs than Lightroom’s and DxO’s desaturated highlights.

5 Likes

Perhaps someone has already covered this here, but I wanted to point out using LUTs can be a little tricky.

Notes to myself: Rule 1 - always read the README

I discovered this while trying to use Fuji film simulations. Two that I downloaded and one set that I borrowed from 3DL files all require the camera profile to be “linear.”

In RawTherapee, I can acheive this by turning off “Tone Curve” and “Look Table” under “Color Management.”

I also found something rather interesting. Early on in this thread someone posted a RAF of a dog standing on its hind legs. So I downloaded it, opened it in RawTherapee, set the dcp file to “linear”, the film sim to Provia, and the results were close. Then I went into LAB and reduced the contrast to -4 and the chromacity to -2. The result was better. Finally, I selected “Camera Standard” instead of the auto-matched camera profile and bang nailed it.

Not sure this helps, but this is a question I’ve been trying to answer for myself ever since people have been touting Fuji film sims and the Hasselblad “Nature” look.

But isn’t his point that they are approximations and you can’t get the exact same results that you can get in-camera?

Ultimately, I wouldn’t care because they are good enough, but the examples provided showed some differences.

That’s a good question, actually. It isn’t to me. Why even use a raw file if you were only interested in the JPEG rendering? But I suppose accurate film simulation profiles might be a big priority to some folks.

I do feel that Capture One in particular comes very very close to the native Film Sims. As do Stuart Sowerby’s LUTs with appropriate processing. Close enough that I can’t tell them apart in direct comparisons.

But even the relatively less accurate Lightroom/DxO/sigmoid variants are close enough to evoke the same look or style, and look no worse in comparison. Just slightly but visibly different.

1 Like

Yes, I think some people swear by the Fuji look and only shoot JPEG because of them. I personally am happy enough with an approximation, and I usually put my own look on my shots anyway. I think you need to look really hard to see these kinds of subtle differences. Some people enjoy this about photography, but I’m more of an “initial impact” kind of viewer. I can overlook many of the finer details and nuances for the bigger picture, as it were.

As for Capture One, I think Fuji had an exclusive agreement and worked closely with them on their film sim profiles, maybe also SilkyPix, which is why they are so good. But I’m not 100% sure about that.

2 Likes

For Fuji, you can reprocess the raw in camera if you want to change or tweak the film simulation.

Yes, and if you have already transferred the photos and deleted them off your memory card, you can still tether the camera and use the X RAW software to try out new film sims.

It’s a nice option to have, but personally, I never do it because it just seems such a clunky workflow solution. I’d rather not tether my camera and use the software, which is slooooow. I actually forget I can even do it. I sometimes wonder if I should have used a different film sim when looking at a photo in darktable, but I usually just shrug and move on. Does anyone here use X RAW regularly?

2 Likes

I think I tried it once, found it slow but may have been that I was using a slower USB port, I find it clunky as well, plus you have to have the same model camera that took the pic plugged in, I’d like to be able to use the pc to process raws on the card in camera, I can’t remember if the fuji phone app allows that as well, though fuji’s in camera raw processing is pretty usable

Oscar Wilde’s criticism of British Art: that it was well intentioned but lacked the flair of the Continent means that I can probably not use Fuji so long as the subject matter is worthy

I thought this picture might have potential in the context of this post

The out of camera JPEG tweeked in gimp

the raw file

dandyraw.CR2 (25.1 MB)

It is worthy I feel because hopefully it is contributing to the cause highlighted by people like plant life, seriously needs it

I don’t know if you can use Adobe Digital negative converter in wine but I have to admit being more vegetarian than vegan both in diet and image processing so I tend to use the out of camera jpeg if it looks okay to me and then use raw processing or a 16-bit TIFF if I feel there is a bit of work to do; as recommended here the Tiff being produced in the camera manufacturers software

it is possible for something that is undesirable in theory to be unnoticeable in practice or by accident or design help the expression of a particular image, reality gap between theory and practice relevant here

It might be different standards for fine art or personal photography and commercial, though that depends on the client who miay have different standards

I tend to rely on comparison between finished products done different ways, not always my choice so practice not theory but the hard work in the theory by others appreciated

1 Like

worth mentioning in this thread that later fuji’s can create 8 and 16 bit TIFF’s in camera