The Panasonic S1H has what looks like a brilliant feature for photographers… the ability to monitor through a 3D LUT. However, when I take a photo with the V-Log profile, the raw file (.RW2) that comes out looks nothing like a log file (using the jpg as a reference, which is encoded with the V-Log profile), through any software (capture one, raw therapee, exposure, Davinci Resolve [converted to DNG]). I also have a Varicam LT and when I shoot raw cDNG files externally to an Atomos Shogun Inferno I can transform the file, through the raw tab, into a V-log / v-gamut encoded image (they match 100% with the internally recorded files, meaning the raw conversion is is a mathematically correct version of V-log and not some chimped fake log file, and meaning my camera viewing LUTS look correct when applied in Davinci Resolve). I’ve tried a similar process with an .RW2 file converted to DNG and Davinci resolve doesn’t give the v-log/v-gamut option in the raw tab. If I try to use the color space transform the results aren’t correct, no matter what I try.
Basically, what I’m trying to do is create a raw file that converts to V-Log / V-Gamut, stripping it of any DCP / built in profile and then have the ability to use a 3D LUT, after any processing. I believe this is a scene referred grading style. The order would be camera linear raw → V-Log/V-Gamut transform → processing (white balance, curves, color balance, etc.) → 3D look LUT into whatever applicable output color space / gamma.
Does anybody know if this is possible and/or how to do something like this? Thank you!
Welcome to discuss.pixls.us! A good place for such discussions, where they garner the attention of weird people like me…
If the Panasonic S1H behaves like its still-camera siblings, the .RW2 file should contain a linear pre-demosaic image array, before any non-linear transform including V-Log. Curious sot that I am, went and got a SH1 .RW2 file from dpreview’s studio scene comparison tool and opened it in my hack software; the unmodified raw data indeed has “linear” characteristics. I also inspected the metadata using exiftool, but I couldn’t find any reference to a log profile of any sort. I did find two MakerNote tags for “Baby” information, BabyName and BabyAge; nothing to do with your inquiry, but, well, interesting…
If you don’t mind, can you post a .RW2 file with the V-Log-transformed embedded JPEG? There might be information contained therein that can provide clues to establishing your workflow…
Hi Glenn, thanks for the info and help! I’ve attached two .RW2 files and their associated jpgs. Both are coming straight from the camera, no processing. They were both shot with the same settings, except Files “1159” were shot with the standard profile and “1160” were shot with V-Log. In other words, I took one photo and then changed the picture profile and shot the next. The jpgs were a simultaneously capture. Hope that makes sense. You will see that the 1160 raw files are around 2.3 stops darker, which I think is some sort of problem with the raw conversion displaying the grey point in the wrong place. But any help is much appreciated. Thanks again
The second raw is named, '20210327_test_0008.RW2; I downloaded that, extracted the jpeg at -JpgFromRaw, and inspected the metadata for both files. Nowhere could I find a tag or value that hinted that the JPEG was log-transformed by V-Log or any other tone curve. The JPEG is definitely log-transformed, compared to the linear raw data; I confirmed this by applying a log curve I had lying around, and the resulting image histogram is almost identical to the extracted JPEG.
To use the V-Log transform in a still-image workflow, I’d surmise you need to have that curve built-in to whatever software you’re using, and you’d have to decide to apply it, no reliance on the .RW2 metadata to tell you to, or how.
Not sure why the files names got mixed up in the upload, but yes the test_0008 file is the V-Log shot file. But, if you import both raw files into a raw editor, the 0008 file is much darker than the 1159 file. It’s also noiser, but has more highlight, which tells me that some sort of different processing is happening to the RAW file. Or that the camera itself is recording it in a different ISO, even though the ISO settings are the same between the files.
That is exactly what I’m looking for, a software that sees the .RW2 (shot with a V-Log Profile) and let’s the user decide what Gamma / Gamut to convert to (i.e. V-Log / V-Gamut). This would allow me to monitor through a 3D LUT in camera and then, in software allow me to apply that same 3D LUT to the V-Log/V-Gamut transformed .RW2 file. It’s such a simple concept and one that Panasonic should be shouting from the rooftops and software developers should be capitalizing on immediately. It’s one of only a few stills cameras (most are Panasonic) on the market with such a unique feature, but it’s the future. Cinema cameras have had this functionality for years now. But as it stands I have to view my shots with stock picture profiles, instead of a fully developed look, in camera. If there’s a way to do this, I would be over the moon!
Also, for reference, here is a .DNG still from my Varicam LT, that was recorded onto my external Atomos Shogun Inferno. This file, if imported into Davinci Resolve, will allow the user to bypass input color space and then in the raw tab switch over to V-Log / V-Gamut. The resulting effect is identical to the internally recorded AVC Intra files recording V-Log.
If there was some way to apply the logic / metadata this file uses onto the .RW2 file, as a possible solution. I tried messing around with Hexfiend, but I’m really getting out of my depth here. Thanks for any and all help.
Yes, it took +2EV in exposure comp to bring the 0008 raw data up to the 1159. And, yep, the same settings: 1/800sec, f0 (?), ISO640 - is there a chance the lens isn’t reporting aperture, and it got changed?
Getting any software developer to incorporate the Panasonic V-Log curve would take some negotiation. I wrote my own software, so I’ve incorporated a HEVC loggamma curve and I use it to do somewhat what you’re after: I apply the loggamma curve to the linear data to spread out the data, then I work on it with a control-point curve that starts with ‘S’ and very often devolves into un-elegance…
A relatively quick path to your objective might be to come up with a V-Log LUT. Assuming the software you use will allow application of multiple LUTs, you could apply the V-Log LUT first, then subsequently your color-grading LUT. If you can apply only one, there may be a (probably hacky) way to combine the log and grading LUTs into one…
So what you’re doing is somewhat similar to what I’ve been doing over the past few weeks, but the other direction, with Sony S-Log2 data.
I’ve been meaning to finish cleaning up some code, but the basic idea (and again, in a week or two I’m hoping to push the hax up to github and write them up) is to implement the S-Log2/3 transfer functions and S-Gamut3/3.cine primaries in Elle Stone’s ICC profile generator at GitHub - ellelstone/elles_icc_profiles: Elle Stone's Well-Behaved ICC Profiles and Code
In my case, I tag Sony S-Log content with the appropriate matching profile, and RawTherapee will then properly linearize it and convert colorspaces. Edit the linearized file to taste (exposure, saturation, vibrance, tone curves)
I can then create an identity HALD CLUT, tag it with the same profile, copy the processing stack from the aforementioned S-Log content that I edited to taste, and boom - I’ve got an S-Log to whatever haldclut.
In your case, you’d do the opposite - load the RAW in RT, and set the output profile to an ICC profile that matches V-Log. But that’ll only go one frame at a time - e.g. it’s gonna be sloooooow.
I won’t have a chance to look at your DNG that came out of the Varicam, but my guess would be that someone at Panasonic did some trickery to implement some of the more esoteric features of DNG to allow them to store nonlinear demosaiced data (fun fact, demosaiced is usually referred to as “Linear DNG” but it doesn’t HAVE to be linear in the spec!). I MIGHT have a chance to look at your DNG for some of the less “common” tags that would hint at this tonight, but we’re kind of in the home stretch before my procrastination on cleaning before my family arrives for Easter bites me in the a** and the result drives my stress levels through the roof.
Absolutely nothing changed (unless you count the roughly 10 second difference in time of day between pictures, it was a sunny day too, haha). My lenses are all manual aperture and consistency was paramount for this test to mean anything.
The LUT transform concept might work, but the problem with combining would mean I’m either editing in some unknown space or editing after the lut, which isn’t really what I want. I would prefer to work in the V-Log/V-Gamut space, before the lut (which is based around a Kodak print film emulation), it’s similar to how a film developer would time film and then print.
Your still image data in the .RW2 file is linear, so applying the V-Log LUT would be putting the image in V-Log space. Then, you look-LUT, curve, or do whatever to color-grade the image.
Thank you so much for the .txt LUT file. But how would I go about using this? I have the program Lattice and tried copy and pasting the .CSV of your text file into an expression, which it seems to accept, but it crashes every time I try to apply it. I currently only have access to OSX.
Hi Andy, Thanks for all the info. I’m gonna have to read about a lot of what you are talking about in more depth. I’m familiar with most of the terms, but haven’t used some the processes myself. I’ve only recently played around with RawTherapee. Needless to say, I’m a little confused as to the workflow, you are suggesting. What is your end goal with the S-Log method you are employing? Sorry, if I’m a total noob. But I would like to learn.
My whole goal is to make my raw photo workflow similar to how my raw video workflow functions in Davinci Resolve. The cDNG file works as I would like in Davinci Resolve. Resolve properly converts the DNGs into V-Log/V-Gamut (which is confirmed by its match with the internally recorded compressed V-Log files). But if you bring the that same cDNG file into any photo editor, it has some generic curve applied. I’m trying to bypass that curve and take the linear raw data and transform it into a mathematically correct V-Log / V-Gamut image. Will RawTherapee allow for this or should I use something else? I have Lattice, which allows for expressions and matrix inputs, but not sure how to implement. Glenn has been super helpful with he .CSV. Thank you both for all of your help. Still learning this as I go.
So you have a mix of “good timing” and “bad timing” here.
Good timing - I’ve been just returning to going through a large backlog of Sony recordings to take another crack at postprocessing/grading/editing them. As part of this effort, I started working on the workflow described above, which was intended to allow me to use RawTherapee to color grade my S-Log clips. Part of the motivation for this was that I’ve been on the fence about Resolve Studio for a loooong time for a variety of reasons, so the intent was to do something similar to what I’ve done in the past - generate LUT, grade in ffmpeg, do cuts/transitions in kdenlive. But kdenlive’s support for >8 bit depth is nonexistence which was a problem
Good timing 2 - Found out that Blackmagic is bundling Resolve Studio licenses with the Speed Editor keyboard. There’s no way I’d pay $300 for the Speed Editor on its own, but when it’s basically “free input device with Resolve license” or “free Resolve license with input device” - I gave in. So I’m probably one of a handful of people here that now uses Resolve. But, um… For 1 day at this point!
Bad timing - Easter family visit
What I can tell you is that the DNG you have APPEARS like it may be set up to take V-Log nonlinear data and allow a RAW converter to understand it as DNG. There’s an esoteric variant of DNG that people frequently call “Linear DNG” which is basically “Demosaiced but still in camera color space and still linear” (USUALLY!). This can be a misnomer because DNG actually supports nonlinear data - that’s the role of the Linearization Table tag.
Your DNG has a Linearization Table, and while I can’t plot it in detail, the Linearization Table has, at the very least, a black level offset which hints strongly that it’s intended to linearize nonlinear data. So my suspicion ( need to confirm) is that the LinearizationTable and ColorMatrix in that DNG are designed to linearize and transform data that was saved as V-Log.
But your other camera is saving a more “pure” RAW format - Bayer-mosaiced, linear, and in camera-native colorspace not “close to camera native but standardized for color matching” like many of the log formats are.
An important thing - ICC profiles go both ways. For input - from nonlinear data to linear. For output - from linear to nonlinear. I’ve been generating ICC profiles for linearization of Sony S-Log2+SGamut3 to RT’s working colorspace (selectable). What you appear to be looking for is: raw file->working colorspace->V-Log - and this can be achieved by using a wide gamut working space in RT, and an output ICC profile that implements V-Log.
It shouldn’t take TOO long to add V-Log to what I’ve been futzing with for the past few weeks, but again - I am at this point almost entirely out of any free time until after the parents leave. They don’t arrive until Saturday but I need to finish cleaning my place or my ears are going to hurt when mom arrives.
As to deltas in raw exposure between “not V-Log” and “V-Log” - I’m a Sony user, but I believe some of the S-Log concepts apply here.
Camera ISO ratings have no direct connection to raw sensor values. They are based on the JPEG generated. The standard does not mandate a tone curve - so if you apply an S-Curve prior to sRGB transform, that can shift the value - and if your data isn’t even sRGB in the first place, but some completely different TRC - that shifts the ISO value even if the sensor peak/clip point (or gain configuration) is the same.
This is, for example, why the minimum ISO for Sony S-Log2 is 800 for a camera that has a minimum ISO of 100. If you take all of the formulas, and map sensor clip to the highest possible code value - then the code value of 128 (for 8-bit) aka “middle grey” will not be at clip*0.18 (like it is for sRGB/rec709) for S-Log2. It’s about 2.4EV down. As to the other 0.6EV - I need to do some more digging for Sony as to whether this is due to Sony applying an S-Curve to “normal” JPEGs, bumping the sensor gain a bit from minimum, or a mix thereof. But it explains why the RAW data for your V-Log shot is underexposed compared to not-V-Log for the same ISO rating. It’s kind of similar to how some cameras have a “DRO” mode that underexposes the raw to preserve highlights and then tonemaps the JPEG back up.
So your RAW exposures shifting for a fixed ISO depending on whether you’ve enabled V-Log or not makes perfect sense - it’s the same for Sony S-Log modes and, again, is because the CIPA standard for the digital equivalent of ISO is based on output image (usually JPEG) behaviors and where the “middle” code value lands, not in any way based on how the sensor’s raw data behaves.
Andy, you know so much more than I do! Thanks for all of this info, I keep rereading trying to understand all of it. I just know what I would like to do, but really digging deep (for me) to find answers and a possible solution.
I’m not sure if this: “raw file->working colorspace->V-Log” is what I’m looking for it’s more "raw file->V-Log->Color Grade->Custom V-Log to (to applicable color space) 3D LUT ->Output to Tiff/Jpg etc. Maybe your saying the same thing, but the “working colorspace” is throwing me. I thought working color space is V-Log. The ideal scenario would be able to have a color profile / LUT at the input that transforms the .RW2 file into V-Log, then I can edit the white balance, colors, contrast, etc., and then after those steps the 3D LUT would be applied to transform it into the final printed look in whatever color space.
Basically, the above workflow would mimic what I’m doing very simply in Davinci Resolve. DNG->Converted to V-Log/V-Gamut in raw tab (turn off input color space), nodes for grade, final node is a Custom 3D LUT with transform from V-Log into rec709.
Any time you have to help or suggest is much appreciated. I’m in no hurry. Happy Easter and have a great weekend.
BTW, since you use Davinci Resolve you can with the above method see what an actual raw converted to V-Log is suppose to look like, with my uploaded DNG.
Danged discuss.pixls.us, doesn’t like .cube files, so download this one and change the file extension to .cube … Never have made one of these before, not sure it’ll work. If it doesn’t, let me know and I’ll debug it.
Thank you for trying to help me Glenn! I tried your file and the results are similar to what I attained in Lattice with a linear to V-Log transform. Basically, this results in a file that is more log like, but it definitely isn’t v-log (i.e. it doesn’t match the reference jpg). It’s actually way less contrasty, less saturated and the colors are pretty different.
Andy mentioned the “Linearization Table” inside of my DNG file’s exif data. I’m wondering if there’s some way to copy that into the exif data on the .RW2 file. Would this possibly allow Davinci Resolve’s Camera Raw tab to see the V-log profile, when it reads the metadata? A LUT could be made as another option (it would be more flexible with the choice of program), but the transform doesn’t appear to be as simple as Linear to V-Log, there’s something else happening.
