CinemaDNG workflow for Blackmagic or Sigma FP cameras in Darktable

Software darktable
21

Welcome, and thanks for the Sigma samples (@LebedevRI needs to vet those before they get accepted into RPU). It’s both interesting (and disappointing) how different vendors implement CinemaDNG, and the fragmentation and problems that leads to.

The Sigma 2K/4K files are different to Blackmagic as they are (thankfully!) uncompressed - this implies bit packing/unpacking which takes some extra work for bit depths other than 8 and 16, which is probably why dnglab fails on 10 and 12. This is weird given some hints of it in the code, so feel free to report that there. @cytrinox

The 6K file is compressed, but I assume it was taken in stills/photography mode, and is therefore of the compliant stills DNG (not CinemaDNG sequence) kind - the compression scheme is not customized in any way, and thus readily supported by dt and vkdt. (We already have the Sigma fp stills sample on RPU.)

22

cool, thanks for the samples! i’m not very familiar with rust/the rawler code, but indeed the 10/12 bit decoding seems to be implemented, but disabled. if i mess with the code like so

$ git diff
diff --git a/rawler/src/decoders/dng.rs b/rawler/src/decoders/dng.rs
index cabbcd97..5106077e 100644
--- a/rawler/src/decoders/dng.rs
+++ b/rawler/src/decoders/dng.rs
@@ -345,8 +347,8 @@ impl<'a> DngDecoder<'a> {
       (Endian::Big, 16) => Ok(decode_16be(&src, width, height, dummy)),
       (Endian::Little, 16) => Ok(decode_16le(&src, width, height, dummy)),
       (Endian::Big, 12) => Ok(decode_12be(&src, width, height, dummy)),
-      //(Endian::Little, 12) => Ok(decode_12le(&src, width, height, dummy)), Not supported by DNG spec
-      //(Endian::Little, 10) => Ok(decode_10le(&src, width, height, dummy)), // Not supported by DNG spec
+      (Endian::Little, 12) => Ok(decode_12le(&src, width, height, dummy)), // Not supported by DNG spec
+      (Endian::Little, 10) => Ok(decode_10le(&src, width, height, dummy)), // Not supported by DNG spec
       // TODO: implement 10 bit BE
       (_, 8) if raw.has_entry(TiffCommonTag::Linearization) => {
         let linearization = fetch_tiff_tag!(self.tiff, TiffCommonTag::Linearization);

i can load the files in vkdt. they are then of course completely garbled because some metadata doesn’t seem right (bayer pattern, colour matrix, …) but at least the decoding works. not sure the endianness is actually correct, possible that the garbled colours are a result of byte-swap too.

1 Like
23

Thanks, verified the samples. All of them load just fine in dt.
There is no 6k sample though.

24

We already have an earlier one - different content though.

25

That’s my point - if at all possible, it’s best to have all samples of a set to be matching.

1 Like
26

I did try to upload the 6k frame shot in digital stills mode to raw.pixls.us but sadly I am consistently getting an error (just tried again and it’s the same thing):

raw.pixls.us is currently unable to handle this request. HTTP ERROR 500

Maybe because there is already a SIGMA fp sample uploaded? It’s available in the google drive link I shared though if it helps.

27

@andabata hi! did RPU run out of disk space?

Yep, just tried, same here.

28

/dev/sdb1 2.2T 655G 1.4T 32% /storage

It should work, unless samples are getting real large :slight_smile:

29

Should be fixed, it now gives error that the sample already exists.

30

@andabata thank you!

@jedsmith now, could you please contribute the (new, full) entire sample set for that camera please? :slight_smile:

31

Trying to upload SDIM4591_6k_14bit.DNG from my google drive link to raw.pixls.us, I now get a more informative error message (thank you for fixing that!):

The file either already exists, the CC0/modified checkboxes weren’t checked, or you forgot to choose a file. Please go back and try again.

I assume that error message is showing because another sample from the Sigma fp already exists, so I can’t upload a duplicate. The existing file is here I believe: SDIM0094.DNG

32

I’m guessing you’ve uploaded SDIM4591.DNG? Thank you!

33

Oh nice, it looks like it went through despite the error message!

34

Hello again, I’ve made some test shots in CinemaDNG 3:1 and 1:1 from a Blackmagic Micro Cinema Camera (BMMCC) and struggling to upload them to the raw pixls database.

EDIT: Looks like they might have uploaded. I can’t see them in the database and the compressed (3:1) file didn’t get any comments or anything filled in.

35

They have to be approved before they show up, I believe.

1 Like
36

I only see one pending upload for Micro Cinema Camera, CAM1_2000-01-01_1659_C0002_000100.dng, so i guess they didn’t upload after all?

37

I’ve also just uploaded a 3:1 compressed DNG that I took yesterday. It looks like it went through.

38

@independent Ok, i guess i’ve found that upload too, verified both uploads, thank you for contributing the samples!

1 Like
39

Reading this: Cinema Raw : Shooting and Color Grading with the Ikonoskop, Digital Bolex, and Blackmagic Cinema Cameras by Kurt Lancaster.

There’s a pretty interesting quote in it from the early days of BMPCC.

Because the Pocket camera records to an SD card, filmmaker Marco Solorio of OneRiver media wanted to know more about how Blackmagic Design could create a raw camera in such a small package. He interviewed Grant Petty, founder and CEO of Blackmagic Design at the industry trade show, NAB, in Las Vegas, April 2013. Petty explained how they were able to get raw CinemaDNG onto an SD card at 24 frames per second, but ran into issues at 30 fps. They really wanted to do raw, so they went back to the CinemaDNG spec sheet. They discovered that not only was there uncompressed raw that they put into their 2.5K Cinema camera, but there were two compression types for 12-bit raw CinemaDNG. Petty explains:
There’s a lossless compression (which is about 2.5:1 to 1.5:1—that’s not very much compression—it’s mathematically clean, It’s perfect). And then there’s a slightly lossy version, which is like a variation of jpg—it’s visually lossless; it’s very clean—and of course you’re still getting 12 bit raw, so it’s awesome. … So we’ve decided to Implement those standards. Blackmagic Design had actually developed their own 3:1 compression codec, but they decided to stick with the open standard of CinemaDNG. The Pocket Cinema Camera (BMPC) contains the CinemaDNG lossless compression, while they added the “slightly lossy” DNG codec to their 4K camera in order to keep the data rate down. Petty claims that, “[m]ost people are going to find that you cannot tell the difference” between the compressed CinemaDNG and the uncompressed. Furthermore, Blackmagic is also developing a QuickTime wrapper for the CinemaDNG format. “What we want to do,” Petty says to Solorio, “is to save the DNG into the QuickTime movie. That would potentially let us decode and read the files. We’ll make it a QuickTime movie, but it’s actually CinemaDNG inside—so that would be much more readable by Final Cut Pro and other applications like Premiere Pro.”

40

Thing is, there is no such thing in the CinemaDNG spec sheet as far as 12-bit JPEG extensions, nor in the original DNG spec sheet.

It’s a proprietary Blackmagic thing in the description near the bottom of CinemaDNG - Wikipedia

Of course the interesting thing is that now DNG 1.7 supports JPEG XL, which I think is going to be the killer app for JXL.