I also discovered that Adobe DNG coverter v. 12.4.0.547 gets an “unable to parse file” error when converting the lossy compressed files, though fine with the uncompressed and lossless compressed BMPCC4k dng files. Updating to v 13.1.0.658, still had the same issue.
up
@Waveluke might you be interested in contributing the missing raw samples from that camera to https://raw.pixls.us/ ? 
Or, is there anyone else with that camera?
up
1 Like
I had uploaded daylight samples that are visible in the RPU apparently much time ago with the 3:1 compression, lossless, and uncompressed variants, a month after the indoor samples, but it thought the 4:1 was a duplicate file despite not showing in the search. Any way, here is a link to the raws on google drive, as I would really like to use these compressed files with Rawtherapee.Blackmagic Pocket 4k camera test dng files – Google Drive
Edit: maybe someone with administrative controls can fix whatever was preventing me from uploading the fourth file of the set (4:1 compression) and take it straight from the google drive link.
1 Like
After a little hiatus, I returned to doing filmmaking stuff, and still found there not seeming to be any fix, based on the Github. Is there anything else I can do to help this get fixed (I only have 101 level programming knowledge).
Hm, no idea why that sample wasn’t marked as verified, i’ve done that now. Thank you!
1 Like
Also, any shots dated for the month of Februrary can be removed, as they were the inferior indoor test, and is a duplicate.
Btw, perhaps the recent LinearizationTable fix helps w/ the earlier saturation issues?
As a semi-unrelated side note: I’ve been looking for examples of their 3:1 and 4:1 lossy DNG compression for quite some time (but I wasn’t at the time you last posted here…), thanks!
(The understanding I have is that they’ve enabled libjpeg’s 12-bit support and used that. Technically 8-bit JPEG is the requirement in the DNG specification and those files aren’t really DNG compliant, but I blame Adobe for not standardizing a high-bitdepth lossy option.)
Would this fix show up in the latest dev version of Rawtherapee?
Not yet AFAICT…
1 Like
Yup. Still under review. Might experiment with seeing if that helps these BMPCC files this weekend.
Do you need color chart raws or pure white frames? Anything else I can do?
Not for now… Main thing is lack of free time on my part. There are a whole bunch of things in RT I want to poke at but haven’t gotten around to. I’ve been recently curious as to the nature of Blackmagic’s lossy DNG compression, since lossy DNG officially does not support more than 8 bits/channel. (I also wonder if BRAW is in any way connected to their lossy DNG approach…)
I think you can build libjpeg for 12 bits, you just have to keep two libraries around (one for 8, one for 12)… Check out also this summary here: CinemaDNG compression modes in slimRAW
Also found this JP4 format description (and demo link within) interesting in how you can rearrange the Bayer channels and then use lossy JPEG on top…
Yup, you can. I don’t remember WHERE it was that I saw a claim that doing that is how BM did their lossy CDNG. 12-bit isn’t a violation of the JPEG standard (but it’s rare as heck, partly due to the difficulty of getting 8 and 12 to coexist in the same software), but the DNG standard pretty strongly implies that 12-bit JPEG is not compliant with the DNG specification. SlimRAW’s discussion does mention 12 bit depth for lossy compression but does not mention that it appears to be 12-bit JPEG.
Interestingly - Support 8-bit & 12-bit JPEGs using the same build · libjpeg-turbo/libjpeg-turbo@7fec507 · GitHub just landed a month ago - WOOT!!!
Blackmagic probably did somehting similar (function namespace mangling) to allow 8 and 12 bit JPEG support to coexist.
I finally had a chance to test the LinearizationTable fix on these, and it does appear to fix the artifacts in 4k_UHD_4-1.dng
I’m honestly kinda scratching my head as to how these even load since I am positive RT isn’t linking against any JPEG library that is built with 12-bit support…
Edit: Weird… The 4:1 and 3:1 files look like they’re using LJ92 lossless, and not typical DCT JPEG, but they are definitely lossy. I’m wondering if they achieved their lossiness by quantizing the input to LJ92 to make things compress better. Even the uncompressed file has a LinearizationTable of a length that should have broken, but didn’t?
I’m now curious to look at the histograms of these.
Edit: Lossy-compressed files have smoother histograms in the region below 1024 than uncompressed/lossless - even weirder.
Edit 2: Also, @kmilos , that JP4 is an interesting find. I had a similar concept going through my head as a potential option for how a camera that was stuck with USB 2.0 (Xphase Pro X2) could save raw data while using the existing compression pipeline. Xphase basically stopped development over a year ago though, so I don’t think that flaw in their pipeline will ever be fixed.
1 Like