Mean/Median Stack of multiple dark frames before demosaicing.

But then the darkframe subtraction also would not be applied.

ERROR, insufficient caffeine at this time.

Good call.

My latest PlayRaw NEF is from a camera with metadata black > 0, Nikon Z6=1008:
https://glenn.pulpitrock.net/Individual_Pictures/DSZ_4168.NEF

I also need a dark frameā€¦

Care about shutter speed, ISO, etc?

shutter speed and ISO should be the same as the other image. Lens and aperture should not matter in this case.

Hereā€™s one, 1/3sec, ISO 400, same as DSZ_4168.NEF:
Lens cap on, dark room, camera wrapped in a black towel to avoid leakageā€¦

https://glenn.pulpitrock.net/Individual_Pictures/DSZ_4437.NEF
This file is licensed: Creative Commons, By-Attribution, Share-Alike.

I call it ā€œBlack Cat Eating Licorice in a Coal Mineā€. As a PlayRaw, Iā€™d be interested to see what folk can do with itā€¦ :smile:

3 Likes

About dark frame subtraction in RT:

1 Like

Love thatā€¦

OK, here are some tests of the issue. Linked below are three raw files, from or derived from the Blackmagic Pocket Cinema Camera OG. According to the metadata, the black level on them is supposed to be 256.

The raw files include a raw file with image content but exposed near the noise floor, and thus has prominent hot pixels and amp glow, named ā€œImage in need of dark frame subtraction.dngā€. Then there is an authentic BMPCC dark frame, labeled ā€œAuthentic camera dark frame.dngā€. Then there is a fake dark frame with less random noise, created following the advice of @entropy512 as seen earlier in the thread, with the metadata imported from a real BMPCC raw file, including the black and white levels, and all relevant raw interpretation metadata.

Also included is the output of Rawtherapee with various permutations of dark frame subtraction. One with no dark frame subtraction and two with the real dark frame, and two with the kludge dark frame with consistent black levels in metadata. Each darkframe output set has one as is, showing the bug with near complete blackness, despite excessive exposure comp, and the other shows that -512 black point compensation, not so coincidentally, twice the value as found in the metadata, seems to fix the problem.

https://drive.google.com/open?id=1sHx5F0WFi1-jyifwxTDFLx6x1E-IbX6v

Just posted below, some details of the problem with more test raw files. Anywhere else I should post the writeup and Google Drive link, ie, did you open an issue for the bug on github? The github link you posted looks like an optimization, not bug issue, so I wanted to ask your thoughts.

Some of you may squawk that I was using an out of date version of Rawtherapee, so I quickly downloaded and installed 5.7, the latest stable release, straight from https://rawtherapee.com/, and noticed the results were the same with the update, with the pure black result from just the dark frame, and the expected result from -512 black point compensation, so no point in re-export of example jpegs.

1 Like

Update, just added .pp3 profiles in the google drive link with self explanatory titles.

1 Like

Iā€™ll try to take a look later this week.

Positives of vacation: Not distracted by work.

Negatives: Frequent unpredictable family distractions. :slight_smile:

I too would love to come up with a way to pre-average multiple dark frames into a single file for RawTherapee to use. I have been using RawTherapee and Hugin for astrophotography processing with good results. I assume the ā€œhugin_stackerā€ command line utility would work perfect to average dark frames. This tool can take 16 bit tiff files. What Iā€™m not sure about is what settings in RawTherapee to use to convert the dark frame raw files to TIFF before stacking. Obviously we donā€™t want demosaicing turned onā€¦ or pretty much anything elseā€¦ butā€¦ still unsure the best ways to make sure nothing else happens.

None of this would be necessary if there was a proper way to select multiple dark frames in RawTherapee. :slight_smile:

Thanks for remembering to include a license with your PlayRaw. :wink:

Just read the exchange between me and u/Entropy512 on this thread. We pretty much dug up multiple ways of creating a pre-averaged dark frame raw file, which is where the subtraction should be happening ideally, IE before raw processing. Read from post 12 to post 21 on this thread to glean a how to on multiple workflows on how to do it with their pros and cons. The easiest (no cli use needed) but technically inaccurate (good for amp glow but poor for hot pixels) is Kandao Raw+. The best way is to take the camera raw dark frames and average them in Siril, with no demosaicing or processing, and then use makeDNG (havenā€™t tried) or the Imagemagick
fits/tiff to DNG conversion and Exiftool raw metadata cloning from original camera file workflow (successfully made it work).

For an example where I made it work (output files), see https://drive.google.com/open?id=1sHx5F0WFi1-jyifwxTDFLx6x1E-IbX6v

No, Iā€™m presenting hugin_stacker as a possible addition to your list. I just donā€™t know the best way to do the initial raw conversion of the dark frames before stacking. Command line would be best. There should be no demosaicing.

Hi itā€™s been 2 years since the last activity [quote=ā€œWaveluke, post:42, topic:9752ā€]
The best way is to take the camera raw dark frames and average them in Siril, with no demosaicing or processing, and then use makeDNG (havenā€™t tried) or the Imagemagick
fits/tiff to DNG conversion and Exiftool raw metadata cloning from original camera file workflow
[/quote]
is this still the recommended workflow?! I quick-read the latest manual and did keyword searches but I did not see anything. it surprises me there is no super simple way of forced averaging whatever is in a directory.

Itā€™s what I use, still works fine, but yeah would be better for a more automated way, or at the very least, allow Rawtherapee to read greyscale tiffs as undemosaiced raw files in linear colorspace for dark subtraction so the workflow is lighter and there is no need to do the Imagemagick and Exiftool commands