Feature request: save as floating-point


(Alberto) #21

@Elle please be aware though that 32-bit float tiffs are still clamped to 0-1; changing that is desirable and hopefully it will be done at some point, but it requires some major refactoring of the internals…


Suitability of custom camera profiles as working profiles
(Elle Stone) #22

I’m posting this just in case it’s relevant. Just now running RawTherapee compiled yesterday from the “dev” branch, this message was printed to the terminal screen:

$ rawtherapee
blockdenom vanishes
blockdenom vanishes

Regarding the “churning” that I mentioned, yesterday while using RawTherapee, after making some adjustments to a raw file and then exporting as a floating point tif, then making some additional adjustments and exporting again under a new file name, and etc for maybe 5 or 6 iterations, my computer froze. After several minutes the computer was still frozen - even the task bar cpu and ram displays were frozen - shutting down individual windows wasn’t possible. So I used the physical “shutdown” button on my computer to shut the computer down.

Since then I’ve avoided doing more than two iterations of “make changes and then export” because at the second export, things start to slow down on my computer - changing windows is slow, etc. I have no idea whether/if/how RT itself might be the problem as coincidence is not necessarily linked by causality. Anyway, when I recompile RT I’ll use “debug”.


(Alberto) #23

@Elle does it happen only if you export as a float tiff?


(Morgan Hardwood) #24

I experienced the same issue when testing something involving saving images to all formats, including 32-bit TIFF. RT ate up all RAM and my swap was almost at 100%. I managed to kill RT from the TTY.


(Morgan Hardwood) #25

@heckflosse told me that “blockdenom vanishes” means that raw auto CA correction was not able to detect correction parameters.


(Elle Stone) #26

Thanks! - sometimes I wish terminal output were a little less cryptic. I guess this means the image in question didn’t exhibit any signs of chromatic aberration that might need correction? Or maybe it means that it seemed like one part of the image needed one set of parameters and another part needed another set of parameters?

@agriggio - I haven’t yet tried exporting at other bit depths. But given what @Morgan_Hardwood reports, maybe the type of exported image isn’t relevant.

My computer has 32GB RAM and 16GB swap, and when my computer froze the taskbar “RAM” monitor did indicate a fair amount of RAM was in use, but it didn’t look anywhere near 100%.

I just pulled the latest RT code and recompiled using “debug”. Next time I start RT I’ll keep a watch on the RAM usage. When starting RawTherapee, are there any command line switches that maybe I should use to get better terminal output, something like using “–verbose” when starting GIMP?


(Alberto) #27

hi,

maybe I am misunderstanding what he wrote, but I still think this is relevant…


(Ingo Weyrich) #28

These blockdenom vanishes are from Emil’s original code and no one ever changed them.
I agree that we should improve that.

Anyway, it means, that raw auto ca correction failed and the image is not ca corrected.

Edit: the blockdenom vanishes also is in darktable version of raw ca correction :wink:


(Morgan Hardwood) #29

Well I did also export 32-bit fp, so my point was that it does seem like this format was the trigger. When I export only 16-bit TIFF, which I usually do, there are no problems.


(Morgan Hardwood) #30

I always read it “blockdemon vanishes” and I find it amusing… reminds me of Archon for the C64.


#31

It looked that way to me too. I read blockdenim the second time. If you change it to blockchain, perhaps you would attract millions of devs :wink:.


(Elle Stone) #32

Hmm, after pulling in today’s code and recompiling, I wasn’t able to trigger the same behavior. I did recompile using “debug” instead of “RelWithDebInfo”, and maybe that made a difference, but I’m not planning to recompile and check any time soon.

The amount of RAM used by RT did continually go up per additional save, regardless of whether saving at 16-bits or 32-bits. But the maximum it reached even saving 32f, even saving 7 or 8 times, was less than 3GB, and the RAM readout on my taskbar never did get high enough to notice.

Oh, wait, is “-DCMAKE_BUILD_TYPE=” case-sensitive? Because I typed “debug”, not “Debug”.


#33

That is why I don’t build anything anymore. It is too much work. All I found regarding your question is https://cmake.org/cmake/help/latest/manual/cmake-language.7.html#variables but it just says that variable names are case-sensitive. No mention of their values. The docs say certain things are case sensitive and other things aren’t so it is a world of confusion.


(Morgan Hardwood) #34

Nope.


(Morgan Hardwood) #35

Regarding Linux, it’s literally a copypasta of two code blocks (one for getting the dependencies and toolchain, one for compiling RT):
http://rawpedia.rawtherapee.com/Linux


(Flössie) #36

Infinite loop confirmed. The trick is to save the f32 TIFF under a new name. One thread then loops in ExifManager::parse() while consuming more and more RAM. I’m digging deeper…


(Flössie) #37

@Elle @Morgan_Hardwood Here’s a fix:

diff --git a/rtexif/rtexif.cc b/rtexif/rtexif.cc
index 47d885e50..b6a60bbd8 100644
--- a/rtexif/rtexif.cc
+++ b/rtexif/rtexif.cc
@@ -3077,7 +3077,7 @@ void ExifManager::parse (bool isRaw, bool skipIgnored)
         root->printAll ();
 #endif
 
-    } while (ifdOffset && !onlyFirst);
+    } while (ifdOffset > 0 && !onlyFirst);
 
     // Security check : if there's at least one root, there must be at least one image.
     // If the following occurs, then image detection above has failed or it's an unsupported file type.

HTH,
Flössie


(Morgan Hardwood) #38

@floessie could you give some more clues as to how to reproduce? I tried saving a 32-bit TIFF, compressed and uncompressed, to /tmp but didn’t notice any leaking memory or strange CPU activity. Does the file browser need to have the output folder open? Does the TIFF need to be compressed? Are any metadata changes necessary?


(Morgan Hardwood) #39

Issue opened:


#40

@agriggio @Morgan_Hardwood @Elle @snibgo Do we need to wait for the engine update to remove the clamping when reading floating point image data ? Scaling the channels seem to make the colors wrong, so maybe having clipped HL is preferable ?