Resizing after editing in devevlopment version of Rawtherapee 5.8 on Linux Mint 19.3 hangs both RT and Linux Mint

Mint · June 25, 2020, 11:46pm

Hi
I am using RawTherapee_dev_5.8-2178-g6cc9537_20200620.AppImage on Linux Mint 19.3 because this dev. version has a FF correction fix for Huawei DNG images. If I use resizing only, no other editing features, RT works fine and ends the export to Jpeg without drama in a matter of seconds. If I go about the usual editing task first like FF correct, lens profile, WB, noise, sharpening etc. and then go to do a resize as the last thing before export to jpeg - RT either stops in capture sharpening or runs through to ready very very slowly in 4-5 minutes where after RT is freezes (hangs) and the system also either hangs or is almost completely unresponsive. If I try the same workflow using the stable version 5.8 (also appimage) this does not happen and RT runs through it all and saves the jpeg without a hick-up. Seeing I need the dev version for the FF correction is there a possible workaround or would it be best to wait for the next stable version?
Kind regards Mint

afre · June 26, 2020, 3:14am

A good start would be to read How to write useful bug reports - RawPedia and find out as much as you can about your situation. Try to turn on one module at a time and export to see which one contributes to the slow down or hang the most.

Thanatomanic · June 26, 2020, 5:15am

@Mint Could you please share a RAW file + pp3 processing file where this happens? I run Linux Mint in a VM, so I can try out what might be going on.

heckflosse · June 26, 2020, 10:23am

Sounds like you run out of memory. I will push a fix soon, which reduces memory usage of full export when local lab is not enabled, by 24 * width * height byte…

heckflosse · June 26, 2020, 10:46am

Maybe also related

github.com/Beep6581/RawTherapee

optionally link against tcmalloc

opened 07:14AM - 19 Sep 19 UTC

closed 04:42AM - 22 Sep 19 UTC

ff2000

external: RawPedia patch provided

Like some others I am struggling with RT's memory usage. The global variables to… control glibc's memory management didn't improve the situation for me. jemalloc just introduces random crashes (if it starts up at all). agriggios implemented in his fork "art" an integration tcmalloc. Tried it and it really improves things! I ported this to RT. Any interest to support this? Here is the patch for RawTherapee: ``` diff diff --git a/CMakeLists.txt b/CMakeLists.txt index b027ea5f7..b5626512a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -136,6 +136,9 @@ option(TRACE_MYRWMUTEX "Trace custom R/W Mutex (Debug builds only); redirecting option(AUTO_GDK_FLUSH "Use gdk_flush on all gdk_thread_leave other than the GUI thread; set it ON if you experience X Server warning/errors" OFF) #option(TARGET32BIT "Build for 32-bit architecture when ON, otherwise 64-bit. Default is OFF" OFF) +option(ENABLE_TCMALLOC "Use the tcmalloc library if available" OFF) +set(TCMALLOC_LIB_DIR "" CACHE PATH "Custom path for the tcmalloc library") + # Set installation directories: if(WIN32 OR APPLE) if(BUILD_BUNDLE) @@ -564,6 +567,23 @@ int main() return 0; }" LENSFUN_HAS_LOAD_DIRECTORY) +set(TCMALLOC_LIB_DIR) +if(ENABLE_TCMALLOC) + if(TCMALLOC_LIB_DIR) + find_library(TCMALLOC_LIBRARIES tcmalloc PATHS ${TCMALLOC_LIB_DIR} NO_DEFAULT_PATH) + else() + find_library(TCMALLOC_LIBRARIES tcmalloc) + endif() + if(TCMALLOC_LIBRARIES) + message(STATUS "using tcmalloc library in ${TCMALLOC_LIBRARIES}") + else() + set(TCMALLOC_LIBRARIES "" CACHE INTERNAL "" FORCE) + message(STATUS "tcmalloc not found") + endif() +else() + set(TCMALLOC_LIBRARIES "" CACHE INTERNAL "" FORCE) +endif() + add_subdirectory(rtexif) add_subdirectory(rtengine) diff --git a/rtgui/CMakeLists.txt b/rtgui/CMakeLists.txt index bc52b64b0..edc2b8202 100644 --- a/rtgui/CMakeLists.txt +++ b/rtgui/CMakeLists.txt @@ -286,6 +286,7 @@ target_link_libraries(rth rtengine ${ZLIB_LIBRARIES} ${LENSFUN_LIBRARIES} ${RSVG_LIBRARIES} + ${TCMALLOC_LIBRARIES} ) target_link_libraries(rth-cli rtengine @@ -307,6 +308,7 @@ target_link_libraries(rth-cli rtengine ${ZLIB_LIBRARIES} ${LENSFUN_LIBRARIES} ${RSVG_LIBRARIES} + ${TCMALLOC_LIBRARIES} ) # Install executables ```

Mint · June 26, 2020, 12:37pm

Hi I have never placed a bug request here but I will read the instructions from the link. I was not going that way as a first simply because I felt that seeing the freeze only happens in the dev version 2178 and not in the stable 5.8 - it might not be appropriate to file a bug as it is a potential unstable version of 5.8 which was only pushed through in order to hot fix a few things. I have been able to get through the jpeg export/save in the 2178 dev version because of the remark given about checking the modules to see which is responsible for the freeze. I just tried an idea of letting RT 2178 work a bit at a time - so I simply loaded the DNG, did a few changes, unloaded, loaded again and then tried to export/save to jpeg. This time it worked, it took its time but it worked. So perhaps not a solution but a usable workaround for now. On memory - I did also think the system just chocked because it is one thing a program hangs but the whole system. However looking at System Monitor neither the CPU nor Ram was in anyway in trouble. If that actually shows if I am out of memory or not I dont know.
Kind regards Mint

Mint · June 26, 2020, 10:05pm

Yes after a few more runs I think you are right about memory. Even though I can export, albeit it takes a wee bit of time, after 4 DNGs the system chokes. It shouldn’t with 8 GB RAM but it does (I have never ever been able to freeze Linux Mint like this in the past - it feels like my windows 8.1 machine from the good ol days. System monitor showed it had hugged 2.9 GB RAM. On testing I saw that on start up of RT it grabs 250 MB, load one DNG it jumps to 1,6 GB, load a pp3 and edit it goes to 1.8 GB. After a jpeg export it stays at 1.8 GB but loading a new DNG it goes to 2 GB without any pp3 or editing. Looks like it keeps stuff in memory (or I don’t know how to clean up in between images). Kind regards Mint

Mint · June 26, 2020, 10:16pm

and I forgot to add RT peaks at 99% CPU (Intel i7) and 5,8 GB RAM during the export of a jpeg

Mint · June 30, 2020, 9:12pm

Hi It sounds like the exact same thing. I dont understand how I can apply the patch or if the patch had already been implemented in RT? I tried the latest appimage for x64 linux dev-2268 but even though it seem to release RAM between DNGs (without editing) when you edit a DNG and then and export a jpeg it does the same. Freezes the computer and even after the jpeg is formed (takes several minutes) the computer is unresponsive. I need to force close or kill RT to get my computer back.
Any tricks I can try?
Kind regards Mint

Mint · June 30, 2020, 9:24pm

Just an add on - perhaps it helps - but I thought I got my PC back after a kill process because I could write the above reply. However it was not true - it was unresponsive, programs would not open or close etc. I had to reboot.