How to optimize RawTherapee?

chaimav · July 25, 2024, 6:29pm

I just got a new computer, moving from a Windows 10, i3 8100 to a Windows 11, i7 13700, so that is two tiers (i3 vs i7) and 5 generations better of a processor. Both have 16 GB of RAM (yes I know, more would help).
While RT is faster, I was expecting a more dramatic increase in speed, but I still find that it says “processing image” for a bit even if I just pan an image.
My question is are there any settings I should be changing to make it faster and maybe take more advantage of the increase in processor cores and speed?

marter · July 25, 2024, 7:44pm

Windows is running a LOT of services you dont need at this time. So i propose to tune up windows first. Then, the ultimate mode of windows will speed up RT definetively, even on old hardware. With these settings, on my 10 years old PC i never read the message “processing image”.
Hth
Martin

tankist02 · July 25, 2024, 8:03pm

Switch to Linux

chaimav · July 25, 2024, 8:17pm

My CPU is only hitting 30% max when I pan, which is why I was looking at RT and not Windows.

(Also, I didn’t mention that I am referring to images that have many edits.)

tankist02 · July 25, 2024, 10:13pm

It may be insufficient memory and/or a slow disc. Also I heard that on Windows people often run some kind of anti-virus software that may cause general slow down.
It might be useful to play with settings on the Performance tab in RT Preferences.

chaimav · July 25, 2024, 10:28pm

That’s exactly what I was asking for advice on. I am open to recommendations to try.

The main drive on my computer is a 1TB Micron 2400 Micron 2400 NVMe SSD | Micron Technology Inc.

My images are stored a 2TB Crucial MX500 Crucial MX500 2TB 3D NAND SATA 2.5-inch 7mm (with 9.5mm adapter) Internal SSD | CT2000MX500SSD1 | Crucial.com

Which drive does RT use to cache the images, the one its installed on, or the one that contains the image?

HIRAM · July 26, 2024, 5:05am

Does this occur on every image or one in particular, do you have a lot of processing enabled, can you post an example raw and pp3 that causes this problem….? Your cpu should absolutely rip thru images, but 16GB for a fast intel chip is fairly minimal (for image media processing) as you stated.

chaimav · July 26, 2024, 12:13pm

@HIRAM, based on your suggestion I started turning off tools one-by-one to see what the culprit was. It seems that Wavelet Levels > Sharp-mask and Clarity is the biggest offender.
Sharp mask
I then arbitrarily raised the maximum number of threads to 10 and it seemed to speed things up. So now my question is, what is a good value for that? My CPU has 16 cores and 24 threads. https://ark.intel.com/content/www/us/en/ark/products/230490/intel-core-i7-13700-processor-30m-cache-up-to-5-20-ghz.html

chaimav · July 26, 2024, 12:16pm

@marter can you try turning on Wavelet Levels > Sharp-mask and Clarity and see what happens?

HIRAM · July 26, 2024, 12:19pm

Automatic is supposed to work best; needing to change that may be an indicator of a bug or incompatibility. What version of RawTherapee are you testing?

chaimav · July 26, 2024, 12:22pm

I am currently using Dev 5.10-359

HIRAM · July 26, 2024, 12:23pm

Ok: did you build this yourself or are you running a github artifact?

chaimav · July 26, 2024, 12:26pm

I downloaded RawTherapee_dev_win64_release.exe from Release Automated Builds · Beep6581/RawTherapee · GitHub

chaimav · July 26, 2024, 12:29pm

I just tried the official 5.10 from RawTherapee.com and its the same thing.

HIRAM · July 26, 2024, 12:49pm

Those two builds have the following limitation imposed:

github.com

Beep6581/RawTherapee/blob/1f8d0a67908de8278e9e6b8e26d2b41bd81ae5cd/.github/workflows/windows.yml#L77


      
                echo "Setting cache suffix."
                export CACHE_SUFFIX="5-$REF_NAME_FILTERED"
                echo "Cache suffix is '$CACHE_SUFFIX'."
              fi
          
              echo "Running CMake configure."
              cmake \
                -G "Ninja" \
                -DCMAKE_BUILD_TYPE='${{matrix.build_type}}' \
                -DCACHE_NAME_SUFFIX="$CACHE_SUFFIX" \
                -DPROC_TARGET_NUMBER="1" \
                -DLENSFUNDBDIR="share/lensfun" \
                -S . -B build
          
              echo "Recording filtered ref name."
              echo "REF_NAME_FILTERED=$REF_NAME_FILTERED" >> "$(cygpath -u $GITHUB_ENV)"
          
          - name: Build RawTherapee
            run: |
              echo "Running CMake install."
              cmake --build build --target install

the target 1 corresponds to:

github.com

Beep6581/RawTherapee/blob/1f8d0a67908de8278e9e6b8e26d2b41bd81ae5cd/ProcessorTargets.cmake#L4


      
          # We have to create a label variable if we want to display it in AboutThisBuild.txt...
          
          # This first choice should be used for official releases
          set(PROC_TARGET_1_LABEL generic x86 CACHE STRING "Processor-1 label - should be used for official Windows release")
          set(PROC_TARGET_1_FLAGS "-mtune=generic" CACHE STRING "Processor-1 flags")
          
          # This second choice should be used for your own build only
          set(PROC_TARGET_2_LABEL native CACHE STRING "Processor-2 label - use it for your own build")
          
          # The flag is different on x86 and Arm based processors
          if(CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL arm64)
              set(PROC_TARGET_2_FLAGS "-mcpu=native" CACHE STRING "Processor-2 flags")
          elseif(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "ppc|powerpc")
              set(PROC_TARGET_2_FLAGS "-mtune=native" CACHE STRING "Processor-2 flags")

This is a pretty severe limitation, as x86_64 came out way back in \huge2003 !
That means you can’t really expect better performance than what you would see on an AMD Opteron, or an Intel Pentium4. Remember those?

What I do for the macOS build is target the oldest processor supported by the minimum OS version supported in the build. I also build for a dual target, so extentions for a slightly more recent processor are also included, and can be used when on a processor at least as old. Specifically the target I am using is nehalem-westmere. That gives me 2008-2010 vintage extensions (i7 era).

marter · July 26, 2024, 12:53pm

@chaimav i tried on several pics and dont see any dely with these settings. Be aware, i only tried on APSC no FF, could be there would be a difference.

chaimav · July 26, 2024, 1:06pm

LOL, I was actually just testing it on 12mp jpg images from a Pixel 7, which is what I happened to be working on when I posted the issue.

chaimav · July 26, 2024, 1:09pm

@HIRAM, I am not sure I fully follow (my limitation, not yours). Are you saying that the automatic setting thinks I have a 20 year processor? Does this warrant a bug report?

HIRAM · July 26, 2024, 1:17pm

Yes and yes. RawTherapee for windows supports a lot of old versions, hence the generic build for the official releases… I think a faster build could definitely be added to the workflow. I would suggest raising the issue as an Enhancement Request: Fast Windows Build.

chaimav · July 26, 2024, 1:47pm