opened 06:49PM - 02 Aug 24 UTC
closed 11:02AM - 05 Aug 24 UTC
reproduce: confirmed
scope: UI
scope: performance
bug: pending
### Describe the bug
This maybe related to https://github.com/darktable-org/dar…ktable/issues/15975
When lightable displays non processed images often it does not use the generated cache but re renders the image from the RAW file
### Steps to reproduce
Best way to see it (in my opinion) if the files are stored on a (slower NAS)
1. Have 2 sets of folders - one is non processed images the other is processed images.
2. Ensure all the cache is generated - for me it is 2k monitor - level 5 cache
3. Best is if the sets are bigger - like 500+ (I am currently working with 10k)
4. Switch to full screen and use the mouse wheel to navigate to the next image (fast)
### Expected behavior
DT should always read the files from the mipmap cache (because all the cache is generated)
### Logfile | Screenshot | Screencast
_No response_
### Commit
_No response_
### Where did you obtain darktable from?
downloaded from www.darktable.org
### darktable version
4.8.1
### What OS are you using?
Linux
### What is the version of your OS?
PopOS 24.04
### Describe your system?
OS: Pop!_OS 22.04 LTS x86_64
Host: Serval WS serw12
```
Kernel: 6.9.3-76060903-generic
Uptime: 21 mins
Packages: 3189 (dpkg), 86 (flatpak)
Shell: bash 5.1.16
Resolution: 2560x1440
DE: GNOME
WM: Mutter
WM Theme: Pop
Theme: Pop [GTK2/3]
Icons: Pop [GTK2/3]
Terminal: gnome-terminal
CPU: AMD Ryzen 9 3900 (24) @ 3.100GHz
GPU: NVIDIA GeForce RTX 2070 Mobile / Max-Q Refresh
Memory: 4342MiB / 64194MiB
```
### Are you using OpenCL GPU in darktable?
None
### If yes, what is the GPU card and driver?
Nvidia RTX 2070 Mobile / Max-Q Refresh
### Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip
1. The issue is expected to have been fixed by https://github.com/darktable-org/darktable/issues/15975 but I am still observing it
2. The issue was first observed in 4.6.0 build of DT (not before)
3. I tried to obtain help in the forums under https://discuss.pixls.us/t/darktable-troubleshooting-cache-speed/44858
4. I tried to troubleshoot on my own.
When I run DT with
`flatpak run org.darktable.Darktable -d lighttable -d pipe`
When the images are not processed often I see lines like this
```
371.5749 dt_view_image_get_surface id 218556, dots 2490x1244 -> mip 2560x1600, found 1066x1600
371.5856 got surface 829x1244
371.5921 done in 0.0040 sec
371.6012 dt_view_image_get_surface id 218557, dots 2490x1244 -> mip 2560x1600, found 1066x1600
371.6144 got surface 829x1244
371.9759 done in 0.0059 sec
371.9868 dt_view_image_get_surface id 218558, dots 2490x1244 -> mip 2560x1600, found 1066x1600
372.0003 got surface 829x1244
372.0058 done in 0.0039 sec
372.0170 done in 0.0094 sec
372.0279 done in 0.0094 sec
373.2449 modify roi OUT [thumbnail] rawprepare ( 0/ 0) 7168x5120 scale=1.0000 --> ( 0/ 0) 7032x4688 scale=1.0000 ID=218561
373.2449 modify roi OUT [thumbnail] flip ( 0/ 0) 7032x4688 scale=1.0000 --> ( 0/ 0) 4688x7032 scale=1.0000 ID=218561
373.2467 dt_view_image_get_surface id 218561, dots 2490x1244 -> mip 2560x1600, found 1066x1600
373.2584 got surface 829x1244
373.2637 done in 0.0039 sec
373.2868 done in 0.0172 sec
373.3096 done in 0.0162 sec
373.3310 done in 0.0184 sec
373.3559 done in 0.0235 sec
373.3789 done in 0.0214 sec
373.4047 done in 0.0240 sec
373.4260 dt_view_image_get_surface id 218568, dots 2490x1244 -> mip 2560x1600, found 2400x1600
373.4497 got surface 1866x1244
```
Another symptom is a significant network traffic (because I am using a slower NAS)
https://discuss.pixls.us/t/darktable-troubleshooting-cache-speed/44858/5?u=vbs
(the second screenshot shows about 35 MB/s when using non processed files)
If my files are processed and I perform the same actions - I don't have the traffic and the screen refreshes instantly
Also - when I run
`flatpak run org.darktable.Darktable -d lighttable -d pipe`
I am only seeing lines like this
```
367.5857 done in 0.0045 sec
367.6020 dt_view_image_get_surface id 218515, dots 2490x1244 -> mip 2560x1600, found 2400x1600
367.6214 got surface 1866x1244
367.6412 done in 0.0074 sec
367.6595 dt_view_image_get_surface id 218516, dots 2490x1244 -> mip 2560x1600, found 2400x1600
367.6792 got surface 1866x1244
368.0311 done in 0.0056 sec
368.0478 dt_view_image_get_surface id 218517, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.0682 got surface 1866x1244
368.0747 done in 0.0045 sec
368.0945 done in 0.0177 sec
368.1118 dt_view_image_get_surface id 218519, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.1348 got surface 1866x1244
368.1413 done in 0.0044 sec
368.1591 dt_view_image_get_surface id 218520, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.1838 got surface 1866x1244
368.1906 done in 0.0046 sec
368.2081 dt_view_image_get_surface id 218521, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.2274 got surface 1866x1244
368.2341 done in 0.0046 sec
368.2509 dt_view_image_get_surface id 218522, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.2715 got surface 1866x1244
368.6070 done in 0.0057 sec
368.6258 dt_view_image_get_surface id 218523, dots 2490x1244 -> mip 2560x1600, found 2400x1600
368.6498 got surface 1866x1244
368.6565 done in 0.0046 sec
```
I tried different settings for the thumbnails

vs

There was no difference in the performance for me. This is in terms - the system always appears to choke with non processed images.
Possible workaround for me is to apply a basic style to force DT to process the files (even with a small change). Not ideal but the best that comes to mind
I tried 1 up (instead of full screen) for some reason - I do not observe issues when I am displaying the files in 1 up. However - I cannot use it as a workaround because I am missing the selecting of the files (what to be processed and what to be discarded).
Thank you in advance.
I could not test earlier as I was off for 1m. (when 4.8.0 was released) and when the previous change was done - I was unaware of the nightly appimage builds.
I am sorry I was not much of a help with this issue.