culling with prerendering (slow laptop)

micharambou · August 19, 2024, 6:35pm

Is this maybe related to

github.com/darktable-org/darktable

DT 4.8.1 - full preview lighttable mode is sometimes slow (forces a full pixelpipe cycle)

opened 06:49PM - 02 Aug 24 UTC

closed 11:02AM - 05 Aug 24 UTC

vancouverbluesea

reproduce: confirmed scope: UI scope: performance bug: pending

### Describe the bug This maybe related to https://github.com/darktable-org/dar…ktable/issues/15975 When lightable displays non processed images often it does not use the generated cache but re renders the image from the RAW file ### Steps to reproduce Best way to see it (in my opinion) if the files are stored on a (slower NAS) 1. Have 2 sets of folders - one is non processed images the other is processed images. 2. Ensure all the cache is generated - for me it is 2k monitor - level 5 cache 3. Best is if the sets are bigger - like 500+ (I am currently working with 10k) 4. Switch to full screen and use the mouse wheel to navigate to the next image (fast) ### Expected behavior DT should always read the files from the mipmap cache (because all the cache is generated) ### Logfile | Screenshot | Screencast _No response_ ### Commit _No response_ ### Where did you obtain darktable from? downloaded from www.darktable.org ### darktable version 4.8.1 ### What OS are you using? Linux ### What is the version of your OS? PopOS 24.04 ### Describe your system? OS: Pop!_OS 22.04 LTS x86_64 Host: Serval WS serw12 ``` Kernel: 6.9.3-76060903-generic Uptime: 21 mins Packages: 3189 (dpkg), 86 (flatpak) Shell: bash 5.1.16 Resolution: 2560x1440 DE: GNOME WM: Mutter WM Theme: Pop Theme: Pop [GTK2/3] Icons: Pop [GTK2/3] Terminal: gnome-terminal CPU: AMD Ryzen 9 3900 (24) @ 3.100GHz GPU: NVIDIA GeForce RTX 2070 Mobile / Max-Q Refresh Memory: 4342MiB / 64194MiB ``` ### Are you using OpenCL GPU in darktable? None ### If yes, what is the GPU card and driver? Nvidia RTX 2070 Mobile / Max-Q Refresh ### Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip 1. The issue is expected to have been fixed by https://github.com/darktable-org/darktable/issues/15975 but I am still observing it 2. The issue was first observed in 4.6.0 build of DT (not before) 3. I tried to obtain help in the forums under https://discuss.pixls.us/t/darktable-troubleshooting-cache-speed/44858 4. I tried to troubleshoot on my own. When I run DT with `flatpak run org.darktable.Darktable -d lighttable -d pipe` When the images are not processed often I see lines like this ``` 371.5749 dt_view_image_get_surface id 218556, dots 2490x1244 -> mip 2560x1600, found 1066x1600 371.5856 got surface 829x1244 371.5921 done in 0.0040 sec 371.6012 dt_view_image_get_surface id 218557, dots 2490x1244 -> mip 2560x1600, found 1066x1600 371.6144 got surface 829x1244 371.9759 done in 0.0059 sec 371.9868 dt_view_image_get_surface id 218558, dots 2490x1244 -> mip 2560x1600, found 1066x1600 372.0003 got surface 829x1244 372.0058 done in 0.0039 sec 372.0170 done in 0.0094 sec 372.0279 done in 0.0094 sec 373.2449 modify roi OUT [thumbnail] rawprepare ( 0/ 0) 7168x5120 scale=1.0000 --> ( 0/ 0) 7032x4688 scale=1.0000 ID=218561 373.2449 modify roi OUT [thumbnail] flip ( 0/ 0) 7032x4688 scale=1.0000 --> ( 0/ 0) 4688x7032 scale=1.0000 ID=218561 373.2467 dt_view_image_get_surface id 218561, dots 2490x1244 -> mip 2560x1600, found 1066x1600 373.2584 got surface 829x1244 373.2637 done in 0.0039 sec 373.2868 done in 0.0172 sec 373.3096 done in 0.0162 sec 373.3310 done in 0.0184 sec 373.3559 done in 0.0235 sec 373.3789 done in 0.0214 sec 373.4047 done in 0.0240 sec 373.4260 dt_view_image_get_surface id 218568, dots 2490x1244 -> mip 2560x1600, found 2400x1600 373.4497 got surface 1866x1244 ``` Another symptom is a significant network traffic (because I am using a slower NAS) https://discuss.pixls.us/t/darktable-troubleshooting-cache-speed/44858/5?u=vbs (the second screenshot shows about 35 MB/s when using non processed files) If my files are processed and I perform the same actions - I don't have the traffic and the screen refreshes instantly Also - when I run `flatpak run org.darktable.Darktable -d lighttable -d pipe` I am only seeing lines like this ``` 367.5857 done in 0.0045 sec 367.6020 dt_view_image_get_surface id 218515, dots 2490x1244 -> mip 2560x1600, found 2400x1600 367.6214 got surface 1866x1244 367.6412 done in 0.0074 sec 367.6595 dt_view_image_get_surface id 218516, dots 2490x1244 -> mip 2560x1600, found 2400x1600 367.6792 got surface 1866x1244 368.0311 done in 0.0056 sec 368.0478 dt_view_image_get_surface id 218517, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.0682 got surface 1866x1244 368.0747 done in 0.0045 sec 368.0945 done in 0.0177 sec 368.1118 dt_view_image_get_surface id 218519, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.1348 got surface 1866x1244 368.1413 done in 0.0044 sec 368.1591 dt_view_image_get_surface id 218520, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.1838 got surface 1866x1244 368.1906 done in 0.0046 sec 368.2081 dt_view_image_get_surface id 218521, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.2274 got surface 1866x1244 368.2341 done in 0.0046 sec 368.2509 dt_view_image_get_surface id 218522, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.2715 got surface 1866x1244 368.6070 done in 0.0057 sec 368.6258 dt_view_image_get_surface id 218523, dots 2490x1244 -> mip 2560x1600, found 2400x1600 368.6498 got surface 1866x1244 368.6565 done in 0.0046 sec ``` I tried different settings for the thumbnails ![image](https://github.com/user-attachments/assets/6c36bdb6-5eb3-4d53-92b6-0ba979c92362) vs ![image](https://github.com/user-attachments/assets/316dc12a-0615-4497-929a-e4e2b026663e) There was no difference in the performance for me. This is in terms - the system always appears to choke with non processed images. Possible workaround for me is to apply a basic style to force DT to process the files (even with a small change). Not ideal but the best that comes to mind I tried 1 up (instead of full screen) for some reason - I do not observe issues when I am displaying the files in 1 up. However - I cannot use it as a workaround because I am missing the selecting of the files (what to be processed and what to be discarded). Thank you in advance. I could not test earlier as I was off for 1m. (when 4.8.0 was released) and when the previous change was done - I was unaware of the nightly appimage builds. I am sorry I was not much of a help with this issue.

?