Thanks, that makes sense.
Seems I’ve found a strange sweet spot for the lighttable zoom - having 5x3 thumbs on screen gives me the best response speed, with disabled color management expose takes just 0.04s, which is actually quite usable. Anything smaller or larger results in significantly worse delays (between 0.15s and 0.2s).
Also FWIW I’ve found that the faster-lighttable branch actually does have impact on user experience. With the regular build it often happens that I move the mouse to a neigbouring thumbnail and stop there, but the highlight stays on the previous thumb. I have to move the mouse again a bit to update. This does not happen with faster-lighttable.