This almost sounds as if you have network attached storage or very slow external hard drive that RawTherapee somehow needs to access, possibly through some slow Windows subprocess… Or maybe your folder holds thousands of files? Or maybe your thumbnail cache is huge, or located on a relatively slow hard drive?
A lot of options, hard to diagnose…