Lightest OS that RPD will run on?

Given you are coming from Mint, I’d suggest Lubuntu 20.04. It’s pretty bare bones, and thus quick to boot. LXQT gets the job done.

Just be aware that downloading from an Android phone in which you need to specify File Transfer over MTP each time on the phone will very likely not work because the host OS Windows and VirtualBox itself will not be able to agree on who controls the device. If this is not applicable, Rapid Photo Downloader should work just fine.