This is something that I am currently doing using the multiprocessing module in Python.
Specifically:
from multiprocessing.pool import ThreadPool
I set my thread count to 32 workers and can saturate the cores.
For my process I am converting RAW to .tiff format specifically to apply a .dcp for color correction.
I found that largest times sucks were the demosaicing and image compression.
When exporting to tiff via CLI one of the missing features from RT is the ability to choose LZW compression instead of ZIP. LZW compression is only available via the UI and not via a CLI parameter.
Here are some of the timings I documented:
On single images:
Zip compression, AMaZE, Sharpening = 36 seconds
Uncompressed, AMaZE, Sharpening = 16.6 seconds
Uncompressed, AMaZE, No sharpening, minimal pp3 settings = 6.1 seconds
Uncompressed, No demosaicing (i had to hack the pp3 to achieve this), minimal pp3 settings = 1.9 seconds
This will help you with utilizing the multiprocessing module and apply_async: