So here's where I got... First, I modified the systemd service config for deluged to limit it's memory and swap usage to around 8Gb, which seems to have gone well. It's still happy and no longer eats RAM.
But, it didn't help when it came time to moving large files from the SSD to the HDD at the end of the download. Load spiked to over 200, VMs got very upset, and arcstat suggested ARC usage went well over the 50% mark to 38Gb at one point, accompanied by a sudden increase in swap use by ~8Gb.