Thanks everyone, really enjoy reading the discussion. Sorry for the lazy untargetted use of the word Linux. My intention was to use a minimum image with no UI but importantly the USB drivers/stack.
I guess the key issue is what the best data rate I could hope for between the VMs.