I've actually considered USB before in years past and discounted it because the speed of USB sticks was very bad (in the context of keeping all your files on one, compiling from one, etc).
Thinking about it again in light of your comment made me realize there very well could be USB 3.0 thumbdrives with suitable speed (based on some Googling around it looks like there are some that are quite fast, but you have to be really careful with brand/model since it appears there is a very wide spread between the performance of different USB3.0 thumbdrives). But, yeah, I suppose using a fast USB3.0 thumbdrive would be another way to achieve this sort of setup.
The fastest USB 3.0 drive I have is an msata card I had left over after I upgraded, which is mounted in an msata SSD to usb converter case. I haven't tried running my normal OS environment off it yet, but overall performance is way better than a typical USB stick, or USB hard drives.
Thinking about it again in light of your comment made me realize there very well could be USB 3.0 thumbdrives with suitable speed (based on some Googling around it looks like there are some that are quite fast, but you have to be really careful with brand/model since it appears there is a very wide spread between the performance of different USB3.0 thumbdrives). But, yeah, I suppose using a fast USB3.0 thumbdrive would be another way to achieve this sort of setup.