Wednesday, October 26, 2016

Where to install a hypervisor; USB or HD Raid Array?

When ESXi was first installed I had the choice of where to install it.  On a usb drive or on the harddrive.  I installed the host OS onto my main hard drive raid array, This I've heard might not be best, however I am pretty new to system administration side.  Hopefully below helps explain why this is bad.  What would have been better was to do 1 of 2 things. 
  1. Install ESXi onto a different usb flash drive. One installed into a port usually directly on the motherboard inside the case This port is usually provided by server motherboard manufacturers specifically for this purpose.  ESXi once loaded by the server will run in RAM and won't access the flash drive unless rebooting the entire host after changing host settings.
  2. Install ESXi directly onto the portable flash media you put the iso install file onto.  This is a nifty trick that allows you to basically load an OS onto the same drive you had the install files.  The install files themselves were only around 360 mb, And the lowest usb flash I ever find nowadays is 2 gig at minimum.  So this is more than possible.  However, I'm a newbie and am expecting to need to reinstall ESXi a few times just for learning purposes.  I also only picked up 1 usb drive so I needed to install this onto something else.
Now, why not just install directly onto the raid array.  Why not have it redundant with my raid 1+0 setup, faster access with my hard disks etc.  Warning: This answer is going to be based off things I've read, and not what I've experienced just yet.  


Basically, we've all gone through accidentally mucking up our home Operating system at least once and needed to format our home pc after we've backed up everything important.  Maybe we just wanted to upgrade/change to a new operating system.  The result is, formatting the drive, and installing everything from scratch.  Except now with a server we basically have all our vm's, and ISO's standalone.  We don't need to back things up to make changes.  VMs that can be loaded on notice.  Multiple datastores of information that can be secured outside of a VM and used by multiple vm's as network shares.  The goal of the server is to basically have the ability to fix/adjust to issues/wants/etc faster by providing a separation of all these types of data so that its never tied down to 1 system. 

Pros to USB:
  • Once a server loads ESXi into RAM it never reads/writes to the drive again unless you make changes to the host OS.  You don't need this taking up hard drive space.
  • If you want to upgrade the Host OS of ESXi, you just format/overwrite the USB with the new host OS.
  • If the usb drive goes bad. It will take a few minutes to replace. No big raid array rebuild.
  • If you're lucky enough to have 2 USB internal ports, you might be able to make an array off them.
Cons to USB:
  • Reading from my USB 2.0 drive is around 15mb/s  Its awful. Harddrive loads should be much faster than this.
  • Chances are you you can't tell how a USB harddrive is doing with smart technologies found on S.M.A.R.T. drives.  Something I need to look into.
  • Finding enterprise USB drives using SLC or MLC flash like high end SSDs isn't cheap.
  • If a USB drive goes bad, you will have to open up the server and replace it.  Depending on the motherboard/server this might be obnoxious. Especially if you don't have server cable arms allowing you to move a server out readily.
  • USB drives are always hit or miss, they just go bad. You will be replacing it eventually so you need to make sure you're ok with the process of doing a restore..

We've basically boxed ourselves into a corner that we can't reinstall the OS without some problems or hoops to jump through since it's on our main raid array.  Can it be done, I think so.  But its not something that I'm positive about just yet.  I wonder if I can install ESXi onto a USB drive while the other one still exists...As always leave a comment below on your thoughts or opinions.

Things TODO - ESXi can use a scratch file/drive.  This should be moved off usb onto something faster.  good place is ssd, but is a normal drive good?  should this be done always if you're using USB boot?

No comments:

Post a Comment