Why would I want to consider ESXi in the first place? After all you can have VirtualBox, VMware player, KVM, Xen, etc. Well first of all because I am familiar with it but also because it is a Hypervisor type 1 and because I think VMware is probably the best platform out there.
If you can install a type 1 Hypervisor the benefits can be substantial, the resource utilisation is better managed and more efficiently shared amongst the various guests VMs.
In selecting my new hardware to rebuild my home ESXi server I have chosen CPU, RAM and motherboard to make the most of virtualisation without braking the budget too much.
My requirmenets were
CPU: high number of cores, Intel Hypertreading Technology and Intel Virtualisation Technology
RAM: needed to be reasonably fast but mostly generous enough. Good £/GB ratio to get as much as I could.
Matherboard: a part for being able to host the CPU and RAM of my choice, I wanted something essential but with at least 4 SATA ports. No matter the type of SATA disks that I was going to use in fact, I wanted to give the various VMs as many independent I/O paths and as much bandwidth as possible. It also had to accommodate at least three network cards in addition to the one on board, more on the why later on.
The configuration I went with
Intel Core i7 4790K (The “K” didn’t matter so much but the price was very low ;))
Corsair Vengeance Pro 16 GB (2 x 8 GB) DDR3 2400 Mhz (I could have taken something slower but again the cost was the same)
Asus Z97-P (Cheap enough, good quality and ticks all my needs)
With this, I had 4 cores with Hyperthreading and VT which could provide 8 vCPUs without relying to much on over allocation. Although ESXi has a great scheduler and will manage more than 8 vCPUs despite the 8 physical ones but for the moment we won’t worry about this.
The same can be said for the 16GB that will be more than plenty for a while and won’t need to make use of memory sharing. ballooning or any other ESXi trickery to over provision resources. As my weak spot will be SATA, the more I can keep in memory the better will be.
The build of this new server was quite a refreshing experience for me. Although I started my geek life in computer shops doing just that, it had been a long while since a built a PC from scratch and things changed substantially since then. I believe my last PC build was something like a PIII socket 1!!! The end result wasn’t so bad.
Get ESXi the way you want it
The part the matters next is how to retrieve, custom and install ESXi for you box. It turns out that there is an ocean of resources out there and some are really well put together.
That is the case for VMware Front Experience which is so incredibly full of information that I will hardly be able to contribute any further. In fact that’s it! End of the article. 🙂
One of the things that I read in this site before I made my final choice of motherboard, was to do with what drivers are currently included in ESXi base image and what is anyway still available to add back in should you need to. You can get a good feeling of the supported devices and technologies thorough VMware Compatibility Guide although not too much will be relevant for home computers like the one I build.
Packages which have been removed but are still available, can be found here and can drive which hardware are you going to need to buy. Not least the type of network cards which in my case is the almighty Realtek for the on board and a bunch of additional Intel NICs.
The list above will eventually be used to create a custom version of ESXi in which we can add back all the drivers that we need and that are no longer included. In addition to the Realtek drivers I have also added back the SATA AHCI controller which I think the majority of us would want in our custom image.
Without even needing to get the base image from VMware, on VMware Front Experience you can get a wonderful tool that will take the base image, add the drivers you want and repackage it. Wow!!
The customiser is a great tool and in its latest incarnation is a PowerShell script that relies on PowerCLI to run. In addition to downloading the script you will also then need the PowerCLI.
As I said the customiser will be able to download the base image on its own but we also need the PowerCLI and the vSphere client for later and most importantly your free licence which still needs to be installed to remove the trial day limit. Considering this you might as well get on VMware vSphere site, get yourself an account and download the following which at the time of writing are about vSphere 6.0:
- VMware vSphere Hypervisor 6.0 Update 1 (optional)
- VMware vSphere Client
- VMware vSphere PowerCLI 6.0 Release 3
Get the image ready
Using the customiser is very easy and well described on its page. The command I needed to run was:
.\ESXi-Customizer-PS-v2.4.ps1 -v60 -vft -load sata-xahci,net55-r8168
Now, I do not have a DVD/ROM on my new box, neither did I on the previous one. I much prefer using USB keys for my installation. After all I would need the DVD/ROM only once! A bit of a waste if I’d leave it there, wouldn’t it?
In order to make your image USB bootable you need to use a really nice piece of software called Rufus which will help you do just that.
So with the image we have just created let’s get Rufus do the rest as described in this article
Before we move onto installing ESXi let’s make sure that Hypertreading and VT are enabled in the BIOS and that the SATA controllers are set as AHCI not RAID. The kind of RAID available on home motherboards is not good for ESXi. The way you do this depends on the motherboard and BIOS you have but generally it is around advanced and CPU properties.
One things you also might want to do is disable anything that you are not certainly going to use e.g. the audio card.
We now can put our USB key to good use and finally boot our new server. The installation process is quite easy and not much get’s to be decided during it. The image is very light and will install in very little time.
Configure you new “home cloud”
This article describes quite well what happens and what need to be done during and after the installation. It is for version 5 but it won’t make much difference at all.
In the meantime that the installation is progressing we can install the vSphere client (VMware-viclient) which we will need in a short while. The client needs a Windows box so find one somewhere and go ahead to install it.
The very first bit of configuration that we need to do after the installation is finished is to give an IP address to the management interface through which we will connect to with the vSphere Client. It will have most probably gotten one from the DHCP but you are better off configuring it with a static IP.
Have a look at this little complementary video I made of the installation and initial configuration.
Once the installation is over and the management network is configured, we can then finish the initial configuration using the fat client we installed on a Windows box somewhere.
First we login. Do not tick using Windows credentials as it won’t work. In our case we need the local credentials we inserted during the installation.
First lets’ check that all is as we expect.
We have our 8 CPUs,
We have our 16GB of RAM. (Note how small is the footprint for ESXi)
Then we want to go ahead and set at least the time, the DNS and same basic networking.
This is why I needed those many network cards. I am running my home firewall as a VM and have created quite a few virtual networks for that. This could be an article on its own as it involves also my router and VLAN tagging so I won’t go any further at this point. Just bare in mind that you will need to at least define a few or at least one Virtual Machine Port Group and that in the majority of cases it might be on the same switch (vSwitch0) in which you have already defined your Management Network.
The storage should already be OK from installation but if as I did you want to add more disks this is where you should go.
Back it up and goodbye
Backing up your VMs will be an essential task to carry out regularly and it was certainly something I used to backup mine before I moved them on the new server. For this I use a nice and intuitive too from Trailed called VM Explorer. In its free version it has some limitations (e.g. it won’t do restores) but nothing that can’t be overcome with some sweat and creativity.
I hope I did not confuse you too much and that I haven’t left too many things out. If I did let me know!
Happy virtualisation