Upgrading the BIOS and IPMI firmware is not necessary but I do highly recommend at least the IPMI update. Depending on the age of your system the BIOS may already be up to date. I purchased three servers at the same time, from MITXPC and two of them already had the updated BIOS firmware. Upgrading the firmware can be a daunting task, especially if you have just bought a new server. The last thing you want to do is break your new server by upgrading the Firmware incorrectly. Luckily the process to upgrade the firmware is pretty straight forward so I went ahead and did it just to be sure. If you aren’t comfortable taking responsibility for upgrading your firmware then just upgrade the IPMI software.

Upgrading the BIOS Firmware and IPMI

Download the SuperMicro BIOS update and IPMI update files from the website here:

https://www.supermicro.com/support/bios/firmware.aspx

https://www.supermicro.com/support/resources/results.aspx

The IPMI update will all you to use the iKVM/HTML5 console screen rather than the older JAVA based console. This is much easier to use and much more secure.

In order to upgrade the BIOS firmware you need to boot to a USB disk and apply the firmware update from there. I use Rufus to create the bootable USB disk. There are no specific configuration details that you need to configure with Rufus other than to use a FAT32 file system and select the FreeDOS bootable image.

Once this is finished, copy the relevant files to the USB drive. You can see below that I have also copied the SuperMicro IPMI tool. This is handy in case you need to reset the IPMI settings.

IPMICFG_1.26.0_20161227 – SuperMicro IPMI Configuration Tool

REDFISH_X10_346 – SuperMicro IPMI Update

X10SDVF6_A03 – SuperMicro BIOS Update

 

Insert the USB drive into your SuperMicro E200 and boot to the USB. You will need to Select the non UEFI bootable device that represents your USB drive.

When FreeDOS has booted, change the directory to the BIOS Update folder and then type:

FLASH.bat

The BIOS update will now run. It takes a while, so be patient. Once it is complete you will see the below confirmation to power-off the system. For good measure it is also recommended by SuperMicro to reset the CMOS.

The BIOS update is now complete. The next step is to update the IPMI.

The IPMI update can be done from the UI very easily. If you haven’t already configured the IPMI, log into the BIOS and on the IPMI tab configure your networking settings.

Type the IP address for the IPMI into a web browser and then enter the default username and password (needs to be capitals):

Username: ADMIN

Password: ADMIN

Under the Maintenance tab, select Firmware Update.

Press the button to “Enter Maintenance Mode” and then upload your IPMI Update file.

Once the files are uploaded you will need to un-tick the Preserve settings options. This is required by SuperMicro as part of the IPMI upgrade. If you leave these options ticked the upgrade may fail.

The upgrade may take a while and the system will reboot once it has finished. Once the upgrade is complete you can see the Firmware Revision is now updated.

After the IPMI upgrade is complete, you can now launch a iKVM/HTML5 remote console session.

Your hardware is now ready to use. The next blog post will be demonstrating the VMware Validated Design and the Automated Deployment Toolkit, which will build the entire SDDC stack in my home lab, FULLY AUTOMATED.

Home Lab Build Series

Introduction – Home Lab Build – From the Start

Part 1 – SuperMicro vs Intel NUC

Part 2 – SuperMicro Build – The Components

Part 3 – SuperMicro Build – The Installation

Part 4 – SuperMicro Build – BIOS and IPMI

Part 5 – Networking Configuration

Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP

Part 7 –  VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight

---

Previous – SuperMicro Build – The InstallationNext – Networking Configuration

The SuperMicro E200-8D and E300-8D are excellent options for a home lab, especially because of their small size, low power consumption and enterprise ready hardware. If you haven’t already read my first blog post, you can find my SuperMicro vs Intel NUC post here.

So you’ve bought a nice shiny new SuperMicro E200-8D and now you’re ready to start building you home lab, right? Not quite. These units don’t generally come as a plug and play unit, there is some assembly required. In my case this includes the RAM, NVMe M.2 SSD, 2.5” Sata SSD, an additional case fan and rack mount brackets. But it doesn’t stop there! Before we start to build our home lab, we need to update the BIOS firmware and the IPMI software, which enables the use of a HTML console session instead of the old JAVA console. I will cover the steps in more detail over the next few blog posts, first the Hardware selection, then the install guide and finally the BIOS and IPMI updates.

So let’s get started.

Bill of Materials

I purchased all of my hardware from Eric Yui at MITXPC. The prices and available hardware may vary, so if you’re interested then you should check out the MITXPC website for the available stocks and pricing. Don’t forget to use William Lam’s virtuallyGhetto discount! In case you didn’t know, William Lam has secured a 2% discount from MITXPC for the community. You can find all of the details here.

I am by no means recommending that you should buy the same hardware that I have, you should buy the hardware that suits your requirements and is also within your price range. I will outline the hardware options with a good, better and best option and you can make your own choices. Please add to the comments if you have any relevant experience on different products that you prefer.

First, here is what i bought.

Product	Part Number	Price
SuperMicro E200-8D	SYS-E200-8D	$799.99
64GB ECC UDIMM RAM (4 x 16GB)	TBA	$329.95
1TB 2.5” SSD	SanDisk X400	$299.95
128GB NVMe M.2 SSD	Plextor PX-128S2G	$59.99
1x Additional Case Fan	FAN-0065L4	$9.95
Rack Mount Brackets	SMC-MCP-290-10110-0B	$44.95
		$1544.74
virtuallyGhetto 2% Discount	VIRTUALLYGHETTO2OFF	-$30.89
TOTAL		US $1513.85

RAM

	ECC	Capacity	Speed	Price
Good	Non-ECC	64GB	2133MHz	$300
Better	ECC UDIMM	64GB	2133MHz	$400
Best	ECC RDIMM	128GB	2400MHz	$1,000

This is a pretty simple decision, what RAM to fit to your SuperMicro E200-8D? Even just looking at the table above shows a pretty clear decision. In my opinion the only question here is what capacity of RAM do you need? 64GB or 128GB. That’s really about the hardest thing you’ll have to consider.

In regards to ECC or Non-ECC, well the price doesn’t change much and the SuperMicro E200 is restricted to 64GB of non-ECC RAM. I can’t imagine why you would need ECC RDIMM (Registered) RAM in your home lab. I have the ECC RDIMM listed there as the “best” option but that is purely on specs. My honest opinion is that the “best” option for your SuperMicro home lab is the ECC UDIMM (Un-Registered) RAM. For the price it’s a good buy and you aren’t restricted to 64GB, which means you can have up to 128GB RAM capacity if you so desire.

This leaves one major decision you have to make, what capacity of RAM do you buy. I opted for 64GB of ECC UDIMM RAM which cost me $330. Unfortunately, in the recent months the price of RAM has increased significantly and it is now approximately $400. I have covered this topic fairly heavily in my previous blog post – SuperMicro vs Intel NUC.

I’ll make the decision as simple as I can for you. How many VMs are you planning on running and how much RAM vs CPU do they require? The SuperMicro E200-8D has 11.4GHz of CPU processing power (1.9Ghz x 6 cores). Divide the amount of RAM you think you’ll need by 11.4GHz and that will give you the approximate RAM to CPU ratio. If this RAM to CPU ratio fits what you need in your environment, then buy that amount of RAM.

128GB RAM / 11.4GHz = 11.2GB RAM per 1GHz of CPU

64GB RAM / 11.4GHz = 5.6GB of RAM per 1GHz of CPU

There are more considerations to factor in to the above calculations that I have covered in my previous post (like VSAN RAM usage), so have a read through that and make a decision on your capacity. As I said, I opted for 64GB RAM (4x16GB) ECC UDIMM.

NVMe M.2

	Sequential Read (MB/s)	Sequential Write (MB/s)	4K Random Read (IOPS)	4K Random Write (IOPS)	Price (approx.)
Good	500	300	90,000	50,000	$80
Better	1,500	600	150,000	80,000	$150
Best	3,000	2,000	300,000	100,000	$300 +

The SuperMicro E200-8D contains an NVMe M.2 slot on the motherboard that accepts an 80mm PCI-E x4 SSD. When selecting an NVMe M.2 SSD there are a few things that you should consider, performance, size, the cost per GB and the bus interface.

Because I am building a VSAN environment, my NVMe card will be used as the caching tier in my VSAN storage, so I don’t need a large capacity card. Duncan Epping has detailed the flash cache calculation for VSAN here. I am running a single 1TB SSD in each ESXi host so using the 10% rule my 128GB NVMe SSD is actually oversized, however they don’t generally come much smaller and the price was great, so I grabbed it.

If I were to do it again, I would probably invest in a higher performance NVMe M.2 SSD. The SuperMicro supports a PCI-E 3.0 x4 interface which can provide higher performance than a SATA 3 interface and this can make a significant difference to the read/write caching performance in your VSAN environment. I went with the “good” option and I should have probably used the “better” option. What you really need to ask yourself is how much performance do you really need and how deep are your pockets?

If you do opt for a really fast NVMe SSD then make sure you also install the 3^rd fan. These small and powerful SSDs can generate a lot of heat and the 3^rd fan blows air straight over the NVMe SSD which will help to ensure longevity of your hardware.
My preference would be to buy the Samsung 960 EVO M.2 PCI-E, which can be found on Amazon starting from US$130. The 960 EVO in a 250GB size provides 3,000MB/s sequential read, 1,900MB/s sequential write and up to 300,000 IOPS. You can find more information here.

2.5” HDD

	HDD/ SSD	Capacity	Sequential Read MB/s	Sequential Write MB/s	Price (approx.)
Good	7200 RPM	2TB +	200	150	$100
Better	SSD	1TB	500	300	$300
Best	SSD	2TB	550	400	$800

There is a huge range of 2.5” SATA drives that are more than suitable for the SuperMicro E200. The major concern here is capacity and price. If you are running VSAN like I am, then even a 7200rpm HDD is going to be a really good option due to their high capacity and cheaper price. Performance is still a concern but VSAN uses a fast NVMe M.2 SSD for read and write caching to provide better performance. The capacity disk will still need to have reasonably good read performance because not all reads will be processed on the high performance NVMe cache, so performance still matters but with only one disk space available in the E200-8D you need to use this space wisely and get the most out of your capacity disk.

Because I am running VSAN and I wanted to enable the compression and de-duplication capabilities, this means I am restricted to using an all flash VSAN with an SSD for the capacity disk. If you were running a hybrid VSAN then you could pair a high capacity 7200rpm HDD with a high performance NVMe card and end up with an excellent disk setup for your home lab.

In my situation, I have attempted to get the largest SSD capacity that was within my budget. I ended up purchasing the SanDisk X400 1TB SSD for US$300. This gives me 500MB/s sequential read and 350MB/s sequential write. The Samsung 850 EVO 1TB was a close contender but for an extra US$100 you get a little bit faster sequential write speed, which isn’t important for the VSAN capacity tier anyway. The SanDisk X400 provided me with 1TB capacity and more than enough performance for the capacity disk tier.

When comparing the NVMe and SSD disks, I have found that this UserBenchmark website has been extremely useful to compare various brands and their performance.

Scalability

Now that you have considered the above RAM, NVMe and HDD requirement, the one final thing that I would ask you to consider is if you will scale-out or scale-up when you require additional resources.

What this essentially means is that once you have consumed all of your resources, will you buy additional servers (with the same resources) or will you replace the components within your server to higher spec items? In my opinion the main constraint is CPU processing power (11.4GHz), so the best option is to lean towards scaling out.
Why is this important? Well, cost. If you have spec’d your servers with high performance items that are lower in capacity, then you will probably need to buy additional servers sooner rather than later. This could leave you out-of-pocket a lot of money. The “best” option is really above and beyond but this shows the capability of the SuperMicro E200 to enable a high performance and high capacity platform. For a home lab, I will always stick with the “better” options as I feel that this provides great performance, more than enough capacity and is very cost efficient.

Overall Component Price

	RAM	NVMe	HDD	Price
Good	64GB Non-ECC	500 MB/s	HDD, 2TB	$480
Better	64GB ECC UDIMM	1,500 MB/s	SSD, 1TB	$850
Best	128GB ECC RDIMM	3,000 MB/s	SSD, 2TB	$2100

I hope that you can now understand the additional hardware components that are required to be purchased with the SuperMicro and can now make an informed decision as to what your options are. I would also hope that you have a realistic performance expectation that is based on the Good, Better and Best components. If you refer to the above table you can clearly see that there is a massive step up from the Better to the Best options. I’d like to say that you get what you pay for but in this scenario I don’t think that anyone requires the performance, capacity or availability that comes with the Best options. I personally sit somewhere between the Good and Better options but if I were to do it again, I would factor in spending $800 on the SuperMicro and another $800 on the additional components. With a little perspective it makes the SuperMicro look quite cheap when you are willing to spend just as much on the components as you are for the SuperMicro E200. This is where the real cost (and performance) is, the components.

Continue on to Part 3 to follow along with the Installation and common mistakes people make.

Home Lab Build Series

Introduction – Home Lab Build – From the Start

Part 1 – SuperMicro vs Intel NUC

Part 2 – SuperMicro Build – The Components

Part 3 – SuperMicro Build – The Installation

Part 4 – SuperMicro Build – BIOS and IPMI

Part 5 – Networking Configuration

Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP

Part 7 –  VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight

---

Previous –  SuperMicro vs Intel NUCNext – SuperMicro Build – The Installation

Most of my colleagues and just about anyone in the IT industry have a home lab. Is there a better way to continue learning your chosen technology? Most of us learn from experience and how better to gain experience than by building your own enterprise environment at home. I have set out to rebuild my old home lab environment and I will be detailing the configuration throughout the process so that anyone can build something similar in their home. I have chosen the hardware platform (SuperMicro E200-8D) and will be detailing the process to build out an entire VMware SDDC lab environment.

There are multiple approaches to building your lab at home. Not many people have the capability or money to run enterprise hardware in your home lab, there are sacrifices to be made and most often these decisions come down to noise, space, power consumption and cost.

If you don’t have a home lab already, build one, break it, fix it, maintain it and learn from your experience!

Home Lab Build Series

Introduction – Home Lab Build – From the Start

Part 1 – SuperMicro vs Intel NUC

Part 2 – SuperMicro Build – The Components

Part 3 – SuperMicro Build – The Installation

Part 4 – SuperMicro Build – BIOS and IPMI

Part 5 – Networking Configuration

Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP

Part 7 –  VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight

---

Next – SuperMicro vs Intel NUC

A couple of weeks ago I was talking to William Lam (http://www.virtuallyghetto.com/) and Alan Renouf (http://www.virtu-al.net/) about their exciting USB to SDDC demonstration, they were using an Intel NUC to deploy a VMware SDDC environment to a single node using VSAN. I offered them the opportunity to test out the same capability with one of my SuperMicro E200-8D servers and they took me up on the opportunity. Since then I have been approached by a number of people with requests for information about why I chose to go with the SuperMicro E200 for my home lab over the Intel NUC. I’ve never written a blog before but I thought this might be a good way to “cut out the middle man” so to speak. So here it goes, my reasons for why I chose the SuperMicro over the Intel NUC.

My previous home labs have generally been made up of used enterprise servers that can be picked up cheaply. These used servers are loud, power hungry and heavy. My goal was to firstly consume less power and secondly make my lab somewhat portable. These requirements appear to be popular amongst the community at the moment and there are a lot of stories about people using the Intel NUC’s to achieve these outcomes. I started to look around and it was fairly obvious that there were two stand-out options, the Intel NUC and the SuperMicro E200. I was left with a decision to be made and had to ask myself some additional questions about what I really wanted in my home lab. I came up with the following requirements.

1. Minimal power consumption.
2. Small and lightweight.
3. Capable of a good consolidation ratio of VMs to Host.
4. Capable of using all flash VSAN (albeit in an unsupported configuration).
5. Enough scalability to expand the lab in the future.
6. Good availability. These need to be up when I am doing demonstrations to customers.
7. Ability to set up an enterprise-like environment for comparisons with customer environments.

The next step was to compare my options. The following table takes information from the respective vendor sites and addresses my specific requirements plus a couple of additional considerations. This table made my decision easy for me. It became quite obvious to me that the SuperMicro was the more superior option for my home lab, in fact the SuperMicro is an enterprise ready solution.

	SuperMicro E200-8D	Intel NUC 7th Gen
ESXi 6.5 Compatible	Native install works	Requires NIC drivers
CPU Type	XEON D-1528	Intel i7-7567U
CPU Capacity	6 core 12 threads 1.9 GHz – 2.2 GHz	Dual Core 3.5 GHz – 4.0 GHz
RAM Type	4x DDR4	2x DDR4 SODIMM
RAM Capacity	128GB ECC RDIMM 64 GB Non-ECC UDIMM	32GB Non-ECC
Intel Optane Ready	Yes	Yes
HDD Capacity	1x 2.5”	1x 2.5”
NVMe / M.2 SATA	YES	YES
SATADOM	YES	NO
Micro SDXC	NO	YES
1Gbe Networking	2x 1Gbe	1x 1Gbe
10Gbe Networking	2x 10Gbe	1x Thunderbolt 40Gbps
Wireless	NO	802.11ac
IPMI	YES	NO
Power Consumption	60W	64W
Rack Mounting	YES	NO
SR-IOV Support	YES	NO
Video Port	VGA only	HDMI 4k
Noise Comparison	2x Fans	Fan-less
USB Capacity	2 Ports	4 Ports
Price Comparison	US$799	US$630

Hopefully the above table has also helped you with your decision. The reason I opted for the SuperMicro E200 is that although it does cost  little bit more, it is an enterprise ready solution that accepts ECC memory, uses a XEON CPU, has a larger RAM capacity and a larger CPU capacity.

To provide more information, here are the more detailed comparisons between the SuperMicro E200 and the Intel NUC.

ESXi 6.5 Compatible

Both the NUC and the SuperMicro require additional drivers and configurations to the native ESXi installation in order to work properly, so this point is more for information rather than defining the obvious choice. The 1Gbe NICs on the SuperMicro work with the native ESXi drivers and will work out-of-the-box, you need to install additional drivers to get the 10Gbe NICs to work. The 10Gbe drivers are supported with ESXi 6.0 and can be found on the VMware Downloads page here: https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI60-INTEL-IXGBE-451&productId=491

The NUC requires additional drivers in order to get the 1Gbe NIC to work. This means that you will need to either build a custom image or install the drivers locally after ESXi is installed. William Lam has detailed the options and procedures here: http://www.virtuallyghetto.com/2017/02/update-on-intel-nuc-7th-gen-kaby-lake-esxi-6-x.html

While you are creating your custom ESXi image for your NUC or SuperMicro I would recommend you remove the native vmw-ahci driver vib from your image. This will force your storage controller to use the newer sata-ahci drivers. ESXi 6.5 contains many new drivers, however the standard image still contains both the native drivers and a newer version. VMware don’t chose to default to the use of the newer version of the drivers because they may not be 100% feature comparable. In this case, if you review the storage controller support on the VMware compatibility list, it clearly states that it is supported with the sata-ahci drivers. The native vmw-achi drivers do not perform well and you will recognise a massive performance improvement by using the new drivers. Anthony Spiteri has done an excellent job detailing the issues and resolution here: http://anthonyspiteri.net/homelab-supermicro-5020d-tnt4-storage-driver-performance-issues-and-fix/ 

Networking

The SuperMicro has a significantly better networking capability than the Intel NUC. Looking at the Intel NUC you could probably use Thunderbolt to Ethernet adapters or USB to Ethernet adapters and build yourself a redundant networking capability. At the end of the day, the SuperMicro has 2x 10Gbe NICs and 2x 1Gbe NICs. Because I want to run an all flash VSAN configuration, I want to use the 10Gbe networking capability to optimise my VSAN performance.

During my 10Gbe vs 1Gbe networking considerations, I also considered the SuperMicro E300-8D due to its 10Gb SFP configuration (and expansion PCIe slot). It is very hard to find a 10GBase-T network switch at a reasonable price and I ended up having to spend the most money on my 10GBase-T 48 port Dell switch. In hindsight, the SuperMicro E300-8D could have been a viable option because I would have been able to run a VSAN supported storage controller and a 10Gb SFP switch is much easier to find at a reasonable price. Of course there is no comparison to the Intel NUC because it doesn’t have 10Gb NICs, let alone multiple 10Gb NICs. I eventually made the decision that a 10Gb networking capability would not only provide me with the performance but also scalability and not need to be replaced in a couple years time.

Storage

The Intel NUC and SuperMicro both have similar storage capabilities. If you are wanting to run VSAN then you will need to buy an NVMe card for the caching tier and a 2.5″ HDD (or SSD) for the capacity tier. The SuperMicro has more SATA ports on the motherboard but no space to mount any additional drives, so not much point in even considering them.

The one thing that you can seriously consider here is to look at the SuperMicro E300-8D. The E300 has a smaller CPU (XEON D-1518 4 core 2.2GHz) but it is larger in size due to a PCIe x8 slot. This would be a great place to use a VSAN supported storage controller!

IPMI

I love that the SuperMicro has a built in IPMI port. This allows me to view a console screen or mount an ISO over the network. To put it simply, I don’t need to go out to my garage to manage my lab.

Noise

Yes, this is where the Intel NUC wins. The NUC doesn’t have any fans and therefore doesn’t make any noise. You could put these inside your house and you wouldn’t know they’re there. In comparison, the SuperMicro could be considered to be quite loud. This wasn’t an issue for me because my lab is in my garage, plus once you turn on my 10Gb 48 port switch, you can disregard any noise that the SuperMicro might be making. Did I really want to sacrifice my cooling capability and running temperature in order to reduce the noise? No, I want fans pushing as much air through my lab to keep it cool and a bit of noise is worth it. In fact, the SuperMicro comes with 2x 40mm fans and a spare slot for a 3rd fan which I promptly populated straight away.

Take a look at Paul Braren’s blog at TinkerTry where he analyses the noise from the SuperMicro servers – https://tinkertry.com/supermicro-superserver-sys-e200-8d-and-sys-e300-are-here

Power Consumption

One of my most critical requirements was lower power consumption. I have had some pretty high electricity bills in the past while running large rack mount servers. I haven’t measured the actual power consumption from between the Intel NUC and SuperMicro however I would be very confident that the Intel NUC would consume less power. Both units are very low in power consumption compared to a large rack mount server so they both meet my requirement.

Rack Mount

This was a big bonus. The SuperMicro E200-8D has rack mount brackets. Not only does this make my lab neat and tidy, its also easily expandable and I can also build in a hot/cold zone within my rack. Where I live it can get very hot in summer (40 degrees celsius or 100 degrees fahrenheit) so keeping my lab cool is a must. By using rack mount panels I have been able to separate the front fan intake on the SuperMicros to the rear hot outlets. I can then duct cold air into the front of the rack and keep my lab operating temperature at a respectable level. If I were to use the Intel NUCs then I would have no way of keeping them cool during Summer.

The below diagrams show the rack mount configuration and part numbers for both the E200-8D (MCP-290-10110-0B) and the E300-8D (MCP-290-30002-0B). Although I could not find these listed for sale anywhere, Eric at MITXPC was able to source the rack mount brackets for me.

Video Port

This might seem simple, however do you have a HDMI capable monitor in your home lab? I don’t. I’m using a fairly old monitor with VGA and DVI ports. The Intel NUC may be 4k capable and offer a HDMI port, which would be great for a media PC but why would you need this in your home lab? If the Intel NUC also had a VGA port then it may be comparable but it only offers a HDMI port. The SuperMicro VGA port also comes in handy when you turn up at a customer site and they don’t have a HDMI port.

USB Capacity

This is a down side for the SuperMicro as it only has 2x USB ports. Because I am running VSAN, both my internal NVMe and SSD can’t be used to boot ESXi, so I use a small USB drive which runs ESXi. While installing ESXi to the SuperMicros I found myself with a lack of USB ports. One is required for a bootable USB drive to install ESXi to, the ESXi install media on another USB drive and a USB keyboard to click through the install. 3x USB ports would have been nice but I mounted the ESXi image over the IPMI connection and clicked through the install process from the comfort of my lounge room, using the IPMI console screen.

This is where the Intel NUC does provide a Micro SDXC slot which you could very well utilise as the ESXi install location. The NUC also has an additional 4x USB ports.

SATADOM

The Supermicro SATA DOM (Disk on Module), is a small SATA3 (6Gb/s) flash memory module designed to be inserted into a SATA connector and provides high performance solid state storage capacity that simulates a hard disk drive (HDD). The SATADOM device could be used as a boot disk for the ESXi installation rather than a bootable USB and are available in 128GB, 64GB, 32GB, and 16GB sizes.

CPU and RAM

The biggest consideration here is performance, capacity and availability, all of which the SuperMicro exceeds the Intel NUC in leaps and bounds. This makes the cost of the SuperMicro look cheap when compared to the Intel NUC based on the numbers, this is detailed below. At a high level the SuperMicro can use either ECC or non-ECC RAM, it uses full sized RAM slots rather than SODIMM, the RAM capacity is 4x larger at 128GB and the CPU capacity is nearly doubled. This makes the cost of the SuperMicro a lot cheaper than the Intel NUC once you start to consider purchasing more than a single unit.

The RAM capacity is a massive point in favour of the SuperMicro’s. This is incredibly important for the consolidation ratio of VMs to Hosts, especially when running all flash VSAN. You must remember to take into consideration that VSAN will consume a large chunk of your RAM. For ease of calculations I will use 10GB as my VSAN memory consumption, however the actual number was 10.5GB. Details of how to calculate your VSAN memory requirements can be found here: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113954

Let’s assume you opt for the Intel NUC with a maximum of 32GB RAM. You instantly lose 10GB to VSAN, so you’re left with 22GB RAM to use in your environment. Each Intel NUC will provide you with 7GHz of CPU processing power and 22GB of RAM. This leaves you with 3.14GB RAM for each 1Ghz of CPU used. From my previous analysis of my home lab my VMs average between 200MHz and 500MHz CPU usage. I will use 500Mhz (0.5GHz) for my calculations and a conservative estimate of my consolidation ratio.

7GHz / 0.5GHz = 14 VMs per NUC

I have approximately 65 VMs in my home lab and this would mean that I require 5x Intel NUCs just to meet my current capacity requirements with a consolidation ratio of 14 VMs per host. What’s worse is that I would be highly unlikely to actually get 14 VMs on each Intel NUC because I only have 22GB RAM available to use. Each of the 14 VMs would have approximately 1.5GB RAM allocated.

22GB RAM / 14 VMs = 1.57GB RAM per VM

Based on the above calculations the RAM is a massive constraint on the use of Intel NUCs in a home lab environment. Realistically, based on RAM consumption I would need 9 Intel NUCs in my lab. I have used an estimated 3GB RAM required per VM for the below calculations.

(3GB RAM per VM x 65 VMs) / 22GB RAM per NUC = 8.86 (9) NUCs

Each SuperMicro E200-8D has 11.4GHz CPU processing power and 128GB RAM (less 10GB for VSAN). Applying the same calculations as above.

11.4GHz / 0.5GHz = 22.8 VMs per SuperMicro

118GB RAM / 22.8 VMs = 5.18GB RAM per VM per SuperMicro

As you can see from the above calculation, with 128GB RAM in the SuperMicro the CPU becomes the constraining factor. Leaving 5.18GB of RAM for each VM using 0.5GHz of CPU. This is fairly consistent in a VMware environment where RAM is more heaving utilised than CPU, so you would be better off ensuring you have more RAM than CPU.

Let’s work out what my consolidation ratio will be based on my actual RAM requirements of 3GB RAM per VM.

(3GB RAM per VM x 65 VMs) / 118GB RAM per SuperMicro = 1.65 (2) SuperMicros

The calculations make it very obvious from anyones perspective. To suit my needs I need 2x SuperMicros or 9x Intel NUCs. I could have stuck with the 2x SuperMicros and setup a 2 node VSAN configuration utilising the virtual witness appliance as the 3rd node, however I want to make this enterprise ready so I opted to meet the minimum of 3 nodes for VSAN.

If you factor in the costs for 128GB of ECC RAM then it can get more expensive. Because I was going to buy 3x SuperMicro servers anyway (for VSAN) then why not be more price conscious and use 64GB non-ECC RAM per server. This meant that my lab was more realistically sized with 3 servers at 64GB RAM in each = 192GB RAM and the cost was $1499 per SuperMicro E200-8D (including disks).

Price

From all of the above details if it isn’t already obvious why I opted to build my lab with SuperMicro E200 servers, do the math on the cost.

9x Intel NUCs x $630 each = $5,670 USD

3x SuperMicros x $799 each = $2,397 USD

The costs I have used above are estimates based on a quick search, you may find cheaper prices if you look harder. I haven’t factored in the cost of the RAM, SSD or NVMe as these would be similar additional costs regardless of choosing the NUC or SuperMicro. There are other considerations that may affect the cost comparison of each unit, just to list a couple:

• The supported SSD and NVMe cards could warrant a difference in price.
• The RAM costs could vary between choosing to use SODIMM or ECC RAM.
• The rack mount brackets on the SuperMicro servers are an additional cost.
• The SuperMicros could likely consume more power during daily use.

If you have read all the way to the end, you are obviously just as interested in getting the “right” configuration for your home lab as I was. Seriously though, if at the end of this article you’re still leaning towards the NUC then just do it. You’re not going to be disappointed.

I purchased my SuperMicro servers from MITXPC as they specialise in micro systems. I found them via Amazon, however the prices on Amazon are significantly more expensive than if you go direct. If you’re interested, ask to speak with Eric Yui as he has been very helpful for me and will look after you.

Home Lab Build Series

Introduction – Home Lab Build – From the Start

Part 1 – SuperMicro vs Intel NUC

Part 2 – SuperMicro Build – The Components

Part 3 – SuperMicro Build – The Installation

Part 4 – SuperMicro Build – BIOS and IPMI

Part 5 – Networking Configuration

Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP

Part 7 –  VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight

---

Previous – Home Lab Build – From the StartNext – SuperMicro Build – The Components