The SuperMicro E200-8D and E300-8D are excellent options for a home lab, especially because of their small size, low power consumption and enterprise ready hardware. If you haven’t already read my first blog post, you can find my SuperMicro vs Intel NUC post here.
So you’ve bought a nice shiny new SuperMicro E200-8D and now you’re ready to start building you home lab, right? Not quite. These units don’t generally come as a plug and play unit, there is some assembly required. In my case this includes the RAM, NVMe M.2 SSD, 2.5” Sata SSD, an additional case fan and rack mount brackets. But it doesn’t stop there! Before we start to build our home lab, we need to update the BIOS firmware and the IPMI software, which enables the use of a HTML console session instead of the old JAVA console. I will cover the steps in more detail over the next few blog posts, first the Hardware selection, then the install guide and finally the BIOS and IPMI updates.
So let’s get started.
Bill of Materials
I purchased all of my hardware from Eric Yui at MITXPC. The prices and available hardware may vary, so if you’re interested then you should check out the MITXPC website for the available stocks and pricing. Don’t forget to use William Lam’s virtuallyGhetto discount! In case you didn’t know, William Lam has secured a 2% discount from MITXPC for the community. You can find all of the details here.
I am by no means recommending that you should buy the same hardware that I have, you should buy the hardware that suits your requirements and is also within your price range. I will outline the hardware options with a good, better and best option and you can make your own choices. Please add to the comments if you have any relevant experience on different products that you prefer.
First, here is what i bought.
|64GB ECC UDIMM RAM (4 x 16GB)||TBA||$329.95|
|1TB 2.5” SSD||SanDisk X400||$299.95|
|128GB NVMe M.2 SSD||Plextor PX-128S2G||$59.99|
|1x Additional Case Fan||FAN-0065L4||$9.95|
|Rack Mount Brackets||SMC-MCP-290-10110-0B||$44.95|
|virtuallyGhetto 2% Discount||VIRTUALLYGHETTO2OFF||-$30.89|
This is a pretty simple decision, what RAM to fit to your SuperMicro E200-8D? Even just looking at the table above shows a pretty clear decision. In my opinion the only question here is what capacity of RAM do you need? 64GB or 128GB. That’s really about the hardest thing you’ll have to consider.
In regards to ECC or Non-ECC, well the price doesn’t change much and the SuperMicro E200 is restricted to 64GB of non-ECC RAM. I can’t imagine why you would need ECC RDIMM (Registered) RAM in your home lab. I have the ECC RDIMM listed there as the “best” option but that is purely on specs. My honest opinion is that the “best” option for your SuperMicro home lab is the ECC UDIMM (Un-Registered) RAM. For the price it’s a good buy and you aren’t restricted to 64GB, which means you can have up to 128GB RAM capacity if you so desire.
This leaves one major decision you have to make, what capacity of RAM do you buy. I opted for 64GB of ECC UDIMM RAM which cost me $330. Unfortunately, in the recent months the price of RAM has increased significantly and it is now approximately $400. I have covered this topic fairly heavily in my previous blog post – SuperMicro vs Intel NUC.
I’ll make the decision as simple as I can for you. How many VMs are you planning on running and how much RAM vs CPU do they require? The SuperMicro E200-8D has 11.4GHz of CPU processing power (1.9Ghz x 6 cores). Divide the amount of RAM you think you’ll need by 11.4GHz and that will give you the approximate RAM to CPU ratio. If this RAM to CPU ratio fits what you need in your environment, then buy that amount of RAM.
128GB RAM / 11.4GHz = 11.2GB RAM per 1GHz of CPU
64GB RAM / 11.4GHz = 5.6GB of RAM per 1GHz of CPU
There are more considerations to factor in to the above calculations that I have covered in my previous post (like VSAN RAM usage), so have a read through that and make a decision on your capacity. As I said, I opted for 64GB RAM (4x16GB) ECC UDIMM.
|Sequential Read (MB/s)||Sequential Write (MB/s)||4K Random Read (IOPS)||4K Random Write (IOPS)||Price (approx.)|
The SuperMicro E200-8D contains an NVMe M.2 slot on the motherboard that accepts an 80mm PCI-E x4 SSD. When selecting an NVMe M.2 SSD there are a few things that you should consider, performance, size, the cost per GB and the bus interface.
Because I am building a VSAN environment, my NVMe card will be used as the caching tier in my VSAN storage, so I don’t need a large capacity card. Duncan Epping has detailed the flash cache calculation for VSAN here. I am running a single 1TB SSD in each ESXi host so using the 10% rule my 128GB NVMe SSD is actually oversized, however they don’t generally come much smaller and the price was great, so I grabbed it.
If I were to do it again, I would probably invest in a higher performance NVMe M.2 SSD. The SuperMicro supports a PCI-E 3.0 x4 interface which can provide higher performance than a SATA 3 interface and this can make a significant difference to the read/write caching performance in your VSAN environment. I went with the “good” option and I should have probably used the “better” option. What you really need to ask yourself is how much performance do you really need and how deep are your pockets?
If you do opt for a really fast NVMe SSD then make sure you also install the 3rd fan. These small and powerful SSDs can generate a lot of heat and the 3rd fan blows air straight over the NVMe SSD which will help to ensure longevity of your hardware.
My preference would be to buy the Samsung 960 EVO M.2 PCI-E, which can be found on Amazon starting from US$130. The 960 EVO in a 250GB size provides 3,000MB/s sequential read, 1,900MB/s sequential write and up to 300,000 IOPS. You can find more information here.
|HDD/ SSD||Capacity||Sequential Read MB/s||Sequential Write MB/s||Price (approx.)|
|Good||7200 RPM||2TB +||200||150||$100|
There is a huge range of 2.5” SATA drives that are more than suitable for the SuperMicro E200. The major concern here is capacity and price. If you are running VSAN like I am, then even a 7200rpm HDD is going to be a really good option due to their high capacity and cheaper price. Performance is still a concern but VSAN uses a fast NVMe M.2 SSD for read and write caching to provide better performance. The capacity disk will still need to have reasonably good read performance because not all reads will be processed on the high performance NVMe cache, so performance still matters but with only one disk space available in the E200-8D you need to use this space wisely and get the most out of your capacity disk.
Because I am running VSAN and I wanted to enable the compression and de-duplication capabilities, this means I am restricted to using an all flash VSAN with an SSD for the capacity disk. If you were running a hybrid VSAN then you could pair a high capacity 7200rpm HDD with a high performance NVMe card and end up with an excellent disk setup for your home lab.
In my situation, I have attempted to get the largest SSD capacity that was within my budget. I ended up purchasing the SanDisk X400 1TB SSD for US$300. This gives me 500MB/s sequential read and 350MB/s sequential write. The Samsung 850 EVO 1TB was a close contender but for an extra US$100 you get a little bit faster sequential write speed, which isn’t important for the VSAN capacity tier anyway. The SanDisk X400 provided me with 1TB capacity and more than enough performance for the capacity disk tier.
When comparing the NVMe and SSD disks, I have found that this UserBenchmark website has been extremely useful to compare various brands and their performance.
Now that you have considered the above RAM, NVMe and HDD requirement, the one final thing that I would ask you to consider is if you will scale-out or scale-up when you require additional resources.
What this essentially means is that once you have consumed all of your resources, will you buy additional servers (with the same resources) or will you replace the components within your server to higher spec items? In my opinion the main constraint is CPU processing power (11.4GHz), so the best option is to lean towards scaling out.
Why is this important? Well, cost. If you have spec’d your servers with high performance items that are lower in capacity, then you will probably need to buy additional servers sooner rather than later. This could leave you out-of-pocket a lot of money. The “best” option is really above and beyond but this shows the capability of the SuperMicro E200 to enable a high performance and high capacity platform. For a home lab, I will always stick with the “better” options as I feel that this provides great performance, more than enough capacity and is very cost efficient.
Overall Component Price
|64GB Non-ECC||500 MB/s||HDD, 2TB||
|64GB ECC UDIMM||1,500 MB/s||SSD, 1TB||
|Best||128GB ECC RDIMM||3,000 MB/s||SSD, 2TB||
I hope that you can now understand the additional hardware components that are required to be purchased with the SuperMicro and can now make an informed decision as to what your options are. I would also hope that you have a realistic performance expectation that is based on the Good, Better and Best components. If you refer to the above table you can clearly see that there is a massive step up from the Better to the Best options. I’d like to say that you get what you pay for but in this scenario I don’t think that anyone requires the performance, capacity or availability that comes with the Best options. I personally sit somewhere between the Good and Better options but if I were to do it again, I would factor in spending $800 on the SuperMicro and another $800 on the additional components. With a little perspective it makes the SuperMicro look quite cheap when you are willing to spend just as much on the components as you are for the SuperMicro E200. This is where the real cost (and performance) is, the components.
Continue on to Part 3 to follow along with the Installation and common mistakes people make.
Home Lab Build Series
Introduction – Home Lab Build – From the Start
Part 1 – SuperMicro vs Intel NUC
Part 2 – SuperMicro Build – The Components
Part 3 – SuperMicro Build – The Installation
Part 4 – SuperMicro Build – BIOS and IPMI
Part 5 – Networking Configuration
Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP
Part 7 – VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight