SuperMicro Build – The Components

The SuperMicro E200-8D and E300-8D are excellent options for a home lab, especially because of their small size, low power consumption and enterprise ready hardware. If you haven’t already read my first blog post, you can find my SuperMicro vs Intel NUC post here.

So you’ve bought a nice shiny new SuperMicro E200-8D and now you’re ready to start building you home lab, right? Not quite. These units don’t generally come as a plug and play unit, there is some assembly required. In my case this includes the RAM, NVMe M.2 SSD, 2.5” Sata SSD, an additional case fan and rack mount brackets. But it doesn’t stop there! Before we start to build our home lab, we need to update the BIOS firmware and the IPMI software, which enables the use of a HTML console session instead of the old JAVA console. I will cover the steps in more detail over the next few blog posts, first the Hardware selection, then the install guide and finally the BIOS and IPMI updates.

So let’s get started.

Bill of Materials

I purchased all of my hardware from Eric Yui at MITXPC. The prices and available hardware may vary, so if you’re interested then you should check out the MITXPC website for the available stocks and pricing. Don’t forget to use William Lam’s virtuallyGhetto discount! In case you didn’t know, William Lam has secured a 2% discount from MITXPC for the community. You can find all of the details here.

I am by no means recommending that you should buy the same hardware that I have, you should buy the hardware that suits your requirements and is also within your price range. I will outline the hardware options with a good, better and best option and you can make your own choices. Please add to the comments if you have any relevant experience on different products that you prefer.

First, here is what i bought.

Product

Part Number

Price

SuperMicro E200-8D SYS-E200-8D $799.99
64GB ECC UDIMM RAM (4 x 16GB) TBA $329.95
1TB 2.5” SSD SanDisk X400 $299.95
128GB NVMe M.2 SSD Plextor PX-128S2G $59.99
1x Additional Case Fan FAN-0065L4 $9.95
Rack Mount Brackets SMC-MCP-290-10110-0B $44.95
$1544.74
virtuallyGhetto 2% Discount VIRTUALLYGHETTO2OFF -$30.89
TOTAL   US $1513.85

RAM

027- RAM

ECC Capacity Speed Price
Good Non-ECC 64GB 2133MHz $300
Better ECC UDIMM 64GB 2133MHz $400
Best ECC RDIMM 128GB 2400MHz $1,000

This is a pretty simple decision, what RAM to fit to your SuperMicro E200-8D? Even just looking at the table above shows a pretty clear decision. In my opinion the only question here is what capacity of RAM do you need? 64GB or 128GB. That’s really about the hardest thing you’ll have to consider.

In regards to ECC or Non-ECC, well the price doesn’t change much and the SuperMicro E200 is restricted to 64GB of non-ECC RAM. I can’t imagine why you would need ECC RDIMM (Registered) RAM in your home lab. I have the ECC RDIMM listed there as the “best” option but that is purely on specs. My honest opinion is that the “best” option for your SuperMicro home lab is the ECC UDIMM (Un-Registered) RAM. For the price it’s a good buy and you aren’t restricted to 64GB, which means you can have up to 128GB RAM capacity if you so desire.

This leaves one major decision you have to make, what capacity of RAM do you buy. I opted for 64GB of ECC UDIMM RAM which cost me $330. Unfortunately, in the recent months the price of RAM has increased significantly and it is now approximately $400. I have covered this topic fairly heavily in my previous blog post – SuperMicro vs Intel NUC.

I’ll make the decision as simple as I can for you. How many VMs are you planning on running and how much RAM vs CPU do they require? The SuperMicro E200-8D has 11.4GHz of CPU processing power (1.9Ghz x 6 cores). Divide the amount of RAM you think you’ll need by 11.4GHz and that will give you the approximate RAM to CPU ratio. If this RAM to CPU ratio fits what you need in your environment, then buy that amount of RAM.

128GB RAM / 11.4GHz = 11.2GB RAM per 1GHz of CPU

64GB RAM / 11.4GHz = 5.6GB of RAM per 1GHz of CPU

There are more considerations to factor in to the above calculations that I have covered in my previous post (like VSAN RAM usage), so have a read through that and make a decision on your capacity. As I said, I opted for 64GB RAM (4x16GB) ECC UDIMM.

NVMe M.2

016 -960pro-980x551-1

Sequential Read (MB/s) Sequential Write (MB/s) 4K Random Read (IOPS) 4K Random Write (IOPS) Price (approx.)
Good 500 300 90,000 50,000 $80
Better 1,500 600 150,000 80,000 $150
Best 3,000 2,000 300,000 100,000 $300 +

The SuperMicro E200-8D contains an NVMe M.2 slot on the motherboard that accepts an 80mm PCI-E x4 SSD. When selecting an NVMe M.2 SSD there are a few things that you should consider, performance, size, the cost per GB and the bus interface.

Because I am building a VSAN environment, my NVMe card will be used as the caching tier in my VSAN storage, so I don’t need a large capacity card. Duncan Epping has detailed the flash cache calculation for VSAN here. I am running a single 1TB SSD in each ESXi host so using the 10% rule my 128GB NVMe SSD is actually oversized, however they don’t generally come much smaller and the price was great, so I grabbed it.

If I were to do it again, I would probably invest in a higher performance NVMe M.2 SSD. The SuperMicro supports a PCI-E 3.0 x4 interface which can provide higher performance than a SATA 3 interface and this can make a significant difference to the read/write caching performance in your VSAN environment. I went with the “good” option and I should have probably used the “better” option. What you really need to ask yourself is how much performance do you really need and how deep are your pockets?

If you do opt for a really fast NVMe SSD then make sure you also install the 3rd fan. These small and powerful SSDs can generate a lot of heat and the 3rd fan blows air straight over the NVMe SSD which will help to ensure longevity of your hardware.
My preference would be to buy the Samsung 960 EVO M.2 PCI-E, which can be found on Amazon starting from US$130. The 960 EVO in a 250GB size provides 3,000MB/s sequential read, 1,900MB/s sequential write and up to 300,000 IOPS. You can find more information here.

2.5” HDD

017 - StorageReview-SanDisk-X400

HDD/ SSD Capacity Sequential Read MB/s Sequential Write MB/s Price (approx.)
Good 7200 RPM 2TB + 200 150 $100
Better SSD 1TB 500 300 $300
Best SSD 2TB 550 400 $800

There is a huge range of 2.5” SATA drives that are more than suitable for the SuperMicro E200. The major concern here is capacity and price. If you are running VSAN like I am, then even a 7200rpm HDD is going to be a really good option due to their high capacity and cheaper price. Performance is still a concern but VSAN uses a fast NVMe M.2 SSD for read and write caching to provide better performance. The capacity disk will still need to have reasonably good read performance because not all reads will be processed on the high performance NVMe cache, so performance still matters but with only one disk space available in the E200-8D you need to use this space wisely and get the most out of your capacity disk.

Because I am running VSAN and I wanted to enable the compression and de-duplication capabilities, this means I am restricted to using an all flash VSAN with an SSD for the capacity disk. If you were running a hybrid VSAN then you could pair a high capacity 7200rpm HDD with a high performance NVMe card and end up with an excellent disk setup for your home lab.

In my situation, I have attempted to get the largest SSD capacity that was within my budget. I ended up purchasing the SanDisk X400 1TB SSD for US$300. This gives me 500MB/s sequential read and 350MB/s sequential write. The Samsung 850 EVO 1TB was a close contender but for an extra US$100 you get a little bit faster sequential write speed, which isn’t important for the VSAN capacity tier anyway. The SanDisk X400 provided me with 1TB capacity and more than enough performance for the capacity disk tier.

When comparing the NVMe and SSD disks, I have found that this UserBenchmark website has been extremely useful to compare various brands and their performance.

Scalability

Now that you have considered the above RAM, NVMe and HDD requirement, the one final thing that I would ask you to consider is if you will scale-out or scale-up when you require additional resources.

What this essentially means is that once you have consumed all of your resources, will you buy additional servers (with the same resources) or will you replace the components within your server to higher spec items? In my opinion the main constraint is CPU processing power (11.4GHz), so the best option is to lean towards scaling out.
Why is this important? Well, cost. If you have spec’d your servers with high performance items that are lower in capacity, then you will probably need to buy additional servers sooner rather than later. This could leave you out-of-pocket a lot of money. The “best” option is really above and beyond but this shows the capability of the SuperMicro E200 to enable a high performance and high capacity platform. For a home lab, I will always stick with the “better” options as I feel that this provides great performance, more than enough capacity and is very cost efficient.

Overall Component Price

 

RAM NVMe HDD Price

Good

64GB Non-ECC 500 MB/s HDD, 2TB

$480

Better

64GB ECC UDIMM 1,500 MB/s SSD, 1TB

$850

Best 128GB ECC RDIMM 3,000 MB/s SSD, 2TB

$2100

I hope that you can now understand the additional hardware components that are required to be purchased with the SuperMicro and can now make an informed decision as to what your options are. I would also hope that you have a realistic performance expectation that is based on the Good, Better and Best components. If you refer to the above table you can clearly see that there is a massive step up from the Better to the Best options. I’d like to say that you get what you pay for but in this scenario I don’t think that anyone requires the performance, capacity or availability that comes with the Best options. I personally sit somewhere between the Good and Better options but if I were to do it again, I would factor in spending $800 on the SuperMicro and another $800 on the additional components. With a little perspective it makes the SuperMicro look quite cheap when you are willing to spend just as much on the components as you are for the SuperMicro E200. This is where the real cost (and performance) is, the components.

Continue on to Part 3 to follow along with the Installation and common mistakes people make.

Home Lab Build Series

IntroductionHome Lab Build – From the Start

Part 1SuperMicro vs Intel NUC

Part 2 – SuperMicro Build – The Components

Part 3SuperMicro Build – The Installation

Part 4SuperMicro Build – BIOS and IPMI

Part 5 – Networking Configuration

Part 6 – VVD – Automated Deployment Toolkit – vSphere, VSAN, NSX and vDP

Part 7 –  VVD – Automated Deployment Toolkit – vRA, vRO, vROps and Log Insight


Previous –  SuperMicro vs Intel NUCNextSuperMicro Build – The Installation

6 thoughts on “SuperMicro Build – The Components

  1. Pingback: SuperMicro vs Intel NUC | Lab-Rat

  2. Pingback: SuperMicro Build – Installation | Lab-Rat

  3. Pingback: SuperMicro Build – BIOS and IPMI | Lab-Rat

  4. Pingback: Home Lab build – From the start. | Lab-Rat

  5. Hello Tai,

    Cool blog, thanks for taking the time writing. I just wanted to leave one comment about the ECC-decision. Some homelabs get a storage function and FreeNAS is a popular choice to run, sometimes virtual. A caveat is that you really should have ECC when you are going to do virtual FreeNAS and ZFS-like things, or you might lose your data some day. From your setup it doesn’t look like this will become your primary storage box, but others might build a different configuration and might need the ECC for that.

    Kind regards,
    Jaap de Vos

    Like

  6. Hi Jaap de Vos, Thank you for the compliment.
    Yes, ECC RDIMM RAM would be essential for dataloss in a FreeNAS configuration. I did consider adding this in to the article however I chose not to because I wouldn’t recommend using the SuperMicro servers for a FreeNAS build. Mostly because you only have enough space for a single drive and that would defeat the purpose of creating a ZFS pool. I would also recommend not virtualising a FreeNAS environment, yes it can be done but FreeNAS expects direct access to the hardware, especially the disks and storage controller. Also the amount of RAM that FreeNAS requires would consume almost all of the RAM in the SuperMicro and leave you running a single VM on a single ESXi host, which begs the question, why not just give it direct access to the hardware?
    I am actually running FreeNAS but I have a dedicated tower server full of disks (approximately 20TB of space) with 64GB ECC RDIMM RAM installed and a 10Gb network card. This is my dedicated external storage that I use for NFS datastores, backups of all my devices, media (pictures and video uploads) and also storage for my OwnCloud environment that I share with my family and friends.

    Like

Comments are closed.