Archive for March, 2018

The Build of my Development VM\Docker Server

0

In my previous post I talked about why I’m building a home lab server, the goals and starting info I was using. In this post I’m going to detail what I eventually got, the build itself and any gotchya’s that I encountered during that process.

My goal is to be able to have 6 Azure medium sized VM’s (4 core, 16GB of RAM) up and running all the time. That will save me $1,000 per month based on Pay-As-You-Go pricing. With still having some headroom left over for the host and other infrastructure.

Immediately I need to support the following VMs:

  • freeNAS Server for local sharing and backups
  • CyberPower UPS Agent
  • TeamCity build server and agent
  • Photon OS for DotNet Core and Docker development

Straight off the bat that’s 2 VM’s that would have cost me $330 a month. With the overall goal of adding up to 4 more. Resource allocation out of the gate will be:

Name CPU RAM
VM Host 1 500MB
CyberPower Agent 1 500MB
freeNAS 2 8GB
TeamCity 4 16GB
Docker Dev 4 16GB

Total

12 41GB

So now that I have my first set of system requirements, the load I’ll be placing on the system, it’s time to start buying the parts. As much as I can, I’m trying to buy the parts new and I was able to for most of the system, outside the processors which I got on eBay.

Part Link Cost
Intel Xeon E5-2670v2 CPU (x2) eBay $380 ($190\ea)
Samsung M393B1K70CH0-YH9 8GB PC3L-10600R RAM (x8) Amazon $190 ($23\ea)
Phanteks Enthoo Pro Full Tower Case Amazon $100
EVGA SuperNOVA 750 G2 PSU Amazon $120
Cooler Master Hyper RR-212E-20PK-R2 LED CPU FAN (x2) Amazon $60 ($30\ea)
PNY VCQFX370LP-PCIE-PB NVIDFIA Quadro FX 370 VIDEO Amazon $10
SAS9260-8I Kit Raid 8PORT w/ Cables Amazon $170
ASRock Motherboard LGA 2011 EP2C602-4L/D16 Amazon $380
Samsung 860 EVO 250GB SSD HDD Amazon $90
Asus 24x DVD-RW Amazon $20
WD 4 TB Black Performance HDD (x4) Amazon $760 ($190\ea)
CyberPower 1500VA UPS Amazon $140

Total

$2,420

To finish off the build above I’m missing 4 HDD’s (another $760) and 64GB of RAM (another $190). So the current total is short about $950 what the final total will be. To fully polish off the system is looks like I’ll be about $900 over budget.

IMG_6548

I also went expensive on the RAID HDD’s, opting for the faster WD Black Performance edition drives with 7200RPM and 128MB of Cache. Those HDD’s alone will turn into about 1/3 the systems overall cost. But that ensures that the VM’s and NAS will have fast access to the bulk of the storage on the system. You can save yourself a lot of money by going cheaper on the drives.

Some notes from this build:

  1. I also do not recommend the Cooler Master CPU coolers that I bought. Both fans mad a lot of noise and the coolers themselves are so large that if you don’t get a wide Full ATX case they won’t fit. I ended up buying a 2 pack of Corasir fans to replace the crap CM ones. Look for another LGA2011 fan or water cooler for your build.
  2. You do not need to buy the EVGA SuperNOVA 750 G2 PSU, you just need a PSU with 2 CPU cables (for the 2 procs). I think any modular PSU will be able to handle that, I know the SeaSonic ones I’ve been buying for years now all came with 2 CPU connectors.
  3. The motherboard on Amazon says it’s an ATX in the title, that’s 100% incorrect. It’s an SSI EEB server form factor board. You can buy ANY case you want, just as long as it supports Extended ATX (EATX) boards (12” x 13”).
  4. The Phanteks Enthoo Pro Full Tower is very wide at almost 9 1/2”, this is wider than my Cooler Master Stryker case which is pretty big. So ensure you plan accordingly with how much space you’re going to need vertically and horizontally.
  5. Never buy parts used in Crypto mining. This is why as best I could I got mine new. The CPU’s I checked with the seller to ensure they weren’t used in a miner environment. This is why I avoided CPU’s from Asia even though they were cheaper.
  6. I picked up a cheap (used) Quadro FX 370 so I could ensure I had a DVI port I could connect to my HDMI based KVM. I selected this card because I knew it would work with that generation of hardware, is low power and low profile. Note that if you pick it up you will need a special DMS-59 cable to convert it to DVI.
  7. I went with VMWare ESXi free edition, but finding out how to do that is a little hard, so here is a link to a VMWare KB article on how to license your trial edition for free.
  8. The RAID card I got from Amazon came with 2 the correct cables and the motherboard came with some for the SSD (where ESXi is installed) and the CDROM. There was no reason to buy additional cables (but ensure your RAID card will come with cables, as they are special 1 connector to 4 SATA ones).

IMG_6549

JP Toto’s excellent article on his build server (which I originally followed) has his out the door cost at $1,105 at 1/29/2017. But with the Crypto Mining frenzy, the prize on almost all components (CPU, RAM, PSU’s, Cooling, MB’s) have all risen. So keep an eye on prices to try and get better deals, you may have more luck at a local computer store getting good deals there as well.

If you’ve stumbled upon this article anytime in the future I highly recommend looking at V3, V4 or higher versions of the Xeon processes. With every version they get faster, more cores and more Cache. But you will need to buy DD4 RAM and a motherboard that supports that processor version, so just ensure you checking that compatibility. From what I read that ASRock MB that I bought will only support up to the v2 editions.

If you’re a First Responder, or know one, check out Resgrid which is a SaaS product utilizing Microsoft Azure, providing free logistics, management and communication tools to first responder organizations like volunteer fire departments, career fire departments, EMS, search & rescue, CERT, public safety, security and disaster relief organizations.

Why I’m building a Custom Development Home Server

I’m a pretty diehard Azure guy, been so since PDC09 when I was trying to get storage tables\queues and VM’s implemented at my job at the time. Almost all of my projects and stuff I land on Azure, it’s just the easiest place for me to do so, if not a little bit spendy.

I wasn’t always an Azure or cloud guy before I was a developer I was a systems administrator, responsible for servers ranging from SQL to Exchange at a small company. We 4 full racks of servers with a bunch of others with network equipment, firewalls, UPS’s and the like. During those days I couldn’t imagine a time not having control of the metal, and there almost nothing like unpacking a new piece of equipment, setting it up and getting it racked.

Back in those days, I had rack and servers all over my place. All the old stuff we decommissioned from the office. When I had all those up and running the noise was insane, but I was used to it from working in the server room most of the day.

Oh how times have changed, besides a beefy workstation and some other small devices and systems everything else I do is Azure based and has been for years. But try as I might there just are some things that I need on-prem at my house. So I will take you through that thought process.

First, where I live doesn’t have the best Internet. I now pay for business-grade Charter Spectrum, but the upload on that service is still pathetic, only 10Mbps. That may sound like a lot, but I plan on having some cameras installed at my place and each one of those requires 2Mbps. I used to use CrashPlan for my backup solution but with them going business only I took a hard look at what I was backing up and $100 or more a year and the cost in resources (upload) was pretty steep. Backups just made sense to have in-house with occasional syncing to OneDrive or some other storage provider.

So I looked into NAS options, finally settling on a Synology one, but when pricing it out it was pretty ouch, $500 for the unit and + the drives I wanted to add to it (with the expansion frame). Pricing that all out it would be about $1,500 for the NAS and storage (with fault tolerance i.e. RAID5) that I wanted. That’s a lot of money!

Recently I’ve started to work on other skills, mainly Containers, and non-Microsoft development stacks. Although I could use Azure for this, spinning up and keeping that stuff in the cloud could get pretty expensive, do I want to spend $300+ a month for some VM’s and serverless resources?

So I decided a local server would solve a lot of my needs. I could use it as a NAS, server to test deployments and any long running processes (like a Team City build server) and the like. So I will need a system with enough cores, ram and disk space for all those workloads.

An off the shelf system is also going to be a little on the big and loud side. I’m not too interested in having another jet turbine in the office. So based on that and the price for a system with enough core’s I’m going to have to build it myself.

I found this blog post, Build Your Own 32 Core Home Lab Server and it pretty closely matched my needs, so I decided to give it a go. The post is over a year old now so I will chronicle my experience buying the gear now, the cost and the setup.

So here are my targets, ideally Id like to get a 48 core system, but just a quick glance at some of those procs they seem pretty expensive. I want to be able to support 6 decently spec’ed VM’s, I’m equating those to D4MS Azure instances which are 4 Core, 16GB of RAM, which cost about $167/mo.

CPU 40 Core
RAM 128GB
HDD 32TB RAID 5

My goal is to keep the cost below $2,500 (which would be 1k above a fully loaded NAS). Which would start paying for itself after 14 months. As much as I can I’ll be trying to buy the parts new from Amazon, and then fallback to eBay when needed (for cost or availability).

If you’re a First Responder or know one check out Resgrid which is a SaaS product utilizing Microsoft Azure, providing logistics, management and communication tools to first responder organizations like volunteer fire departments, career fire departments, EMS, search and rescue, CERT, public safety, disaster relief organizations.

Go to Top