I’ve started replacing my old (6 year) old server! So join me on the journey of putting together a 100TB 10Gbit Server, installing Proxmox, migrating the old data and many more topics!
Building a new server
My old server is now about 6 years old and really needs to be replaced. I’m using this opportunity to consolidate my current storage server, ESX server and another server into one.
Later I’m probably going to look at building a Proxmox cluster, partly to try it out but also to look at making certain Home Assistant domotica components highly available so that all the house automations like lights will always work, even if there is a hardware failure!
This article primarily serves as a textual accompanying article to my video series. Anything command-line or links in the videos can be found here!
If you have any comments or questions, feel free to comment on the specific video or on the bottom of this article!
Hardware used for the server
— All hardware components used can be found here–
The hardware used for the server is going to be based around a Ryzen 1700 and Gigabite AX370-Gaming-5. The reason for this is because I’ve upgraded my desktop to a new Ryzen 3900x setup so I can pass the “old” desktop hardware on to serve as my server hardware for a few years. The plan is to keep this cycle alive where I upgrade my desktop even 2 to 2,5 years and then the same hardware can have a second life in one of my servers!
There are downsides or rather complexities to using desktop grade hardware instead of server grade or Enterprise grade hardware. But with a bit of careful planning most of these can be worked around pretty well and it potentially saves you a lot of money re-using “older” desktop hardware this way!
The server will be housed in the Excellent Inter-Tech 4u 4420 chassis which has 5x SAS 6Gbps backplanes allowing for up to 20 drives and with a bit of modification (see the videos) up to 24 drives!
To run these I’ve added two LSI/Avago/Broadcom SAS HBA controllers which are running in IT mode. If you want to use one of these controllers yourself but it’s not running the right firmware, take a look over here for a good guide on how to flash it.
Videos
The videos are the main thread for this topic, if you have watched the videos and are looking for the command-line parts, look below!
New server chassis (Inter-Tech 4U 4420)
In this video I receive a new chassis and we take a look at it!
New Server SSD hot swap bays and cable length
I modify the chassis a bit, add some 2.5″ hot swap bays and talk about the build, trying to figure out all the cables and lengths needed.
What hardware do you need for a DIY NAS?
Let’s talk about the rest of the hardware, what am I using and why, how does the PCIe layout work and what do you need to take into account?
Building the server!
Let’s build this thing, including some in depth details and a timelapse!
How to install Proxmox and setup a ZFS Pool + VM
Installing Proxmox from start to finish, including setting up a boot SSD ZFS mirror, a ZFS Pool and installing a simple VM!
This video comes with it’s own article for the text bits, click here to go there!
ZFS considerations (Performance, RAIDz vs RAID and Mirrors)
Let’s talk a little bit more about ZFS, the difference of RAIDz vs normal RAID and how to scale performance with ZFS!
Migrating data using ZFS send and receive, importing a ZFS pool and creating a ZFS mirror pool
In this video we take a look at moving the data from the old server to the new and finally creating the ZFS mirror pool with the 8x10TB disks!
Changing all fans from Stock to Noctua including Noise and Temperature tests
I’m hoping that changing out the stock fans with Noctua fans will bring an improvement in sound level AND make everything run cooler, check to video to see how that worked out!
Seagate IronWolf 10TB Firmware bug
Next to the video, this part also has it’s own dedicated article!
Last server bits (Storage, Networking, HA-HA?
Shopping links
To keep the channel alive, buying using my affiliate links helps me out greatly. If you need any hardware or are looking for a specific part used in one of the videos, check out the following links.
Most links will ask if you are looking on Amazon or Aliexpress. I prefer Aliexpress myself.
(I have listed newer models then what I am using currently, if you have the money it’s best to go for a newer CPU and motherboard but the older might be cheaper)
Sadly the Inter-Tech 4U 4420 is currently only available on Amazon.de:
http://geni.us/0hzpX
Motherboard and CPU:
Gigabyte AX370-Gaming-5 – http://geni.us/mbE6
(Gigabyte AX470-Gaming-5) – http://geni.us/l4wzDsL
(Gigabyte X570 Aorus Pro) – https://geni.us/9aWi
Ryzen 1700 – http://geni.us/ARHouUh
(Ryzen 2700) – http://geni.us/UutH1gh
(Ryzen 3700) – https://geni.us/2z5Bh
Memory:
64GB Kit F4-2400C15Q-64GFT – https://geni.us/SjF9
Storage:
SSD – Samsung 960 Pro – https://geni.us/yTlXg5L
SSD – Samsung 860 EVO – https://geni.us/TZwHT
SSD – Seagate IronWolf 110 (Power Loss Protection, High Write Endurance) – https://geni.us/5S1Vj6V
HDD – Seagate IronWolf 10TB – https://geni.us/MuSpu8B
Graphics card:
Zotax GT710 PCIe x1 (ZT-71304-20L) – http://geni.us/rCeww
Network 10Gbit Equipment:
Mikrotik Routerboard CRS305 – https://geni.us/MpKA321
Mikrotik Routerboard CRS309 – https://geni.us/lW61Mn
Network 10Gbit Cards:
Intel x540-T1 (10Gbit copper) – https://geni.us/L9hlP9O
Intel x550-T1 Newer (10Gbit copper) – https://geni.us/mY8MK
Intel x540-T2 (10Gbit copper) – https://geni.us/Rdw8
Intel x550-T2 Newer (10Gbit copper) – https://geni.us/5Y8U
Mellanox ConnectX-2 (SFP+) – https://geni.us/LtHyGs
Mellanox ConnectX-3 (SFP+) – https://geni.us/tJ5SydD
Aquantia 1/2.5/5/10Gbit (Copper) – https://geni.us/9JcUw
SSD Hot Swap cage:
2×2.5″ in 3.5″ SSD hot swap cage – https://geni.us/yZtMr
LSI Card + Cables:
LSI/Broadcom SAS2008 based cards – https://geni.us/yOdyF
70cm SFF-8087 to SFF-8087 – https://geni.us/xJeS
50cm Forward SATA breakout SFF-8087 – https://geni.us/WoM1
50cm Reverse SATA breakout SFF-8087 – https://geni.us/2yj3Tc8
Other:
Very nice 1 meter 4x SATA to 4x SATA sleeved – https://geni.us/dUzZm
2 Fan PWM Split cable – https://geni.us/VEZqo5a
3 Fan PWM Split cable – https://geni.us/30vFSAJ
“Hook and Loop” cable ties – https://geni.us/jlkl
Screw driver set – http://geni.us/hStmKJj
(For other recommended tools see: https://quinled.info/2018/10/01/tools-and-equipment/#screw-drivers)
Concluding remarks
Hopefully you’ve found these videos and text bits helpful. Let me know if you are missing something or have a specific part you’d like a video or more information about.
Feel free to leave a comment on this article or below one of the videos, those are always very much appreciated! 🙂
Hi. How would you do this build with a quarter of the space (25tb) and sticking to 1gb nics?
I currently have a Ryzen 5 2600x , Asrock B450 Pro4, 16gbs ( currently) which I will bump up to 32gbs. Inside of a rosewill RSV-L4500. With 3 – 4 in 3 hotswap drive bays. My Primary usage is HyperV and plex.
Sounds like you have the specs pretty much under control!
Hi there! I’m beginning the homelab/server journey and I after many days of researching, reading, having second thoughts, I’m still having a lot of doubts, but I found your videos and this blog and I’m really grateful for your explanations.
Would you mind if I contact you through email (or reddit) to ask you a few questions?
Thanks for all the cool videos!
I am planning on watching them all and building a test system and after that maybe converting my primairy Windows Server with hardware RAID to FreeNAS with ZFS.
My only concern is that what to do when a disk breaks down and needs to be replaced. I am pretty much a Windows type of guy but i am really willing to give FreeNAS a try but i would like to be more comfortable when it comes to troubleshooting.
Will you by any chance make a few more videos in the future with regards to (general) troubleshooting?
Thanks again for all your hard work!
Hi,
I just upgraded my case and now accommodate 128,5 TB in my homeserver with a total of 25 drives.
WS2016 here, I moved from WHSV1 to WHS2011 and now WS2016.
Current setup is a gigabyte h370m-ds3h and intel i7-9700, 2 sas hba cards and a fractal design 7xl case.
I am still in a gb lan environment, what does the 10gb bring you ?
For me it saves me a lot of time transferring and working with large files on my workstation. My main workstation has no HDD storage anymore for years now, only SSD, so all bulk storage is on the server. I also live video edit from the server. Sometimes I do need some large files locally and transferring like 30GB over 10Gbit is much much faster then 1Gbit. 🙂
Really sorry to ask this, as I’m sure it’s been asked a dozen times, but I really like that case you built your system in.
I’m in the process of transferring my home server, which is in a Fractal Define R6 case, to something with hot swap bays and a server rack. I found the Rosewill on Newegg, but this case looks like it has much better build quality.
Where can I order this case from and have it shipped to the USA?