To build our DIY multi-tenant, multi-client “cloud” backup solution we’re going to need a “server” and storage. I use the term server loosely because this is anything but serve grade hardware. But since it serves the purpose of holding home backups and nothing else, it also needs to be cost effective. I believe I have found the right middle-ground in that, creating a “fast enough” setup which is also cheap enough to run.
This post is part of a multi-part article.
You can find the main index here
True server hardware, it is not
We need to have a remote server that is available 24Hr a day. But what is available and what are the things to take into account?
True server grade hardware?
I work with real server hardware in my professional work daily and while those machines are great, they are most certainly not cost, power or even noise efficient for home usage.
Desktop grade hardware?
So, next up would be desktop hardware. But even the cost of a decent CPU can run you upward of 200$, not to mention everything else you need such as a motherboard, memory, a CPU, case, etc.. So even that hardware, would be quite a big investment, even while keeping it cheap!
Getting closer but we won’t be able to match cloud costs easily with this still, can we get something decent cheaper?
Old hardware you might still have?
Old hardware? Sure, that can be used but it needs to have at least one USB3 port for how I intend to use it. But since this machine will be on 24Hr a day, power usage plays an important role. Here in the Netherlands we pay about 20 cents for 1 kWh.
A setup that uses 25 watt on average costs 43,80 euro to run here in the Netherlands. Running a machine that consumes 100 watt will cost you 175 euro per year.
Running an older server for one year is almost more expensive then a whole new mini-PC! So don’t overlook the fact of power usage.
Raspberry Pi or ARM?
I guess it could work, all the software can be compiled so should be available. But performance and functionality will definitely be limited. If you are looking at storing anything more then a few GB, I would look at buying something more powerful.
Buying a new Mini-PC
So I started looking at a relatively new mini-PC’s based on the Intel Apollo Lake architecture. This architecture is now available in mini PC’s in the form of quite affordable low-power 64Bit Quad-Core CPU’s which have at least a reasonable amount of computational power.
Looking around on Aliexpress I found the following two models, the PCG03 Apo and the PCG35 Apo:
*** I have since made a video about this machine, I have added it below!
These are basically the same PC with a few little differences. The PCG03 Apo comes with the following:
- An Apollo Lake 64Bit Quad-Core CPU
- 4GB DDR3 memory
- 32GB eMMC “SSD”
- Empty M.2 SSD slot
- SDcard slot
- 3xUSB3.0 + 1xUSB3.0 Type C
- Built-in Intel HD Graphics 500
- 1x HDMI + 1x D-Sub/VGA
- Realtek Gigabit LAN
- Intel 802.11ac WiFi, Bluetooth4.0
- Fanless cooling design
- I will be testing how well this performs and if adding a simple fan might be a good idea or isn’t needed
- Comes with Windows 10 Home edition
The PCG35 Apo has the following extra features:
- Upgraded CPU running at 1.5Ghz to 2.3Ghz instead of 1.1Ghz to 2.2Ghz
- Slightly faster GPU clock
- 2.5″ Slot inside of the chassis for HDD/SSD
These mini PC’s generally sell for ~160$ and ~180$ but the price fluctuates a bit. Click here or on the pictures for the current prices! I currently have not yet received this hardware but from testing in my virtual environment with a single core, performance should be more than sufficient! I have ordered the more powerful PCG35 because I want to test the hardware for some alternate purposes also.
The Mele PCG series isn’t for sale anymore, this is a good competitor which has the newest Gemini Lake chip! : https://geni.us/38TzGV
This machine can be overclocked!?
Yes it can to get just that extra bit of performance, check it out over here:
Windows or Linux?
Actually, the amount of memory (4GB) and 32GB eMMC are perfectly suited for running a little Linux box. With windows that it comes installed with (it can be used in this setup) you only have 9GB of free space which is a bit tiny. During my testing my root partition in Linux used around 5GB of space with everything installed leaving you with 27GB of free space.
Partially why this low-end hardware is sufficient is because of how the setup works. A lot of work such as deduplication and compression is done on the client so this little server only needs to ingest and hold the data, no processing needed!
But we need storage, and lots of it!
So the above hardware facilitates in everything our “server” needs, except for storage. But these little boxes don’t offer any way of adding (much) internal storage.
So the next part of the plan is adding USB storage. A while back I picked up an ORICO 9558U3, this is an external USB3 5-bay disk enclosure. It has no other features (such as RAID or cloning or whatever) besides presenting 5 disks to the OS through a SATA/AHCI-to-USB3 bridge, perfect for our needs and it works well in Windows or Linux.
On average this model goes for about ~150$, but sometimes you can find deals as low as ~120$, check the links regularly! Click here or on the picture for the current prices.
*** I have made a video about this cabinet and added it below:
But it’s USB!
Yes, but decently built. The chassis uses a built-in Jmicron controller and has a built-in cooling fan, tool-less disk trays with lock and an included beefy external power adapter.
I have been using one of these for a few months already and so far it’s been perfectly stable and with 5 disks it gives me about 200MB/sec to 300MB/sec of sustained transfer speeds, perfect for our application. Especially since our “backup server” will be behind a residential internet connection which won’t have that much bandwith anyway.
Combined, these two components form a “just add disks” setup. Looking at the costs so far we have ~160$ for the “server” and ~150$ for the external storage cabinet making a total of just a bit over 300$!
I challenge you to find anything remotely close to that price provinding the same functionality and performance!
Just add disks?
Yes, just add disks! But this is also where things get a bit more complicated or rather, custom per setup. As I mentioned previously, this hardware should have no problem addressing multiple tens of terabytes of data since it’s only running the storage back-end, scaling is no issue.
For myself, I’m looking at getting several 10TB disks. I favor the Seagate IronWolf (Helium version) series, but for more information about which disks to use, check the next article. There I will also discuss the different configurations you can build. There is several different paths you can take which will require different amounts of disks depending on need and features desired.
16 thoughts on “DIY cloud backup: Server and storage hardware”
Firstly, thank you for a great series of blog posts. Got me thinking and trying out stuff…
You said in the article: “I challenge you to find anything remotely close to that price provinding the same functionality and performance”
Challenge accepted 🙂
I have two suggestions, one is less and one is more expensive…
1. HP ProLiant Gen8 MicroServer – Celeron G1610T
It’s a bit old in the tooth but they still sell new ones for around 200€ which is I think excellent value for money.
4 bays for hard drives (+you can mod it a bit for an extra 2.5″ drive in the optical bay).
It has both an internal USB port and an internal SD card slot both of which can be used to install an OS so you can use the disks for storage exclusively.
It has a remote management network port (HP ILO; you need a license for that but they’re around 20€ on eBay) so you don’t have to go to your parents or to the basement when you need to set/fix stuff up.
You can upgrade the CPU and RAM (and use ECC if you need/want to).
The main drawbacks for me are it is probably more power hungry (though not obscenely so) and it’s let’s say not the quietest computer I’ve ever owned.
So both functionality* and performance would be better with this one for 2/3 of the price.
Or you can have roughly the same price with space for 9 disks (4 from the HP and 5 from the ORICO 9558U3).
*With one less disk slot
2. Dell PowerEdge T30 for around 430€
If you need a ‘slightly’ more powerful processor – this one comes with a 5th gen Xeon E3-1225.
I own the previous version – T20 and it’s an excellent computer for the price. It’s actually my main home server and the had specials before the new one came out so I bought a second one for around 250€ for testing and spares when the first one fails.
It has space for 4 disks oficially but it’s a fairly well ventilated enclosure so you can get more in (you do need an expansion card then as there are only 4 SATA ports on the motherboard).
Up to 32GB ECC RAM.
And it’s quiet. Really quiet. Until you stuff it full of disks anyway 🙂
I use Proxmox on mine which installs on ZFS if you wish which enables you to run everything you specified above and so much more (for the added cost of extra RAM of course).
So this one is more expensive (both outright and electricity costs wise) but far more flexible/functional though of course overkill for a backup server as the one you’re describing here.
Thnx and thnx for the comment!
Both great suggestions. What I suggested was very much a singular purpose design, to keep it as cheap as possible.
The microserver is an awesome alternative and a neat design, especially with being able to keep the disks inside of it! Performance wise I don’t know how fast the Celeron will be compared to the Apollo Lake processor, maybe the Apollo Lake would be a bit faster, but the Celeron would still be more then powerful enough to run as a pure back server. Too bad it makes a bit of noise, otherwise it would have been perfect and close in costs!
The Poweredge sounds nice but would become much more expensive. Awesome little machine if you are intending to do more with it then use it as a backup server but otherwise a bit too much overkill.
Proxmox is awesome too. I’m stilling a Ubuntu storage server and VMware machine at home but when I’m replacing hardware I think I’m going to build a single box with Proxmox.
Thanks for putting all your effort in this blog. It helped me a lot solving the crashplan nightmare which will happen next year when the friends option will be stopped. I like to mention that its quite easy to setup minio as a docker deployment. My XPEnology NAS with the latest DSM 6.1 on it does support docker pulls from docker hub.
SSH to synology and raise privilege to root (su),
Create a docker directory to store the data which is handled by the docker container. Eg. /volume3/docker/minio
then execute: docker pull minio/minio
followed by: docker run -p 9000:9000 minio/minio server /volume3/docker/minio
you can now access the minio webpage hosted in a docker instance / container on the synology nas.
So you can skip the hardware purchase if you have already a synology nas.
Great Blog and information. Appreciate you taking the time to put this together and maintain same. Bravo!
I will be following your lead and try to build something similar to your model. That said, I have a question. I read on a Mele re-seller site that the “Maximum External Hard Drives Capacity: 1TB”. Do you know if this a accurate?
Curious to know as I would like to configure in similar fashion to your set up, that is with multiple high capacity HDD’s.
Hey Scott, thnx! I have no clue why they would state that. Both internally using the M.2 or SATA connector or externally using USB3 there are no weird limits in place. I currently have 50TB connected using USB3 without issue. It’s just an Intel chipset and doesn’t have any limitations regards storage (or anything else really).
can you run consumer hardware 24/7 without any problem? what is the advantage of a server hardware then?
(Enterprise) Server hardware has the advantage of using higher rated components so failure is less. The solution I’ve built is fine for running 1 or maybe 2. But if you where thinking of running 10 or 20 the amount of maintenance for this setup would rise too much where more expensive hardware becomes logical to keep the amount of maintenance down. So for home usage and sharing with some friends, this type of hardware is fine, it’s been running rock-solid for a few months now! 🙂
Great write-up! I’m considering a similar setup (also for a ZFS RAIDZ2 array). Still happy w/ it? I’m considering thought to use it w/ 2.5″ HDDs – would you happen to know if the could be installed in the 3.5″ bays?
Is 4GB enough for running ZFS? The typical rule of thumb is a GB of RAM per TB of storage. I think this is mostly to avoid disk thrashing updating all the metadata that ZFS uses.
It really depends on the load of the server. Yes and No 4GB is enough for a ZFS server with 30TB of effective space. But since this is only being used as a simple NAS and S3 backup target it works fine. I’m not using any snapshots or other advanced techniques and when I log into the box even if it’s been up for 30 days it’s still only using 3GB of memory. Although it has a Gigabit connection and serves some CIFS over it for the 2 local clients, most traffic is under 100Mbit because it’s all running over the internet.
The rule of 1GB per 1TB is proven over exaggerated, that whole statement was never meant that way, but there is still merritt to having enough memory and ZFS does scale very well with more memory giving you better performance, so in that regard that’s true. But it still depends on what you use it for and what you expect from it, if I would have had 100 local users, no way this would be enough, but for it’s current usage, it works perfectly 🙂