How to install Proxmox and setup a ZFS Pool

This article is to accompany my video about setting up Proxmox, creating a ZFS Pool and then installing a small VM on it. It’s a tutorial from start to finish!

Part of multiple articles

The article index for this 100TB 10Gbit server series can be found by clicking here.

Video

How to install Proxmox and setup a ZFS Pool + VM

During this video we’re installing proxmox, creating a ZFS pool and installing a VM, here are the parts in order with the video. It should contain everything you need to accomplish the same as in the video!

Again, main explanations will be in the video, below text is to go along with it!

Download links

• Proxmox ISO file     –     https://www.proxmox.com/en/downloads
• BalenaEtcher     –     https://www.balena.io/etcher/

BIOS Settings

Some BIOS settings need to be changed for Proxmox do be able to do it’s virtualization, mainly those are:

• SVM Mode – Enabled (Enables CPU Virtualization)
• Storage Boot Option Control – Disabled (Disables loading PCIe add-in card ROMs)
• Other PCIe Device ROM Priority – Disabled

Fixing Update Repository

By Default Proxmox is configured for the Enterprise repository, without a license you can’t use those so to reconfigure it to the community repository, do the following.

cd /etc/apt/sources.list.d/
ls

rm pve-enterprise.list
cd ..

After this, we are going to edit the  “sources.list” file and replace it’s content. You can delete lines in nano by using “ctrl+k”.

nano sources.list

deb http://ftp.debian.org/debian buster main contrib
deb http://ftp.debian.org/debian buster-updates main contrib

# PVE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve buster pve-no-subscription

# security updates
deb http://security.debian.org buster/updates main contrib

Save the file by hitting “ctrl+x” and choosing yes

Erasing partition table on disks

First we need to delete the partitions on the disks, we do this using a tool called “gdisk” which is installed by default

First look up the disks listed in /dev/disk/by-id:

cd /dev/disk/by-id
ls

Then, select the disks you want to clear and run the following command as you see in the video:

gdisk /dev/disk/by-id/xxxxxxxxxxx

There you can clear the disk by listing the partitions with “p” and then deleting them with “d”. Once you are done deleting the partitions, exit with “w”.

Creaintg a simple RAIDz pool

To setup a RAIDz2 pool use the following command (the amount of disks can vary to your needs)

zpool create NAME -o ashift=12 raidz2 /dev/disk/by-id/DISK1 /dev/disk/by-id/DISK2 /dev/disk/by-id/DISK3 /dev/disk/by-id/DISK4 /dev/disk/by-id/DISK5

To look at the pool status run:

zpool status

To turn compression on for the pool run:

zfs set compression=lz4 POOLNAME

Creating ISO storage

Here we create a dataset using the command-line using:

zfs create POOL/ISO

Video transcript

For people who don’t enjoy videos and would rather just read, here is the script I used while preparing for the video. It’s raw and not edited to be in an easy to read post form! If you find this useful, please let me know and I’ll try to post it more often!

Script

Hey guys, in this video we’re going to take a look at installing proxmox. This is going to be a bit more basic level tutorial video but I have some more in depth stuff planned for upcoming videos!

In this video we’ll run through downloading the ISO and creating a USB stick. Then we are going to look at some BIOS settings and after that run through the installation process together.

Once that’s done, we are also going to run through creating a simple RAIDz pool from 5x2TB disks. In a future video we are going to dive a bit more in depth into some more advanced ZFS Mirror pools with SSD caching.

In the last part of the video we do some basic setup stuff for a new proxmox server and to end we’re going to install a small Linux VM together!

I’ll make sure to include time jump links again, Ok, let’s go!

—— Intro

Right, so to get started we need to download the newest version of proxmox. Links to everything and command-line commands will be put in a blog post you’ll find in the description, please check that out if you want to perform any of these steps yourself! On the download page download the ISO file, we are going to use the 6.0 version.

While that’s downloading we also need a tool called “balenaetcher”.

Once both are downloaded and etcher is installed, insert your USB stick (Anything 1GB or more should be fine) and start etcher. Point it to the downloaded ISO file and then to the USB stick, hit write and wait. 🙂

You might get some partition popups, just click cancel on those.

Ok, stick done! Let’s go to the server and check some BIOS settings to make sure we avoid some little problems.

— start server and hit del like a moron

In the BIOS it’s probably best to load optimized defaults and then make the following changes:

First we want to enable “SVM Mode”, in my BIOS that’s located under “Advanced Frequency Settings” and then “Advanced CPU Core Settings”. There you just set “SVM Mode” to enabled. This enables certain virtualization features inside of your CPU which Proxmox wants to use.

Then in the BIOS tab you want to set “Storage Boot Option Control” to Disabled and also set “Other PCI Device ROM Priority” to disabled.

This disables the loading of option ROMs during boot. Since we won’t be booting from the network or any storage connected to the LSI controller, we don’t need to load their BIOSses while booting. If you do want to boot from the LSI controller you need to set the first option to “Legacy Only”. Setting it to UEFI can work but that means you need to do certain Proxmox stuff differently and I don’t recommend that right now!

Ok, hit F10 to save all changes and hit F12 to get into the boot override menu. There you should see your USB stick listed, now it depends on your system if you need to select the UEFI version of your USB Stick or not. In my case I need to, so I select “UEFI USB Disk 2.0 PMAP”.

A thing of note, a mouse could actually be easy to use during the graphical installer part, if you have one, make sure it’s connected!

Once booted you should be greeted with the following screen. There hit the first option to Install Proxmox.

Ofcourse we agree with the license agreement 😛

Then we get to the storage configuration for where you want to install Proxmox. Here we are going to make a few changes!

Hit Options and change EXT4 to ZFS (Raid 1). As you can see all the disks Proxmox detects are now shown and we want to select the SSDs of which we want to create a mirror and install Proxmox onto. Set all the others to “– do not use –“. Then we want to do a little tweaking in the advanced options.

Only change compression from on to LZ4. This shouldn’t be necessary but I like to be sure. Then we also want to change the hdsize value. These are 256GB SSDs but I never use the full size if I can avoid it. So in this case I’m going to enter a value of 200GB.

Hit ok and Next!

Fill in your Country details and once everything is correct hit Next again.

Set a password and enter an e-mail address. This address is important because the server will send you messages when something is wrong!

Select the correct network card you connected and make sure to fill in a correct FQDN hostname like you see here. Then also fill in all the IP details, it’s best to always use a static IP for your virtual machine host machine.

Once you hit Next you’ll see a quick summary screen and once you hit Install the installation starts!

Ok, all done, let’s reboot!

Once the installation is done, Promxox should automatically boot. If it doesn’t, check your BIOS that you have the Linux BootLoader or SSDs used selected as first boot devices. Since we are running a mirror, make sure to select both so that if one fails it will still boot from the other one!

OK, that’s done. Great! Almost done with all the basics! Let’s configure a few things more, but using the web interface and remote sessions!

First we are going to make sure you can update Proxmox to the latest version. To do that we need to enable to community repository.

To do this we log into the web interface, the address and port number for this are listed on the screen. The first time you will get a security warning, you can ignore this safely. After we proceed you’ll see the proxmox login prompt, there we log in with “root” and the password used during the installer.

Every time you log into this interface you will get a pop-up that you don’t have an official license. There are ways to remove this pop-up but I don’t really condone those methods. You can use the software without paying for it and they only give you a single pop-up when you log in. If you hate the pop-up that much, buy a license. 😉

Ok, once in the interface, the first thing we want to do is get the latest updates. But this doesn’t work out of the box, you need to reconfigure proxmox to use the community repository.

To do this, open a shell and choose xterm. This didn’t work while recording so I opened a putty session instead, basically the same thing. Just connect to the IP of the proxmox host and login with root and your password again.

There we are going to change the repositories available. All these steps will also be type out in a blog article so you can repeat them more easily.

Basically, we are removing the pve-enterprise repository file and then editing the default one to include the community version.

After that we can run a “apt update” to get all the latest information and then install it using “apt dist-upgrade”.

Once that’s done, give the machine a reboot and you’re done, updating will now also work from the GUI. To make sure the GUI reflects all changes, give it a refresh once the server is back up.

Ok, next let’s configure a simple RAIDz pool. I’ll be doing videos on more complex ZFS technical details and how to create more advanced pools but for now this is fine to start out with.

My disks have already been used so although they show up under “disks” I can’t configure a pool on them, let’s fix that. Also, although configuring a pool from the GUI now is available I’d rather still do it from the command-line. That way I have a bit more control over it and I also like to use /dev/disk/by-id links instead of /dev/sda sdb etc.. Using the /dev/disk/by-id links it doesn’t matter how you reshuffle your disks, even between controllers or systems, they will always work!

Ok, let’s first clear those disks. To do that we go back to command-line and use a tool called “gdisk” to basically delete all partitions from the disks.

These are my Samsung disks I want to use. Let’s erase them. To erase you look at which partitions exist using “p” and then d both those partitions and exist using w. And then we repeat that for all the disks!

Now if we check in the GUI we can see that all disks are available, but as said, I’d rather use the command-line anyway.

Again, commands will be listed in the description if you want to be able to read them more without the video.

I’m not going to explain the zpool create command all too much. Just know you should basically always use the -o ashift=12. The “rz2TB” is the name I’m giving the pool and I’m choosing a raidZ2 which uses 2 parity disks, so basically equivalent to RAID6 protection wise. I’m using old 2TB Samsung disks which have been in service for over 8 years now, but safe then sorry. 😉

Ok, zpool created and available! Let’s make sure compression is turned on for the whole pool. LZ2 compression is basically free on modern processors so it’s good to always enable it!

Ok, the GUI is seeing our new pool and everything looks correct! But it’s not available to use yet. You still need to add it as “storage” within Proxmox. You do that by adding it in the storage menu. I enable thin provision because that works best if you want to host VM’s on it.

Ok, all done and online! Next, lets create a place where we can put ISO files and make backups to install a Ubuntu VM.

To do so we create whats called a ZFS dataset on the command-line. A dataset is a sort of mix between a partition and a directory. If you are going to create directories with a specific purpose and want to maybe share them later on or set a quota on it, it’s good to make it a dataset.

Then we are going to add that dataset as a directory to Proxmox so we can upload our Linux ISO install file to it!

In storage we now add a directory, call it ISO and point it toward the dataset on the pool. Here we say we want it to house everything except disk images and containers and we’ll allow up to 10 backups.

Ok, go to the ISO storage and upload your favorite Linux ISO file, in my case a netinstall of Ubuntu 18.04LTS. I won’t go through all the settings right now, but we’re using the ISO image just uploaded and putting it on the boot SSDs for speed.

Ok, once created, click on the VM, 100 and start it. Then you can click the console button to view it. Ok, I’ll run through a very quick install of Ubuntu!

And well, that’s it! You’ve now setup a Proxmox machine, made it so we can run updates, created some ZFS storage and even installed your first virtual machine!

Hopefully this tutorial was helpful to you. I’m quickly going to continue using my server to migrate data from the old server to the new one! This was a bit of a “basic” video to get started, more advanced and in depth videos coming up soon! A like and subscribe is always appreciated, a share even more and I hope to see you back next time! 🙂

Concluding remarks

Again, this post if part of a larger series, click here to go to the index page.