Setup Proxmox with Raid 1

Chat
,
1

Hi all,
I have to build a machine with different VM’s

  • Nethserver as fileserver and VPN server (1or 2 vm’s?)
  • 1 Windows 10 for Lexware and Elster
  • 1 Windows 10 for Star Money Business

Hardware

  • 64 GB RAM
  • AMD RYZEN 5 or 7
  • SSD’s

I have to build a raid 1, what is the best and not to expansive way to do this?

  • The Soft-Raid at the Mainboard?
  • A low priced Raid Controller? Which one do you prefer?
  • Only the VM’s at a ZFS-Mirror
  • Or a mix of all?

Thanks in advance.

2

On the Proxmox host? If so, just use the Proxmox installer for this. IIRC, you have the choice of a ZFS mirror (which would be my preference) or standard Linux software RAID1. See:
https://pve.proxmox.com/wiki/Installation

1 Like
3

Thanks for your answer. I think the easiest way to recover is to build it at the proxmox host.
Which advantages do ZFS has?

4

More than I have time or space to list, but here are a few big ones:

  • Everything has checksums (data and metadata), so ZFS knows if your data’s been corrupted. If it has a place to get a clean copy, it will repair any corruption. If it doesn’t, at least it can definitively tell you there’s a problem.
  • Snapshots are baked into ZFS, they’re near-instant and near-free (they take a tiny amount of metadata, other than that, the only additional space they take is the delta in the data). The standard .qcow2 disk image files do support shapshots, but it’s a much slower operation.
  • Data compression is enabled automatically.
  • If you have a ZFS system for backups (like maybe a FreeNAS/TrueNAS server), you can do them use ZFS replication for rapid and frequent backups using PVE-zsync (PVE-zsync - Proxmox VE)
  • Online pool expansion is trivial–if you need more storage, whack in another pair of disks and run something like zpool add rpool mirror disk1 disk2. The disks are formatted, brought online, and the pool is expanded, all in one step.
  • ZFS supports mirrors of arbitrary width–if you need more redundancy, you can set up a 3-way, 4-way, etc. mirror. Write performance would obviously suffer, though.
1 Like
5

I usually use 2nd hand Server HW with a classic HW-Raidcontroller for PVE, they are affordable and you can get good performance with it…

grafik
grafik2771×1421 542 KB

I suggest to setup a Raid10 (this is also suggested from proxmox staff…) with 4 - 8 HDDs or better SSDs (ext4 FS).

And for the Backups I use PBS (Proxmox Backup Server) with a RAID-Z variant (ZFS Raid). For offsite backups also PBS via OpenVPN connection from HQ to branch…

1 Like
6

@m.traeumner

Hi

I’d second DanB, as this is a fresh setup, make the most of it, and setup ZFS RAID (Mirror) using the Proxmox installer directly.

I’d also set a memory limit for ZFS, see here:

https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_limit_memory_usage

If not done, Proxmox will tend to use up too much RAM…

Read also the bit about swap, best BEFORE setting up!

My 2 cents
Andy

1 Like
7

fun because proxmox do not recommend to use zfs over an hardware raid

Do not use ZFS on top of a hardware RAID controller which has its own cache management. ZFS needs to communicate directly with the disks. An HBA adapter or something like an LSI controller flashed in “IT” mode is more appropriate.

I must admit a still keep away from ZFS pool, still use LVM and software raid (yes I can be old school)

8

@stephdl what I mean is:
PVE with HW-Raid and ext4 FS, ZFS only on the PBS-Backupserver…

2 Likes
9

Thanks to all for your replies.
I think I would prefer the ZFS-Pool, so I wouldn’t have problems with any Soft-Raid driver.

10

@m.traeumner

A wise choice…
Read about such cool features like zfs-send/zfs-recieve, zfs-sync, etc.
You’ll love it. And it’s fast!

My 2 cents
Andy

2 Likes
11

Thanks again for all tips. I set it up with zfs and it’s really fast. The Host System is Proxmox 7.
This is the Hardware:

  • AMD Ryzen 5 3600
  • 64 GB RAM 3000 Corsair DDR 4
  • 2 x M2 SSD Samsung 980 Pro (ZFS Raid 1)
  • Asus Prime B550 Plus

Guests are:

  • Win 10 with StarMoney Business for remote access
  • Win 10 with LexWare for remote access
  • Nethserver as DC, AD, FileServer, Nextcloud, Office
  • Nethserver for VPN
  • Debian Server with Apache and PostgresSQL for a webbased ERP (kivitendo)
5 Likes