Setup questions: Nethserver/VMs/RAID - Proxmox?

Support
1

Dear Community

I’d appreciate some guidance on how to properly build a new home server environment with Nethserver.

The background: I currently have a Ubuntu system which is about 11 years old now and has switched hardware a couple of times. All the configuration has been handcrafted but that means that the big upgrades between LTS versions are getting tedious. And it’s a 32-bit OS. So time to change. Having looked around a bit I concluded that Nethserver would probably be my best option as almost all of what I currently use is built-in or there is a module.

At this point I have a dilemma how to set it up.

My currrent system has a mdadm RAID1 mirror, and I’ve been happy with that as a certain level of protection (I could be doing backups better), so was happy to see that Nethserver can take care of that. But I also reckon that some of my tasks might be best run in a little VM on the side rather than manually installing things into the Nethserver. While looking into running VMs on NS I saw that some suggest running NS on top of Proxmox and having the other VMs alongside NS rather than within it. But then we have questions of how to manage the disks (RAID with MDADM is problematic? ZFS with its overhead?).

I’m happy running regular old-fashioned Linux systems but I have no experience (or very minimal) with virtualisation and containers and the new filesystem types and so on and don’t have the time any more to be learning a lot. So I’m looking for a relatively simple solution consistent with what I know. (Hence the current preference for a lightweight VM or two rather than containers for things like my own web pages and scripts)

I tend to reuse old hardware and expect to run the new server on an old laptop with 8GB of RAM and a 750GB HD to which I intended to add another 750GB HD for the mirror. Now I am wondering whether the Proxmox idea is a good one but rather than using a mirror I’d do better to run the system using a lightweight filesystem on one drive and use the other drive (maybe bigger than the current one) to keep snapshots and so on of the VMs. Is there a merit to that approach?

Given what I’ve described, how would an experienced reader suggest I set my system up? Now and again I have thought about putting the server in the cloud instead but there always seem to be benefits to having it locally.

I’m happy to answer any questions about the details to help refine the advice.

Thanks

1 Like
2

I’m not sure I can give a whole lot of guidance, but you can definitely rule out Proxmox/ZFS on that hardware–the “ZFS tax” isn’t nearly as bad as many make it out to be, but ZFS still likes its RAM. Proxmox with a different RAID solution could still work–or with a backend datastore. RAM will still be a limitation, of course, and I’d guess CPU horsepower will be as well. But with that said, Proxmox has worked well for me for a number of years, and I think I’d tend to agree with using a dedicated hypervisor for anything you actually want to use in production.

3 Likes
3

@ylavi

Hello Yitzchok

And welcome to the NethServer Community!

Like Dan above, I also suggest using Proxmox. Both of us also use Proxmox in our home environment too, like a lot of the “good guys” on this forum!

I provide for SME clients, about 20-30. All running on Proxmox, NethServer running AD, File Print, Mail, Zabbix Monitoring and Nextcloud. I also build the Home Networks of a few friends.

This Home Network is running for a friend:

Bildschirmfoto 2020-04-14 um 00.15.03
Bildschirmfoto 2020-04-14 um 00.15.031570×1588 403 KB

Proxmox is running on a HP Proliant Microserver G8 with 12 GB RAM, 2 mirrored WD Red Disks. These are only used for the OS, Storage is all done on the Synology NAS.
(16 GB RAM would be max, but my friend ordered a wrong brick or got a wrong one supplied by ebay and never got that solved out…).

Here, the firewall is, as is at almost all my clients, an OPNsense firewall, open and free as NethServer or Proxmox. Only, in this case, the firewall is virtualized inside Proxmox, meaning that must be running before I can reach the site externally. But this runs SO stable…

Proxmox, with live backups and snapshots can give a good peace of mind!

Note: I would suggest getting a box like the microserver with 8-16 GB RAM. They’re available eg on eBay for a reasonable price. And 5 years of use without headaches… :slight_smile:

Hope that helps!

My 2 cents
Andy

2 Likes
4

@danb35 and @Andy_Wismer, thanks for your replies

I understand that you’re both saying that I could possibly manage with this one server if I mirrored with something other than ZFS (if so, where can I find how do minimise the risks of VMs messing up the RAID1 due to known incompatibilities?).

And also that I’d be better off adding another box so that I have one for Proxmox with its OS and to run the VMs and the other dedicated to the VM storage?

It seems to me fairly obvious that the latter approach would be better than having one more powerful machine, and presumably also would allow more flexibility for changes in the future (migration of VMs from one host to a new one, or switching from one NAS box to another by having the VM host move the data over)

I’ll look into finding another machine. I might even manage with another laptop (I like them because they’re quiet and small and low-powered in the first place) as the current one, between the internal drive and the DVD bay and ESATA should be able to take three drives. But is it wise to have only 3 drives in a storage machine? I see that RAID5 is now considered unwise so do I need to aim for 4 drives with RAID10 (with a hardware controller too?) or just go without a RAID and be good about backups such as automated ones to the cloud?

5

@ylavi

Good Morning!

Beautiful sunshine for the last three weeks, local farmers are a bit worried about an upcoming drought period, like 2018 (A record dry year, in Europe), at least here in Switzerland and the surrounding countries!

BTW: Where are you located?
I’m in north Switzerland, right next to the german border town of Constance, on the lake of Constance.

Proxmox hardware:

Generally, I’d avoid using a laptop / notebook as a server, intended for 24*7 running. They CAN be used as a test server or such, but better not in production. Laptops sound convienient, come with a built in UPS and are easily portable…

But: They easily heat up, especially if doing Proxmox backups (Don’t underestimate the power required to compress a terbyte sized image file!). They don’t really give you the option of BONDing your NICs - using a USB NIC with 100 MB/s isn’t really a help!
Also, Laptops aren’t very energy efficient when running as “server”, and a load of stuff plugged in a Laptop negates it’s mobility, and can be a BIG issue if the disk is removed by a touching a cable…

You could start with just one more powerful box, like the HP Microserver mentionned above. They give a good price/power value, and last very long.
As starter, this single box could also be your storage, as Proxmox divides up the OS disk for small and larger storage, the small usually being 100GB, the large the rest. Both use LVM.

You would have the option of later BONDING the LAN Nics, or adding in a NAS at any time which suits your budget and planning. And enlarging your single node environment into a Proxmox cluster with a second node, also when it suits you.

Note:

I have about 20-30 SME clients of 2-50 people, all running Proxmox underneath and Synology NAS as shared storage. About 30% of these use RAID5 on the Synology NAS and this for the shared storage. All these Synologys have themselves an external USB3 Backup, so we are well protected in case of failures, cryptolockers and any other issue. Almost all clients have the same NAS at home, running as external storage.
In a pinch, say a failure during Xmas or New Year, when supplies are a lot slower, and so is the international mail / freight system, the NAS can be taken to office - and the Proxmox storage is quickly reconfigured, so we could be up and running in less than half a day, even if our shared storage goes out, and replacements are 4-5 days away…

-> I’d opt for getting a NAS for shared storage soon, at least a 4 bay or better, a 6 or 8 bay. Start with 2 disks (6 TB WD Red Pro are a good choice and also a good value!), and move up according to budget and need…

If needed, I can help you with setting this up!
You’re not the first I’ve helped here on this forum, and won’t be the last!

My 2 cents
Andy

6

The stupidest comment…
mdadm/RAID1 has to be mandatory for fault tolerance about disk. But it’s useless on a virtualized guest.
Let the host (Proxmox, virtualbox, ESX, whatever) handle the disk redundancy, deliver to the guest only the space you need.

7

@pike

A RAIDED VM does have use: It’s VERY effective at eating up resources on your Host machine!

:slight_smile:

But does NOT provide for any really usable redundancy or any other advantages.
(In other words, fully agree!)

My 2 cents
Andy

8

Hello ylavi

all work about RAID and this thing is on the side of VM Hosts. So you do not need it on Neth side. a very simpel way for a VM Host is XCP-NG and Proxmox. XCP_NG is very light needed leth tan 64 GB on Harddrive an between 1 and 4GByte RAM. RAID 1 support ist integratet…

to handel the server you can use xcp-center (sorry it is a windows tool)

This system is based on XenServer from Crytrix. And its running witth olde or smaller servers too…

https://xcp-ng.org/

Please start with 8.0 late than upgrade to 8.1.x

It is easy to handel I am usigng it more than 10 years (ncuding xenserver)
default is LVM (over RAIDX) to handel the VM HDD with LVM Snapshots etc …

The community is very helpfull like the neth community

If you like to install neth as a VM in XCP-NG use the centos7 template…

If you like to know more, be free to ask me :slight_smile:

But Proxmox is nice too :slight_smile:

9

You could use XOA instead; if you build it from source it’s pretty full-featured and can do just about anything xen-center can do. But I didn’t like that it didn’t have the management tools built in, in comparison with Proxmox.

1 Like
10

@Axel

AFAIK, Citrix XEN and the free XCP-NG can’t handle simple Linux Containers - they force you to use Docker instead… (I personally don’t like the excessive ports and networks Docker Containers implement).

Neither VMWare, XEN or HyperV can do full virtualization AND Linux Containers…
Only Proxmox has both built in!

The other three are still mentally in pre 2010 state! :slight_smile:

My 2 cents
Andy

1 Like
11

yes can handel container like AD in Nethserver

12

@Axel

Linux Containers aren’t shown here as a feature / capability:

and there’s no features to be found on their home page:

https://xcp-ng.org/

And: Docker isn’t equal to a Linux Container!!!

Bildschirmfoto 2020-04-23 um 13.01.57
Bildschirmfoto 2020-04-23 um 13.01.571854×1466 354 KB

My 2 cents
Andy

1 Like
13

That is true … for a bigger system like more the 2 Servers Proxmox is very nice…
but to let some VM fly XCP-NG is very nice … Most of my servers are running with XCP-NG most custumer had only one or two. and the Overhead is very small …

I dont like Linux container i had in the past a lot of trubel with secaurety , revisions …
but is nice too but not for my custumers …

but the world is better with mor nice solutions …

and Proxmox is a nice one …
and XCP-NG is mutch better now than Xenserver …

If you need Linuxcontainers use Proxmox or install a Linux Container Head in a VM …

But for my application Linunxcontainer are noch usefull … to mutch mix ups between version …

1 Like
14

@Axel

Linux Containers are IMHO much easier and performanter to use than Docker.
I use them mainly for simple or high cpu usage stuff, like ZoneMinder with 10 cams, 4 of them Full HD…

Proxmox has a very cool system for Linux Container Templates, making spinning up a Test-Linux (Centos, Debian, Ubuntu or whatever) a matter of 1-2 minutes!
AND it provides a firewall for the Container, if requested!

My 2 cents
Andy

15

Yes dan thats true I am Using Orchestra too, but sound like [ylavi] need a simpel and small solution.
I let XOA -> Orchstra running on a HP thinclient with 16G HDD and a external HDD for backup it handel a APC UPS. A scrip is stoping and starting the VM in ther rigth order… Hardware was araound 150€ including USB3 HDD with 4TB

XOA is not helpfull on a singel Sever :wink: …

But you ar rigth is a fine tool :slight_smile:

XOA/Orchsetra is a nice WEB base head for a small serverfarm …

But if you need more like HA Storage or HA VM thats mutch easyer with Proxmox …

HA is avialbel in XCP-NG too but with a differend way …

16

@Axel

Most of the VMWare / XEN / Hyper-V depend all too much on a Windows Server / PC for Administration. As @danb35 mentionned, Proxmox includes a full Web-Interface for everything.

Scripts for backups of VMs? You mean with XCP-NG I’d have to resort to hand scripts for VM Backups? I did that 20 years ago! Proxmox has live backups in 1-2 clicks, and the whole sheduling included!

My 2 cents
Andy

17

VMWare now has Web interface. Maybe not good as Proxmox but it has it…

1 Like
18

@pike

For a lot of duties in a VMWare ESXi cluster a Windows Server is still needed - to administrate a Linux box? I don’t quite understand that, but I DO know that MS bought in to VMWare’s parent company, and have people on the board… :slight_smile:

Citrix makes business with Windows Desktops, and so does Microsoft, so that’s clear there.

My 2 cents
Andy

19

Thats is no problem with XCP-NG …
but its not nice to export a VM from far with XCP-Center… It is very slow (by them self and with the download speed) most of my custumers uses VDSL 50 …
XOA/Orchsetra do that by them self with a some clicks.

No i mean the stoping in order by UPS with a external power fault. and restart in the rigth order when power is back.

AD/Ldap -> PBX -> Fileserver -> Cloudservices …

In some regions with a lot of green power in Germany the grid is a littel bit unstabel (like now wind and sun, no idustrial consumtions)

20

It’s essential, unless you want to manage everything from a Windows machine–and since I didn’t have a Windows computer available when I first started using xcp-ng, well…

No, XOA does that as well.