Hi everyone, I wanted some advice regarding the configuration in replication of a neth7 server on hypervisor proxmox 6.3. At the moment I have a virtualized server on proxmox and it works fine, I wanted to make it available on my cluster using replication. Not having available a very performing hardware to host ceph, I thought a zfs replica on three new disks to be installed on the three nodes. Do you have any advice to give me and above all some configuration examples? Thanks
Hi Francesco
To make fast migration available, I use NAS as Storage for most of my clients (NFS in Proxmox). Some use dedicated NAS, usually Synology (At least 4 Bay) in RAID10. Some have just a generic NAS (Also Synology) and the VMs run from there. All this work well!
Fast Migration, in case it’s needed. OK, i can’t use all the nifty ZFS stuff, but fast Migration was required…
Create a dedicated network for storage (can also be used as cluster network).
It takes 90 seconds flat to migrate a NethServer or Windows Server VM from one Proxmox to another…
See here for a few ideas:
https://wiki.nethserver.org/doku.php?id=userguide:nethserver_and_proxmox 
My 2 cents
Andy
Hi andy, I already use a nas with share nfs (openmediavault) and I have 2 VMs in HA. I wanted to make nethserver replicate on the disks of the nodes to have sufficient redundancy. My nas is mounted on a minipc and therefore I have not conceived it as a redundant system at the disk level then raid … eccc. I wanted to install ceph but at the moment it would take a more expensive investment as I was recommended a 2.5 GB dedicated switch as a starting point, preferably 10Gb and related cards on the nodes.
Hi Francesco
I also plan for using CEPH/ZFS at home, but my two Servers (9 year old HP Proliants ML110 G6 with 16 GB max) just can’t cope with that. End of this year my budget should allow newer servers and switches to allow that scenario…
You need more than 32 GB RAM on each node to make such a scenario viable.
And for Backups of your VMs look at PBS (Proxmox Backup Server).
A Backup (incremental) of a VM (Windows/Linux no matter) with 1 TB allocated Disk takes 5-10 seconds!
Tip:
A Synology DS918+ with RAID10 (or RAID1 for starters) work well as shared storage with 4 GB RAM, more would be better. The price for those boxes are OK. They can be NIC-Bonded for better LAN performance.
My 2 cents
Andy
If your Proxmox has enough space (Not the standard 100 GB) allocated to local, you can make an additional backup there (daily). These can be restored to the local storage fairly quickly, in case your NAS gets screwed up…
PS:
On Synology, I let Synology back itself up to an external USB3 Seagate Backup Hub Plus (8 or 10 TB version). But I never let HyperBackup touch live VMs! -> This will result in corrupt Backups of the VM, as these can change while backuping, providing a non-consistant image. A waste of space & time!
My 2 cents
Andy
1 Like
Yes andy we seem to have very sumile cluster configurations. I have 3 servers node 1 is a xeon 32 GB node 2 i5 4650 16 Gb 3 node hp microserver gen 8 16 GB. I currently use vzdump and the differences are noticeable from pbs which uses deduplication and incremental. In fact neth7 is the only vm to which I perform the backup locally (100 GB in just 10 minutes on the local volume). I thought of making a dedicated nas with a xigmans distro that I use on 2 VM nodes and I would like to dedicate 2 boards for bounding increasing the bandwidth. At the moment the nodes use a dedicated switch for operations of move and more (link1). Installing pbs on VM is possible but it is not recommended for a matter related to the node and its failure.
1 Like
Hi
I do notice that you don’t or hardly use LXC…
You should try that. Load a few standard Templates (On shared storage). Like Ubuntu, Centos 7 / 8, Debian 10 and whatever you may want to play around with. It’s extremly fast to setup a bland Debian: less than a minute!
I would not put PBS together with a Node. Even in times of MS-Dos, you never put your Backup the same place as your Data. A lot of folkes put their Backups in those days on the D: Disk, not realizing that C: and D: were just partitions of the same hard disk. If one was dead, the whole thing was dead…
My 2 cents
Andy
1 Like
Yes hi Andy the LXC container they always use a little while left out because of the permits that in some operations it is difficult to fully control the car but anyway I’m very fast thanks
1 Like
This doesn’t need to be terribly expensive. A small 10GB switch is around US$120 new from Mikrotik, a used Chelsio T420 or Intel X520 is <US$50. SFP+ optics around US$18 each, and fiber patch cables US$10 each, at fs.com. If you don’t need too many nodes at 10G, it doesn’t have to be terribly expensive. Sure, it’s more than 1G, but not awful.
2 Likes
Thanks Dan, for the costs I have seen on several online shops and the prices are about what you say. But experience has taught me that it is better to wait a little bit and spend some extra money so as not to always find something to please. 
Speaking of LXC andy, I have had problems on lxc and linux especially when you have a credited linux VM on AD and ssh access and failed init group and pam errors. I don’t know if you have had the opportunity to try an AD accredited linux lxc and log in with a domain user …
Hi Francesco
I’d have to try that soon. I DO have a Debian 10 I use for remote access. I access the box with RDP. It’s a LXC on Proxmox. I’m using Mate as Desktop, as that works well with xRDP…
But did you have the LXC as “Privildeged”? That may be needed, for network access, etc…
My 2 cents
Andy
hi, andy i also use mate and debian … However on debian lx or ubuntu lxc the problem is the same. I tried with or without privileges. Let me know !I forgot, I was able to investigate the problem submitted to the permissions on the internet and in some posts they refer to proxmox and AD.
1 Like