Some hints on proxmox please

Hello Andy

I was reading your support it this post.

And this caught my attention

Andy, how is that a problem?, besides not having a backup?
Isn’t this the default way to install a proxmox server? (A lot of disks, choose a zfs raid and configure external backups, and cross your fingers that never failed as a single server :robot:)

Reading twice, I think to understand that Murphy occurs here. Then, having good backups: is correct the way proxmox was installed?

By this I mean, I’m still looking for the time to create something similar to your diagrams.

  • The VM/nethserver stored in a synology (nfs? or what is better?, I appreciate the advice)
  • I want to have 2 proxmox : one productive (A) and other for emergencies (B)
  • Both using the storage of the synology for the running VM.

I think in this aproach:

  • For proxmox A & B, use the proxmox as storage for the VMs.
  • 1st Reinstall/prepare the proxmox B (with enough space to host the proxmox VM and maybe some additional vm. Just in case something goes wrong with the synology).

For the proxmox B

  • Put 2 big hard disk, boot the proxmox ISO, choose a RAID-1 in ZFS and that’s it, I appreciate the advice.
  • Configure the storage area in the synology (nfs?)
  • Move/copy that vm from promox-A to the synology share (backup first)
  • Make the promox B use the VM from the synology share/storage

If this works, reinstall the proxmox A in a similar manner (It has 4 hard drives)
Create the datacenter with both proxmox (need to learn how), then migrate the running VM from proxmox B to promox A.

And I have some doubts with the interfaces, this is the configuration, this is for proxmox A

auto lo
iface lo inet loopback
iface eno1 inet manual
iface enp10s0 inet manual
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.16.1
netmask 255.255.248.0
# this is our fortigate
gateway 192.168.16.99
bridge_ports eno1
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
# I have this interface disconnected in both proxmox
address 187.xxx.yyy.25
netmask 255.255.255.240
bridge_ports eno2
bridge_stp off
bridge_fd 0

and for proxmox B this:

auto lo
iface lo inet loopback
iface enp0s31f6 inet manual
iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.16.21
netmask 255.255.248.0
# this is our fortigate
gateway 192.168.16.99
bridge-ports enp0s31f6
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
# I have this interface disconnected in both proxmox
address 187.xxx.yyy.25
netmask 255.255.255.240
bridge-ports none
bridge-stp off
bridge-fd 0

In my tests if I turn off the VM/nethserver in promox A, and turn it on in proxmox B, nethserver works correctly. I guess that’s okay.

But I would like to know how to configure the second interface and not “burn” two static IPs (187.xxx.yyy.25); I think that using a static IP in both proxmox is wrong, in this case is the same (.25) but I don’t want to use this or a second, don’t have more IPs to spare.

This seems the right path: (I appreciate the advice)

Configure the second interface as an IP for a DMZ that is managed by my Fortigate, and when I need to publish some other external service (server: mail, dns) just create the policy to handle it (learning)

That is, if I install a server with any of these services, I could use an IP from the DMZ that will manage the fortigate. Then I suppose I should use an IP from the segment : 10.10.30.x and with this my virtual servers use IP like 10.10.30.2+


(just as an anecdote) BTW, I still have an old server with some usb storage connected, If I remove the usb, the servers goes boom every time!

Regards.

Good Morning Enrique!

Elleni, who had the problem, actually hard bad luck with brand new hardware: at least 2 defektive components…
One was one of the RAID SSDs, the other was a large external USB disk to be used for Migration from old ystem to new, and also as a first backup.
-> Really bad luck!

Your planned environment, as I understand it:

2 (newer) Proxmox Servers, 1 Synology NAS…
This means, that Backup is either in the same place as the running VMs, or on one of the Proxmox, also where VMs are running.

This IS possible, but not ideal.

As you know in IT/Computers, it’s never a good idea to backup data on the same disk.
Like eg in a Windows PC, the HD is partitionned into a C: and D: Disk. I’ve seen enough people who saved (backuped) their stuff to the D: Disk (Hey, it’s on another volume!) - without realizing that that disk is dead, both C: and D: parts are no longer accessible.

True, here we do have redundancy (possible for all 3 devices). Still, if a disk fails, the load (and strain) will be much higher on the other disks.

My suggestion for the available hardware would be as follows:

Both Proxmox are equal, each one alone can run all you need (or most, especially all critical servers/applications). This way each can be a failover for the other. Both have local storage for Backups and VMs.

If possible, the NAS should be “dedicated” to Proxmox duties. It can be used for other minor stuff, but should not be in heavy use by users or other, non proxmox stuff.

  1. If all VMs are using the NAS, if one Proxmox fails, the other can start the VM without any problems. (If a PC in a network fails, and the bookkeeper needs a XL file, he can still open that filr from ANY PC in that network!).
  2. If the NAS fails, both Proxmox have space (and current backups) and can start the VMs locally. The Proxmox VM / Backup storage need to be regularily synched or fed with data (Backups).
  3. Backup jobs would be defined alternately to use the NAS and Proxmox.

Worst come to worst, you may lose a day at most.

2nd part following…
Andy

1 Like

Sorry, I forgot to say: The backup would be out of proxmox & synology: an external usb drive connected in the synology and a second small synology for the backups. I just plan to have enough space on both proxmox to run the VMs in case the synology fails,

Edit 2.

Well, both are in production now, but I plan to reinstall both in reverse order, from B to A.
Proxmox A is more powerful than B.

Edit 3.
And after some weeks playing with our fortigate, I see now how the DMZ works. This for the use of some other VMS in proxmox;not nethserver related, and thinking twice, maybe some services in nethserver can be used inside our DMZ (to be analyzed).

@MrE

Part2…

You already have the (shown) NAS2… :slight_smile:

The Synology (here MYHO-NAS-PVE) should use RAID10 (Need to be set manually at setup). A Hotspare would be nice, but not a must! The FileSystem of medium/larger NAS by Synology use BTRFS, which is quite OK and in my experience quite stable (5 years not one issue because of FileSystem!).

The NAS should have 5 shares (At least where Proxmox is concerned):

  • PVE_Container
  • PVE_ContainetTemplate
  • PVE_ISOImage
  • PVE_DiskImage
  • PVE_VZBackupDump

These are shared with NFS to both Proxmox (And also the second Synology).

  • NFS Root permissions are needed on all shares.
  • Do not set the “Check” Bit when setting up the share - this cannot be removed, and can and will cause data corruption.
  • Use Synologys Hyperbackup to do Backups to: 1) The external USB 2) The other NAS.

Use RSync Server in Hyperbackup as target, not “other Synology NAS” option.

Set up a dedicated rsyncuser on both synologies with access to the shares.
-> The second Synology should also have all shares, it can be a (slower) stand in for the first, if defective.

The shares above are mounted in Proxmox (On the datacenter level -> Storage) using the names as follows:

  • Container
  • ContainetTemplate
  • ISOImage
  • DiskImage
  • VZBackupDump
  • VZBackupDump2 (This one points to the 2nd NAS!)

Part3 following…

3 Likes

@MrE

BTW:
How many NICs do your servers have each?
How many NICs do each Synology have?

Both have Two;

  • proxmox A is a Dell server a R320 (from memory)
  • proxmox B is a generic PC: 1 integrated NIC and 1 PCIe; this can’t run right now the VM of NS on the disk; I need to restore the backup on a synology and then run manually.
  • Synology have 4, right now I use one for the local lan (192.168.16.11); one for the dmz; other for “backup admin” but I can re-use, the last just in case something goes wrong; like some month ago when the synology greeted me with an amber light in the morning and I need to use the assistant.

That’s the reason to use the more powerful synology for both as storage to run the VM. It’s a RS3618xs (used as file server and a volume of 200GB for an IPcam to see the servers room)

Proxmox: 2 NICs in each Proxmox is great!
The NAS also both have 2 NICs?

For the IPcam, I would suggest setting up ZoneMinder (as an ultra thin LXC Container in Proxmox): It’s very comfortable - much better than Synologys Camera Station, and has very cool features!
Takes about 30 mins to setup, if first time, allow yourself 1-2 hours, easy!

1 Like

I read about this project many years ago, when the requirements were “huge” and we could not afford it. It’s amazing, that it can now be used in proxmox. I take note.

Part3:

Backupjobs on Proxmox would be defined to do Backups alternately on both NAS, say MO,WE,FR,SU on one NAS, TU,TH,SA on the other NAS.
Ideally, you can keep at least 7 generations of each VM (one daily).

DMZ Network:
In your case, this would be using vLAN. Proxmox makes using vLANs fairly easy.
I think the Fortinet takes a little more effort, but is also doable.

Cluster Network:
Not a MUST, but can increase performance a bit! Especially when migrating VMs. (90 second live transfer!)

Do you use an iPhone?
-> We could chat on Facetime.
I do not use anything by Zuckerberg, no FB, no WhatsApp or Insta!

If not, we can still use Skype…

Andy

Zoneminder:

And runs with 11 HD Axis cams as a LXC, not even a full VM!

Cut & paste a few commands from the HowTo into the CLI of a freshly set up Debian or Ubuntu LXC “Server”, and it’s ready to go!

1 Like

This synology can use 12 disk, at this moment 6 is are used in RAID 6.
BUT I can ask for 6 more disk just for this task and use RAID 10, and just for proxmox. Then create the 5 shares.
image

Noted

Time to practice!

Noted too.

As said, I can help with setup, we’ld need to use something like AnyDesk
(Not Teamviewer! They locked me out and want me to buy a PRO version, which I do not need…)

1 Like

iPhone is off-limits to me, I’m not a fan of its overpricing. And as you may have noticed, I am not a native English speaker. And I’m not in Zuck’s club either, well, a little, I use Whatsapp to communicate with my first family circle, only.

Even skype, since MS acquired it, I use it very little. I can install it, it still works my account I tried it this year.

But I do use Telegram. Now, for speed I use DeepL to translate from Spanish to English.

Regards and thank you for your patience!

I do use Telegram for Chat!

I don’t use Skype much, it’s only on when I’ve agreed with someone to use it.

My brothers family and kids all use WhatsApp, but I refuse to use Anything by Zucki.
He may be rich, but he doesn’t have enough money for my personal data!

So I am forced to use Skype to communicate with my Brothers family. They live in Singapore, halfway around the world from Switzerland.
Note: My brother uses iPhone and Mac at home! But his kids are forced by school to use Win10, even though I bought a Macbook for his daughter… :frowning:
And he won’t buy an iPhone for his kids… (Mind you, my brother is financially very “well off”! He had very good luck selling is Singapore Penthouse 5 years ago!). A bit of an “Uncle Scrooge”, to quote the Donald Duck’s Uncle!

My Spanish is bad to very bad. I can understand a bit - but what spanish speaking person speaks slowly - I think speaking slowly is not compatible with spanish or italian people… :slight_smile:

Jo no hablar espaniol! is about all I can write…

But I can speak/write English, German, French and a little bit of Italian (More understand than speak, and not writing, only reading).

1 Like

I appreciate your offer, you can’t imagine how much.

I’ll be out of the office and back until Monday (some family health issues, not CoV19 luckily)

I need to acquire those 6 hard drives for the 5 shares that proxmox requires. I think I only have 2 spare. I’m going to write an email to one of my colleagues to arrange for the purchase of the hard drives.

Everything you’ve written has opened up my vision. Especially as you show the network schematic.

Can you let me get the hardware I need, so we can follow up on your advice?

Regards

P.S.

We use UltraViewer at work because of this problem with TV.

Sure we can.
It’s always best to have everything ready!

Having a party at home and getting the drinks at the last minute - and then the drink supplier has a fire at the shop! That makes the party a bit Stressful!

Have a great weekend!

Andy

1 Like

Or more fun! :crazy_face:

A party with no drinks?

High and dry?

:slight_smile:

Oh! I thought it was drinks on fire. At the party, like this
drinks on fire


It’s almost 1AM here, my mind is falling asleep more than it should.