VMware ESXi 7 vs Proxmox

Support
1

NethServer Version: Nethserver 7.9.2009 final

Hi,

I think I made a mistake. I have been looking for a solution for virtualizing my servers for months and it seems I made the wrong choice.

Unfortunately, no one near me deals with Proxmox, instead several people use the free VMware Esxi 7.0. Therefore, I installed VMware on the new server and moved all servers to the new one. These include the Nethserver, Home Assistant, media server with TV tuner recording and IP streaming, CCTV video surveillance system and others.

All VMs are working fine, but there is a small but serious problem. I can’t shut down VMware from the UPS in the event of a power outage, and we have a lot of power outages at random times.

The UPS is an APC Smart-UPS 1500 RM AP9617 with NMC. This was previously able to shut down the Nethserver using NUT in the event of a power failure. If I configure it in the Nethserver VM with NUT, the UPS can still shut it down. But there is a problem because this way I can only shut down the Nethserver and I can’t shut down the other VMs and VMware.

I found a solution by installing PowerChute Network Shutdown as a virtual machine and this would in principle shut down all VMs and then VMware as well. However, this doesn’t seem to work on free VMware. I get an error message when PCNS wants to shutdown the VMs and VMware and the shutdown process does not start.

In the PCNS error.log (/opt/APC/PowerChute/group1/error.log) I see the following error message when issuing the shutdown command:

ERROR Thread-30 com.vmware.vim25.ws.WSClient com.vmware.vim25.RestrictedVersion: Current license or ESXi version prohibits execution of the requested operation.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) - Exception caught while invoking method: ShutdownHost_Task
2023-07-13T16:54:11,296 ERROR Thread-30 com.apcc.m11.components.shutdowner.hosts.ESXUnmanagedStandaloneHost com.vmware.vim25.RestrictedVersion: Current license or ESXi version prohibits execution of the requested operation.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) - hostShutdown

Anyone using the free version of VMware ESXi with Nethserv and other VMs? Did you manage to solve the shutdown with the UPS in the event of a power failure?

Be careful, because it turned out that others did not test the operation of the shutdown and now found out that this could be a problem and everyone around me is rushing for a solution…

The title of the post is VMware vs Proxmox, because I’ve been looking for a solution for days, but I can’t find it, or they don’t answer on other forums. It occurred to me how I could solve the migration from VMware to Proxmox, but I got stuck with that as well. There is only one server that runs VMware and has a 32 TB disk array, of which 18-20 TB are VMs and data. I can’t even save it, I don’t have that many disks or storage space. I wouldn’t upload it to the cloud…

Can switching to Proxmox be a realistic solution in the midst of the above circumstances?

Problems also arise with Proxmox. Does the UPS work under the above conditions? The server is a Dell server Perc H730 with hardware RAID controller, how to install Proxmox without zfs using hardware RAID? Does it handle the Perc H730 controller at all, because I couldn’t find accurate information on that either…

Sorry for the long post, but I’m frustrated.

Thanks for the help and information.

1 Like
2

Me.

No.

But “most” of the problem (IMVHO) is in two points:
1: APC wants you to buy the network card. Or the network Software
2: VMWare does not like that much that you use it “in the pro-way”

So you can swallow down the APC VM for managing the UPS, or use a RaspBerry as a NUT Server, and use guests or host as… nut client. Before starting that route, consider to spend enought time to look for howtos and feedbacks about the integrated nut client of ESX, if available.

IMVHO… depends from the environment.
If you have ESX certified hardware, most of times Proxmox will work as a charme, but all tweakings and KB you know from your experiences will be lost in the wind.
Also… Proxmox is a “shell” around Debian with an Ubuntu Kernel… using KVM and LXC. It’s quite… “shifty”: updates are quite frequent, so “major maintenances” for version upgrade might be needed compared to ESX.

Moreover: Proxmox delivers separately the hypervisor (more than 1gb), the mail gateway (more than 1gb) and the backup provider (close to 1gb). ESXi latest iso is less than 700gb. Of course ESX supports a pinch of salt of devices compared to Proxmox, but something tells me: do I really need all that stuff?

Last but not least: it’s undisputable that Windows in various flavours works without issues in Proxmox. Not every scenario and software, however, is “Windows in Proxmox” certified, but I doubt that for Windows as ESXi guest.

ESXi 7 is still supported until april 2027. Proxmox is an hell of product, and I’m fond of it. Currently ESXi 7 is more reliable and with an enterprise timing, allowing to schedule a 3- month interval maintenenance having patches and vulnerability correction available.

3

@pike
More frequent maintenance and updating of Proxmox would not cause any problems. Solving the problem is almost impossible due to the large amount of data and the intractability of data backup.
The new server is a Dell PowerEdge T630 Perc H730 with a hardware RAID controller and I understand that it is VMware compatible. The old servers were two of them Dell PowerEdge R210 II, but one failed and the other was taken away after the new server was launched. I could no longer put enough disks in them and there was no RAID.

I understand you are also using the free VMware ESXi vSphere 7.0? Couldn’t you get the UPS to shut down the VMware host and VMs either? What happens when the power goes out?

I have read a lot about the VMware NUT solution, but it is not recommended because the NUT server installed on the VM does not work safely. It would require separate hardware and I would like to avoid that.

I don’t understand why the free VMware can’t receive a shutdown command from the PCNS VM or only the paid versions can?

4

Hi @steve

I think I have a solution for you: Cascaded virtualization!

Background:

I used to ba a VMWare ESXi guy myself, having used VMWare products including VMWare Server since before 2000!. I started to move to Proxmox after 2012.

A longtime client of mine still had 2 Novell Netware servers running, after the move to Proxmox I still had one server running VMWare ESXi just for the Novell servers. These were NOT productive, only there for legal / legacy reasons. But I did not like the server running just for Netware, and NetWare will not run in almost ANY virtualization except for XEN and VMWare.

When I found out about cascaded virtualization in an article describing virtualizing VMWare ESXi in Proxmox, I though I’d try it… That was a very successful test!

Notr only could I remove a server, but I got a free backup of the complete VMWare including both VMs running there.

Check the lower left of the network plan:

Bildschirmfoto 2023-07-15 um 17.42.21
Bildschirmfoto 2023-07-15 um 17.42.212758Ă—1628 1.04 MB

I’m here using VMWare ESXi 6.5, but the original document I read was using ESXi 7.x, what you have…

Since. installed, VMWare ESXi has been running more than two years - without a single issue!
I only use this for Novell Netware, but I could be a migration strategy for you, until you get everything running the way you need it!

Proxmox includes Backup, but has an enhancement called PBS (Proxmox Backup Server), also free to use. This is a game changer, allowing for incremental backups of VMs and Containers, but also has Deduplication - extremly fast and efficient.

I’d go for it - Proxmox running ESXi inside, for the duration of your migration…

My 2 cents
Andy

4 Likes
5

Hi @Andy_Wismer,
useful information but I think you misunderstood, sorry for my English.

The migration took place a month ago to VMware, but UPS does not work with ESXi. I am looking for a solution, but it seems that the free VMWare cannot be shutdown by PCNS.

I thought about replacing VMWare with Proxmox, but I can’t find a solution or information on some details there either.

I don’t know how to solve the migration due to the large amount of data and VMs because there is no other server and place to save. approx. 20 TB.

Currently, the best solution would be to manage VMware’s cooperation with UPS.

6

I did understand the issue, but I’d anyway suggest a move away from VMWare - people here know me as an advocate of Proxmox!

Maybe, but would entail purchasing a license for Proxmox (Very complicated, feels like a Starbucks menu when you really only want a simple Expresso! And - VMWare is fairly expensive!

I used ESXi with the free available NUT plugin, available at the time for ESXi 5 / 6, I don’t know if it’s still available.
This could shut down ESXi and all VMs from my NUT server, usually a simple Raspberry PI3.

Proxmox can run NUT itself, as client / server / standalone.

For my clients I have Proxmox licensed, using the simplest Suppport OIption: Community.
This gives me access to certified updates, and even better, this repo entails maybe 3-4 reboots / year, compared to the No-Subscription Repos Proxmox uses for the free version.
This costs only 100.-/year! (Well worth it, as I support Proxmox, ZFS and CEPH for Linux with this!).

Migration Hints:

You could use an external USB Disk to store the VMs, I use such USB3 Disks like Seagate Backup Hub with 8-12 TB as Backups for the Synology NAS, they would be suitable for the task.
Make a Backup from VMWare to the USB Disk, best would be remove VMWare Tools first!
You can’t remove the tools if migrated to non ESXi hardware (bare metal or other Hypervisor), but it works if you remove it while still on ESXi.

Proxmox can import these files (VMDK) , including MAC address and IP.

This means you can easily import all VMs to run in Proxmox!

You would still need the qemu-guest-agent installed on all VMs, this makes for very fast and reliable shiutdowns of VMs and Containers, another advantage is that Proxmox uses Microsoft’s ShadowCopy to backup Windows VMs!

If you need help or tips, I can provide assistance, you won’t be the first I’ve helped on this Forum!

My 2 cents
Andy

5 Likes
7

@Andy_Wismer
yes, I remember that you are a fan of Proxmox, in a positive sense. Thank you for your helpfulness.

I would like to solve the problem, but I’m very afraid of another migration because I don’t have the tools and I don’t know Proxmox. Unfortunately, VMware is not thoroughly tested either…

If I don’t manage to find a solution soon, I will have to seriously consider migration and think about how I can solve it, and this will unfortunately mean additional costs.

Thanks and Regards

8

Hi @steve

My offer of help / know-how / etc. still stands.

There are, of course several options…

What are you using at the moment to backup your VMs?
That could be part of a streamlined solution…

My 2 cents
Andy

9

Thank you very much.

I do not make a backup at the same time, because the Nethserver running on VMWare would be the backup of our server operating on the server farm. This is a small data max. 600 GB. The other data is critical, which is approx. 16-18 GB and irreplaceable information, works, documentation, developments, etc.
I can’t save them now, I trust the RAID until I solve the backup.

Coming back to VMware and PCNS, I see that PCNS can run a command on VMWare when an event occurs. I’m trying to test this now.
If I ssh into VMware from the PCNS VM and run it
/sbin/poweroff
command, VMware shuts down the VMs in order and finally VMware shuts down as well.
I wrote a script for ssh connection and running the command. I placed it on the PCNS VM and ran it. The script establishes the ssh connection, but the VMs and VMware do not stop either. I can’t figure out why it runs directly with ssh and why it doesn’t run from a script…

Content of the script:

#!/usr/bin/expect -f
spawn ssh root@192.168.1.190
expect “Password:*”
send “password\r”
expect "$ "
interact
/sbin/poweroff

Do you have any ideas for this? This might be off topic, but I’d appreciate some help…

Thanks and Regards

10

Solved it. I modified the script and PCNS can connect to VMware and shutdown it correctly.

2 Likes
11

Please share :wink:

12

@jfranco
Maybe my joy was too early…

I think you know this… Unfortunately, PCNS cannot be installed on VMware. javaAPI is disabled in free VMWare, so PCNS cannot stop free VMware.

But PCNS is able to run a script as a result of the events received from NMC, although this is not supported by Schneider Electric lately. There is no script in the PCNS source and it is not documented.

The solution was that I wrote a script that can be used to remotely stop VMware and set PCNS to run when the event occurs. It worked great during on tests, but there was a problem because there was a power outage early yesterday and VMware did not stop. It turned out that I restarted the PCNS VM earlier and the script setting disappeared from pcnsconfig.ini.

I have since tested it and but the setting is lost when the PCNS VM reboots. It is included in the PCNS settings but disappears from pcnsconfig.ini. I don’t understand the reason, but I think that PCNS does not support running the script so that it is not possible to stop VMware by bypassing the javaAPI.

At the same time, I am testing the available versions of PCNS and the configuration options to find a definitive solution.

I don’t know if VMware and Schneider will close this backdoor if someone publishes a solution for this. Maybe it has already been closed and that’s why it doesn’t work reliably?

If I manage to solve this problem, I will apply.

Regards

13

Hi @jfranco,

How can I send you a private message? Could you send it to me so I can answer it?
I have a little problem with PCNS and VMware, maybe you could help me test it…

Thanks

14

Hi Steve,

Have you thought about changing to Xen, Libvirt or Proxmox?
Besides these related issues, you’ll also have problems with backups!

Thanks,

15

Hi,

I used Xen many years ago, later switched to libvirt. Now I use VMware. Proxmox has limited support for hardware RAID controllers, so I chose VMware. But nowhere was it described or warned about what to expect in addition to the restrictions on their website.

Now it’s too late to switch to Proxmox, it would be a terrible job that I can’t solve now. I can save the important stuff from the VMs and take a manual snapshot from time to time…

16

@steve

Proxmox CAN run your whole VMWare ESXi 1:1 as “Nested Virtualization”. I have a client using this for two Novell NetWare Servers, needed for Archival / Legal reasons.

This works rock solid, and would allow you a step by step migration for your VMs to Proxmox (More efficient than nested virtualization!), and you get the advantages of Proxmox Backups of VMs.

Migration of the VMs sounds complicated, but actually, Proxmox can read and use VMDK files, so it’s mostly a case of removing the VMtools, setting um the MS MergeIDE utility and converting the file from the command line… Depending on VM disk size, and your hardware, an hour is not a bad estimate (Per VM!).

Proxmox can use hardware RAID controllers well, but works better with ZFS. I have several clients still using hardware RAID on Dell & HP servers…

To err is human.
To learn from them is a sign of wisdom.
Making the same mistake twice is a sign of stupidity
Doing it three times is confirmation thereof!

:slight_smile:

My 2 cents
Andy

1 Like
17

@Andy_Wismer
This may be off topic, I apologize in advance.

I borrowed two used 10TB disks from a friend and I have a used 500GB SSD. I installed Proxmox 7.3 on these.

I’ve never used Proxmox before, but I’m trying to move VMs from VMware to Proxmox and I’m just hoping I’ll have enough disk space. If it succeeds, the VMs will run on it for the time being until I install Proxmox on the Dell server instead of VMware and migrate the VMs.

But I have a problem. I only see free space on the 500 GB disk and I don’t see where the LVM created from the two 10 TB disks is mounted… All I see on the Proxmox interface is that there is no free space on the vg0 LVM.

I can’t find where on Proxmox to set the uninterruptible power supply for shutdowns due to power outages. It doesn’t have NUT, does it have to be installed separately?

Is it possible to configure NUT on Proxmox like Nethserver to communicate directly with APC’s uninterruptible network card?

Thanks

18

Hi @steve

As you’re the original poster, it’s your decision how far off-topic you want to go… :slight_smile:

Proxmox / Disks

Generally, it’s best to install Proxmox ONLY on the intended system disk(s), no matter if using real HW RAID or just simple disks or plan to use ZFS from boot on.
Proxmox will reserve 10% of that disk as LVM, the rest wuld be in LVM-Thin.

Proxmox uses XFS as standard (like NethServer) if you do not specify something else, like ZFS or CEPH!

LVM is for ISO Omages, Backups, ContainerTemplates.
LVM-Thin is for VMs and Containers.

Thin only allocates space, if actually used - this makes for faster backups, and less space used on the whole system.

Any other Disks, it’s best to format / install when Proxmox is up & running. You have the option of using ZFS or not, mirroring, whatever.

Proxmox / Nut:

Proxmox works very well with NUT, as server, client or standalone, it just does not have a GUI for this.
Proxmox can shut down all VMs correctly when being shut down via UPS. If no qemu-guest-agent is installed, this needs to use the BIOS of the VMs virtual board, not very reliable. Very rock solid and fast if the qemu-guest-agent is installed.

As Proxmox id fully based on Debian (Indeed, they’re one of the biggest contributers, especially in ZFS), installing Debian components is easy… instead of yum install it’s apt install… :slight_smile:

Here a some of my personal notes to this:

————————————————————
NUT in Proxmox (Als Client)
————————————————————

apt-get install nut-client nut-server

systemctl status nut-monitor.service

systemctl enable nut-monitor.service

systemctl restart nut-monitor.service

nano /etc/nut/nut.conf
MODE=netclient

nano /etc/nut/upsmon.conf
MONITOR ups@192.168.151.80 1 monuser secret slave

systemctl enable nut-monitor.service

Check server with this:

upsc ups@192.168.151.80

————————————————————



————————————————————
NUT in Proxmox (Als Server)
————————————————————

apt-get install nut-client nut-server usbutils

systemctl status nut-monitor.service

systemctl enable nut-monitor.service

systemctl restart nut-server.service



nano /etc/nut/nut.conf
MODE=netserver

nano /etc/nut/upsmon.conf
MONITOR ups@192.168.175.62 1 monuser secret slave

systemctl enable nut-monitor.service

upsc ups@localhost

————————————————————

If you have further questions not yet answered here, drop me a PM!

As you see, I’m almost always around to help people on the right road / track.

My 2 cents
Andy

BTW:
Proxmox released version 8 a few weeks ago, it can be upgraded any time yyou want!

2 Likes
19

Thank you @Andy_Wismer.

How will I be able to send a private message if I have questions about this?

20

Click my Symbol / Image, then Message… :slight_smile:
Left top of my post…

1 Like