Proxmox to host Nethserver and NAS solutions

tessierp · April 14, 2020, 12:36pm

@Andy_Wismer
Following on our previous discussions which were going out of scope from the original subject.

Agreed with some of the points of your last post

I don’t mind paying a small amount for certain things but if I see an equivalent that is totally free with great support for home use, then, why would I pay? It is like this ClearOS vs Nethserver I mentioned before. Nethserver is an amazing solution. They sell it to businesses but for home use, it is FREE and the community is very helpful! Why would I pay for ClearOS then?

I heard Proxmox before and with you I managed to learn a lot more from it. And from my still very limited knowledge, again, why would I pay for VMWARE or use other solutions when this does the same as the highend ones but is free for home use? It really did influence my decision of how I will reinstall Nethserver and I will probably be installed Proxmox on the box I am currently using Nethserver on and create a Nethserver VM with it! Then I can backup the images to a NAS and restore as needed! So simple! But I will need to experiment on that part as I will need to figure out how to passthrough NIC cards to Nethserver, etc. But I need to decide on the mobo I will use for it. What do you think about this : https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U#Specifications

BTW, I did read that last part about consumer grade SSD and you’re right. Which brands would you recommend for a UNRAID solution?

Edited : Meanwhile looks like I found the answer to my question : WD RED SSD

Thanks again!

For those interested, here is the original post : After AD creation, network interface got bridged and unable to undo where it evolved into an entirely different subject

Andy_Wismer · April 14, 2020, 12:54pm

@tessierp

The board looks good, has the right amount of connections. I just have no experience with the NIC, but RTL is usually well supported. Proxmox has a Debian basis, so you could check if that NIC is supported in Debian. What’s the going price for these boards?

All the other Big Ticket virtualizations (Hyper-V, Citrix, VMWare) have VERY complex pricing schemes. I feel like standing in Starbucks and being hit over the head with their large billboard advertising their coffees…
And I only wanted a simple coffee with cream. Proxmox has a nice simple licensing pricing.

SSD for caching: I’ll need to look for my eval sheet i made about a year ago… I’ll report on this later. That looks good (WD Red SSDs).

(@pike)
UNRAID:
The difference of giving a time bombed OS / Software like UNRAID or the way Proxmox or NethServer does it, with no usage linits, but no unpaid access to “certified upgrades” is quite simple:
A time bombed OS must include a key, unreadable or unmodifiable by the user to function!
And that simply means: This is NOT open source!
Think twice what that implies!!!

I’m a user of open source stuff since 1997. I’ll admit that my personal workstation is a Mac, but I do have a VM Linux and Windows on it (Parallels). Professionally, I use Linux as my main server for my clients. Nowadays, my main Linux Server is a NethServer, with AD, Mail, Files, Print, Zabbix Monitoring and NethServer.
My position on open source is: Use it if it fits, don’t bitch about it if it hiccups. If you can, improve it and give back to the community. This last bit is what I’m doing right here: giving people pointers, tips, know how and the where with all to have a pleasurable and working experience with open source solutions.

I just love the hipocrisy, when a MCSE says Linux is peanut stuff, and the dude is holding an Android phone in his hands… What is the biggest phone system globally? Right, Android!

And what is Android, if not a little bit non-standardized Linux?

Yeah, it’s the ONLY Linux I know which MUST come with two built in Mail clients (Gmail & generic Mail client), locked CalDAV and CrdDAV libraries (You need to buy an App to use them, but Google Calender uses those libraries. There’s just no interface for a user to configure them…

But it isn’t built by the company which burnt billions trying to create a smartphone environment like iOS or Android…

Here are two “Best practices” papers i did concerning Proxmox, in mid 2017…
The models may have changed, but the principals remain…

Unfortunately in German, if Google Translator doesn’t cut it, I can provide an english version…
(I have english as mother tongue, but german is my second language (dad’s side) and french is my third human language.
Programming languages is something completly different

Using virtualization like Proxmox, with it’s fast snapshot and live backups function gives a big peace of mind when updates / upgrades / or an itchy coding finger makes the reboot of your server into a nerv-game…

Andy

tessierp · April 14, 2020, 1:13pm

It will be hard to translate the image from German to English but, since an image is worth a thousand words. I get the idea of what is being done. Basically, Proxmox gets all it needs from the NAS on the network. I see the use of Thecus, a Chinese brand I think right? That was my very first NAS, but bought a very low end model but it worked well for the most part. Again just to be sure, Proxmox is free for home use correct? If profits are not involved…

As for UNRAID, the use of a thumb drive key with the license does not make it true OpenSource you are right. That was my only preoccupation when I started to learn about UNRAID, the fact that the OS is on a thumb drive with the license, you need the thumb drive to start everything and then the OS runs from memory.

I did also consider other options like :

OpenMediaVault (if I remember correctly, the main developer of FreeNAS didn’t like how things started to go and left and started his own thing). OpenMediaVault is true OpenSource, supports VMs and Docker. The community is also very helpful there which is something I liked. I probably would not use the VM and Docker parts if I’ll start using Proxmox for VMs.
There is also RockStor. From what I read, Rockstor only offers support for NAS, but that also makes it more specialized unlike OpenMediaVault. Would just need to read on how well it supports AMD Ryzen and how well it works with non ECC RAM and if it would work with what I intended to do with UNRAID or OpenMediaVault.

As for the price of the motherboard I mentioned, it is about 300 Euros. It is expensive yes. And as far as I know, for Ryzen CPUs (just to be clear not threadripper), it is pretty much the only option to get IPMI. And I haven’t seen any addin cards that would allow me to get something similar…

Andy_Wismer · April 14, 2020, 1:38pm

Hi

Proxmox:

Like NethServer, Proxmox is completly free. There are three flavours of a commercial NethServer, encompassing Firewall, VoIP and GroupWare (NextCloud, Webtop, SoGo), which all three can run on one box. These need to be paid for.

But as with Proxmox: You can use both commercially (in your business or company) - no problems! And I mean the FREE version.
Both have support, Proxmox maintenence entails access to a tested, certified update repo, same goes for Nethserver.

I had a couple of Thecus as NAS. AFAIK, Thecus, QNAP and Synology are all taiwanese companies (ROC, not PRC). They have the same work ethic as mainland chinese…

FreeNAS origins

FreeNAS started out as a fork of the M0n0Wall firewall, developed by a swiss guy.
It is the same basis as PFsense, and also OPNsense.
FreeNAS changed a lot around version 1.0, if i recall right, that’s when the bickering started and OpenMediaVault was created (or forked).

RockStor

I have made a couple of tests with RockStor, it was quite satisfactory sofar, but they weren’t really heavy duty benchmarking. Rockstor is also completly free, if you want to support the project, you can buy an installer stick from them, or buy support. You could test RockStor on your board for a day or two, when you get it.

I’ll provide a translation of my papers by early evening (Time MET, it’s now 16:00 here…).

Andy

Andy_Wismer · April 14, 2020, 2:11pm

At the moment right now I’m working from home, my business partner is on site at a hotel. We need to migrate 4 relativ new Win10 Workstations P2V (Physical to Virtual). With Clonezilla, 4 machines cloned in 3 hours to NAS, one restored to Proxmox and now up and running. The only (slight) snafu is i forget the newer machines are EFI, Proxmox supports this, but you need to set the VM Options right. But the Win10 PC is now available as a VM!

Andy_Wismer · April 14, 2020, 2:18pm

@tessierp

Did i forget to mention that if you use Proxmox, a Linux Container is only a few clicks away?

A dedicted PLEX server, on a Linux Container. Or whatever you imagine, There are a LOT of templates on board, be it SugarCRM, SuiteCRM, OwnCloud, NextCloud and lots more.

Setting up a Linux Container is done in 1-2 minutes, meaning up and running!!!

I use containers eg for: FOG Project (cloning), PI-Hole (DNS Blackhole), ZoneMinder (VideoMonitoring) and a few other stuff…
These are really lean & mean!

Andy

danb35 · April 14, 2020, 3:13pm

The product that is now called FreeNAS is related to m0n0wall only in their common FreeBSD origins.

There was never a FreeNAS 1.0. There were versions up to 0.7.something, following which iXSystems bought the product name and applied it to completely different (though still FreeBSD-based) software, which was released as FreeNAS 8.0. The former product became NAS4Free, which then, more recently, became XigmaNAS (because reasons, I guess). The project that became OMV wasn’t so much a fork as a reimplementation of a NAS under Linux.

tessierp · April 14, 2020, 3:14pm

Same here, working from home. UEFI is what I intend to use everywhere now. It really is the way to go in 2020.

I was thinking about all we were talking about. So right now my Nethserver has 3 NICs. One for the WAN (RED), and two NICS for internal LANs (GREEN). If I will inside this in Proxmox, how will that actually work? What I mean is I don’t want to dedicate all 3 cards to just Nethserver otherwise that wouldn’t work for anything else. I guess I would have to create 3 VLans and get the traffic routed to a single NIC interface now and everything going in and out for every other VMs goes through there? I’m just asking because this is really a new approach I would be taking as opposed to having one OS to one specific machine… So if I have 3 NIC in that machine, I could dedicate 1 NIC to NethServer and Nethserver would have 3 VLans, and that leaves 2 NICs on the physical machine for proxmox and for something else if I want to. Or am I completely wrong?

Also, I did find an interesting solution to my remote management issue. I don’t really need a IPMI motherboard. What I need is a Remote Power Management unit, something like :

Some models are more expensive than others. I’ll have to continue my research.

Andy_Wismer · April 14, 2020, 3:52pm

Hi Dan

Now that you mention it, I recall the move to 8.0… long time ago…

Thx for the background info…

Andy

Andy_Wismer · April 14, 2020, 4:02pm

Hi

Was quickly out…

Proxmox and your NICs:

Don’t start building your house from the roof, start from the foundations…

In your case, this would be your Proxmox, not your NethServer…

Proxmox has / needs a NIC (at least one!) for use.
This becomes vmbr0 (It is in effect a Bridge).
The Bridge, logically inside Proxmox is connected to a Virtual Switch (called vmbr0).
To this Switch is where your VMs are connected. Your NethServer would use this connection as GREEN (LAN).
Proxmox is accessible on this connection by default.

The other NICs can be used for normal Ethernet traffic, or eg for PPPoE trafic for Internet.

Proxmox does NOT need to be connected logically to these LANs, it just passes traffic thru.
If wanted or needed, Proxmox can also be linked to these NICs (logically).

If you need several “virtual” LANs, these could have more (each has it’s own) virtual Switch inside of Proxmox. These virtual Switches can be mapped to a NIC (say as vmbr1 or vmbr2).

Note: This is all without vLANs. If you like using vLANs or need to, you’ll find Proxmox VERY flexible. Each VM can be vLAN enabled. The vSwitch can be vLAN enabled, and also Proxmox itself… But remember KISS, don’t build in complexity if it’s not really needed.

Remote Power: I use zWave Power Switches, controlled by my Home Assistant. Combine that with WOL, then you’d get the best of both worlds.
But ILO is still VERY cool, albeit rarely used, it the server runs well…

tessierp · April 14, 2020, 4:16pm

I guess I will have to produce a lan diagram for you to see what I have right now and what I would like to replicate with Proxmox.

zWave Power Switches, just looked them up, very nice. But then, I’m not sure if that would work well from the outside because the day I do get a Raspberry 4 with Home Assistant on it, it will sit behind the firewall, where I was intended to put my new lan architectures. But anyways, I’ll first produce the LAN schematics of the current and new one and then you tell me if that can be done if that is OK

Andy_Wismer · April 14, 2020, 4:25pm

Ever think about a simple polling job on the Raspberry, say every 5 minutes…

If xx is not working, power off, wait 1 minute, power on…

I didn’t know I was supposed to open the door for my butler, i always thought it the other way around.

As you said, a picture says more than a thousand words…

tessierp · April 14, 2020, 5:47pm

Yes that is indeed a nice solution, polling and turning on automatically. But I also like to have the option to shut it down or on with a Remote Management PDU but that could be done at a later time.

So I prepared some schematics (lunch time for me). I prepared it quickly so be gentle lol.

CURRENT NETWORK :

What I was thinking of going based on our conversations :

Let me know if it is unclear or does not make sense. The Proxmox box would have 3 NICs. I would like to separate the WIRED from the WIRELESS LANs. So I was thinking of dedicated 2 of the NICS to the Nethserver VM and to use the remaining NIC for Proxmox and I guess create a VLan to pass the connection to Nethserver who would, essentially, take care of assigning IPs to all my devices.

What do you think?

Andy_Wismer · April 14, 2020, 5:59pm

Hi

The schematics make sense and are understandable…

I’d like to warn you that separating WLan and LAN might not quite the idea you’re thinking of.
I used to think like that, until i started getting myself distributed speakers (No, not Sonos!).
At first I started using Apple’s Airport Express, hooked to active speakers. had about 4 of them around my appartment. The 5th one had to use wireless, as i didn’t have wiring in the bathroom.

That’s when i realised, making 2 networks out of my home network isn’t a good idea. At the time, apple (they’ve lost their lead in this inbetween) were well ahead of their competitors with Audio streaming. Licensing isssues made them insist on the streaming in the same LAN…

So it was either Wireless or LAN - or make both back into one network with wired and wireless parts. Consequentially a second WLan only for guests with no access to my LAN was set up on the OPT1 NIC of my firewall. I had that with a capture portal, and a ticket system…
(That came from the OPNsense firewall, and works very well.).

As I see (Plex and others) that media is one of the uses, I’d like to warn you before you buy stuff, and find out later why they won’t work as expected.

In your second schematic, the PC on the right would not be able to contact anything on the WLan with DLNA (Chromecast) and vica versa… Or Chromcast and Plex not communicating…

You CAN get around these issues, like for using mdns-router for Apples AirPlay or something that does that for DLNA…
But that’s often finickly…

tessierp · April 14, 2020, 6:04pm

Hmm. I’m curious. With my current setup it works. LAN 1 communicates with LAN 2 without any issue. Why it not work if I move everything inside Proxmox? Or perhaps I’m not understanding correctly.

Andy_Wismer · April 14, 2020, 6:06pm

The problem isn’t with pinging or web stuff or mail.
Network drives also work.

But anything to do with Media Streaming for home or personal use, like Apples Airplay would not allow that, DLNA / Chromcast can have issues, can also work…

It doesn’t matter which is the router / firewall.

tessierp · April 14, 2020, 6:11pm

I see. So PLEX and chromecast, I better keep that on the right side of the network and the second network for guests and have a specific router for that.

Andy_Wismer · April 14, 2020, 6:16pm

My home LAN:

And 17 years ago, in 2003:

Yeah, you didn’t want to have my electricity bill in those days…

But you can see in image 2 that the WLan was separated…

All images are vector graphics, all my own copyright. What looks as same PC / Servers will display different drives on a detailed view…
Windows, Mac, Linux, OpenBSD, Solaris, Novell Netware, Windows-CE & Newton, even IR-LAN connection for PDAs like Psion…
Details like IP and IPX matching (Simple Hex translation), but also the Old Appletalk protocoll integrated and matching network numbers - todays generation of sysadmins are oblivious to all that…

Todays network will be a little larger, but needs to wait until lockdown is over…

tessierp · April 14, 2020, 6:26pm

Damn!!! That was a hell of a network you had 17 years ago.

I like your current network, it is simple, you keep everything on 1 network. So you use your firewall in dedicated box? Any reason why you wouldn’t move it inside a VM? Out of curiosity, what is AWr7-PVE-4 which seems to stand on its own?

Having one network only is probably best. I am just concerned that, with intelligent devices, I would run out of IP addresses but that is probably unlikely to happen for a long time. Perhaps I should keep everything on one network for now, and if I really need it open the second network for guests (have a wireless router as AP).

Just out of curiosity, from what I explained, the way I intend to use my NIC cards, did that sound feasible / possible to you?

Andy_Wismer · April 14, 2020, 6:44pm

17 years ago, there were VPNs to my office, and my Housing I had at an Internet provider…
I tended to overdo things - and in minute detail.

Here in 2008 you can see SME-Server, the predecessor to todays NethServer.
Already downsized, but still way too big.

A short anecdote 4 weeks ago. My business partner, Karl has a Raspberry at home, and after 6 months i finally pursuaded him to flatten the old FHEM installation and install Home Assistant…
He had constant problems that about 20 minutes after booting Home Assistant for the first time, his whole home network would cease to work - no internet, no in house connectivity…

Three attempts produced the same result, so two weeks later, he still hadn’t installed Home Assistant. I told him, i can’t be, i’ve installed 4 in the meantime with NO issues.

Just for background information: Home Assistant involves a Docker installation inside a base installation of Linux on the Raspberry. After first boot, it grabs the latest components on the internet and reboots… Internally, between Mini-Linux and Home ssistant in Docker, the Network 172.30.30.0 is used…

No my friend Karl has - out of historical reasons - at home for the last 10 years exactly 172.30.30.0 as his home LAN…

And the Home Assistant takes about 15-20 minutes for initial install…

Sh*t, I thought. There’s no way to change the hard coded Home Assistant environment inside without screwing it up, so we changed Karls home network to 172.30.33.0 - then everything worked as expected…

Now, if Karl had a NethServer as his router / firewall, that would have badly screwed up his network, as NethServer with AD doesn’t easily allow an IP change. And using Home Asssistant would not have been possible. But with a separate box, that’s not much of an issue…

This is not one such story, I’ve learned the hard way, to let a specialist do his job, without additional headaches like a Linux Container / Jail containing AD inside a VM and adding complex routing, DNS and firewalling!

I can handle all that, but it’s error prone, flakey and a PITA (Pain in the A…).

A router and firewall are highly specialized components of a network. If you screw up your nethserver due to a typo, how’re you going to find a solution on the Internet when your router is down?

Another old Tip: NEVER use the networks 192.168.0.0 or 192.168.1.0…
Most boxes, routers or whatever you buy (ZyXel…) use one of those networks as default. So your home router would be the first box shot down due to duplicate IPs…