A NethSecurity 8 Installation Attempt

Andy_Wismer · October 19, 2024, 12:43am

Hi All

I did - a while ago - say I would attempt a fair, neutral look at NethSecurity 8. I do admit liking OPNsense as Open Source firewall and still mostly use OPNsense.

Due to the almost unbeatable price for Unifis newer UCG-Ultra / UCG-Max firewalls I have started using these boxes. Even on “cheap” China-Boards / Boxes, and using OPNsense or any other Open Source firewall, I can not beat the thruput / price / features / size of these hardware.

As I am also a longtime advocate of independant firewall as a box, my first test of NethSecurity will be on a suitable box, here a PCengines APU box. These have a AMD G-T40E Processor running at 1 GHz, 4 GB RAM and 3 Realtec NICs and this model is equipped with a 120 GB SSD. I do have a set of the newer APU4d4 boxes, with a Quadcore AMD, and 4 Intel GBE NICs, but otherwise not much difference. The Quad core is a mite more performant, yes.

So, after downloading and flashing the live/installer image with BalenaEtcher and booting from USB:

First impression: nice
Hardware recognized as APU (upper left).

First BIG question: What is this?

Without any information, NICs 1 & 2 are bridged (even though 2 is not used or even connected…).

Really BAD !

It gets worse: 20 years after Microsoft and the rest of the world tried to stop people from using the stupid Pre 2000 Microsoft suggestion of using .local or .lan as a domain name, some still pushes this very bad security suggestion in using .lan as domain name.

Already plenty of poats in the forum of people with Mail, AD, LE and other issues due to installing their system on .lan…

Only A records, no CNAMES, PTR, TXT, SVR, MX or other records in DNS?

Sure I can use CLI to edit the config for DNSmasq - but I’ll be honest. I am a user of OpenWRT, which uses DNSmasq AND provides a halfway usable Web-GUI. NethSecurity, based on OpenWRT, still shows a extremly primitive, basically unusable interface.

The field “Name”:

Very clear information… Is the “name” field a remark or comment / notes field, or is the “name” something important - DNS is about Names after all…

Not good at all!

Next one: Threat shield

As the website already states:

Yet even here, nothing is clearly stated. All Maintainers are called “Community”. And all Confidance is unknown.

Of 50, none have anything other than “community” and Unknown Confidance…

Other projects at least display one suggestion…

During installation/ booting, I did recall seeing Wireguard installed. Where are any Wireguard VPN options?

Firewalling seems to emphasize the use of Masquerading in the NAT settings:

Masquerading isn’t used in the Docs, which talk about “Hairpin NAT”

And fact is: Hairpin NAT or Masquerading are only “stupid” workarounds for peple who do not really understand DNS or Split-DNS, the only correct way to handle this problem introduced by NAT.

The only mention of Split-DNS I could find in the docs:

People who do not understand the basic workings of the Internet (And DNS is one of the most Important of all underlying services) are just a risk for any other Internet operator or server admin. The risk for Spam and more just increase…
Bliss through Ignorance is a bit like the much cited Security by Obscurity…

Security by Obscurity

A typical one is changing the SSH port.

→ While this does result in less log file entries, it does NOT increase security in any way.

Today, it’s no more the 1990ies, any half-baked script-kiddie has access to Port-Scanners and can Google how to use them in a simple attack vector… Sheesh!

There are only about 65000 ports, any scanner with a current CPU will take a few milliseconds more to probe all ports for a ssh demon listening in the background. Code-wise? A one liner will cover it!

Updating System:

Since the recent release of NethSecurity project milestone 8.3, the update is a bit smoother.
No more fatal error on the screen but the nethSecurity box still reboots silently in the background.
This hardware does include a beeper, OPNsense has a GUI option to sound this for booting and shutdowns, a good option to have and use when available.

A silent reboot without ANY info is really uncool. It’s 2024, provide feedback on what the box is doing!

Conclusion

Sorry guys! While I’m still fully behind NS8, NethSecurity seems not quite at the level…

Not only is NethSecurity very lacking in Key functions, “sensible” suggestions are also very lacking, like which Threatshield lists are better or not (At least suggest one!). Others (competitors) do!
But the fact that NethSecurity still endorses very old strategies and mindsets is generally not a good thing for the Internet.

While I happen to have a good comprehension of how DNS works back and forth, I am aware that for most others DNS is a sealed book. But DNS strategies should be at least mentionned and important ones like split-dns emphasized.

The fact that I will find wrong hostnames or fqdns in logfiles when trying to solve issues is often due to the fact that CNAME can’t be used from the GUI, neither can incorrect PTR records be purged or corrected. Most of these stem from the fact that several A records are used instaed of a single A record and several CNAME records, resulting in a single, correct PTR record.
Having to additionally manually sieve out these false records when comparing logfiles is additionally error-prone, a less than ideal situation for any firewall operator!

Summary:

Good:

Clean GUI
Fairly responsive GUI
Correct hardware identification

Bad / Stupid presumptions:

Unasked for, unlogical “bridging” of two NICs
Unfinished look and feel for Threat Shield, lacking preset suggestions
Use of .lan as default domain name
Non working LetsEncrypt options if set up using a .lan domain…
Missing WireGuard options
No usable DNS (which doesn’t give back wrong information.)
GUI and Docu are in several places “out of sync” or do not contain any usefull information, like “Name” field in DNS.
Fast, silent reboot without ANY information when updating.
Locking down LDAP or AD as a data source (for example a typical use is for VPN Road Warrior use) in a Open-Source firewall is just not cool, nor really acceptable nowadays. NS7 did have “Account provider” access built in, a major “downgrade”.
The fact that an optional Phone-Home for Nethesis to get user data and stats and win additional clients by using this info to provide a free per device stats interface for users seems like a missed opportunity to me
Too many key services of NethSecurity 8 as firewall are dependant on external services. It’s OK for me if we are talking about updated blacklists or Antiivirus signatures. If not current, a day old, OK. But DPI to use Netify? Other firewalls can do self contained DPI…
Same eg goes for the Hotspot. To use a Hotspot, even if I explicitely do not want or allow Social Media logins, I still have to setup an Ikaro service in the cloud somewhere, or use Nethesis own Ikaro server. OPNsense handles a Hotspot with Ticket system all internally, self contained. Such reliance on external services would already be a showstopper for several client use cases I have, as the server is isolated and serves an isolated network (Eg a school).

At the moment, I do not deem this product really usefull for any of my clients or myself, neither for Office, Home or Lab use. I will keep an eye open in the near future, and probably keep a VM running to play around, but I will not be dedicating hardware at the moment.

A small note to using hard-coded colors in firewalling: The Red / Green basics:
It is a true and well known fact that most people in IT and Networking are male.
It’s also true, but much less well known that color-blindness is a mostly birth deficiency (Not intended as any form of insult or disregard) in males…
And color-blindness mainly affect green and red…
Depending on race / region, the values for infliction can reach 20% of males…

In traffic lights, they (color-blind people) can help themselves as the position of Red and Green are always top or left for Red, and bottom or right for green. (Only one known exception to this rule, and that’s the light in new York, Green on Top!).
On a firewall table, not seeing green and red priorities is a major disadvantage. And not being able to choose more appropriate, visible colors for themselves makes it doubly worse!

I am myself not inflicted with color-blindness, but in my professional life I have often met up with people having this issue…

My 2 cents
Andy

Note to all: I have actually done this installation to SSD, but some screenshots were missed. So redone everything again, but only as “live” system for documentation purposes.

NLS · October 19, 2024, 8:54am

Thanks for the analysis for all of us that haven’t tried it yet.

For me any system that targets into taking the place of the middle-man (I don’t even say gateway), between LAN and WAN, that doesn’t implement a PROPER FULL DNS service (and as we are in 2024, also in a friendly GUI), is a show-stopper for me. This is an ongoing discussion since the days way before NS8 (and NethSecurity), I really don’t understand why not embed one of the very nice FOSS DNS server solutions out there.

transocean · October 19, 2024, 11:44am

Hi @Andy_Wismer

Thank you for this test and the detailed reporting. I had already been toying with the idea of refueling my OPNsense hardware with Nethsecurity and then having quality from a single source together with NS 8. I’m not going to do that now.
I also have the problem with red and green. But it’s not because I’m colorblind or have any other eye problems. But when I look towards Berlin, the red-green problem is also noticeable.

Greetings from Hessen to Switzerland…

Uwe

dnutan · October 19, 2024, 12:12pm

It appears to be that some info is collected but do not know to what extent:

Andy_Wismer · October 19, 2024, 12:24pm

Yes, but it seems this info is restricted to those with a subscription…

oneitonitram · October 19, 2024, 12:53pm

Very good and critical feedback for the growth of the Nethserver platform.

MAybe the developemtn team could work with you, basd on the feedback you have provided to build NS into a more robust solution.

We understand you have Almost my age, as indutry experience and knowledge in implementing firewall systems.

While i understand and beleive soem fo the feedback will b implemented by the developemtn team.

am sure some may not, because

either they are trying to e too simplistic for the not so dvanced user, comapred to soemthing like opnsense and others.
Bebacsue mabe some industry trends are changing and the likes.

in relation to wiregard interface. I am not sure why it kees getting igonred even in nethsecurity, as it was ignored in nethserver,and a 3rd party module had to come to the rescue.

if its underlying in the base, just support it in the backend.

Andy_Wismer · October 19, 2024, 1:07pm

Wireguard may still have it’s issues, IPsec and OpenVPN handle Provider-Failover a mite better than WG, but that’s about all the caveats. And the GUI (On some platforms) will not allow 2 VPNs running to 2 different targets with WG, IPsec and OpenVPN have no issues with this.

But, comparing to IPsec and OpenVPN running on the same hardware, Wireguard get’s about twice the throughput - and that’s a lot. Much simpler code, good encryption and very fast.

Even in Italy, you CAN race a Ferrari against a classic Piaggio Vespa…
But it won’t be fun, interesting or fair - just boring…

Yet, internally, WireGuard is now KING. At least on NS8 - and it seems on NethSecurity 8.
So why no GUI?

OpenWRT does have a GUI for WG and for DNSmasq…

My 2 cents
Andy

filippo_carletti · October 19, 2024, 1:17pm

Lack of developers. We’re working on it.

Andy_Wismer · October 19, 2024, 1:34pm

This is at least a Statement - understandable and acceptable for most!

Developers are hard to come by, they don’t grow on trees (Except maybe in India… ).
And really good developers are even rarer!

Thanks for the feedback!

My 2 cents
Andy

oneitonitram · October 19, 2024, 3:05pm

you could outsource some from Kenya

dnutan · October 20, 2024, 5:43pm

25 posts were split to a new topic: A NethSecurity 8 Installation Attempt - OFF-TOPIC discussion

Niwre · October 20, 2024, 8:35am

@Andy_Wismer Thank you very much for this constructive report, I really appreciate your opinion. And to be honest, I envy your extensive knowledge, not only in the DNS area but in the entire IT sector, you simply have years of experience.

Regards
Erwin

oneitonitram · October 20, 2024, 2:20pm

Point of correction, unless you are using the cloud version. of erpnext, which in itself is also opensource, you have no any fees for ERPNEXT.

its one of the most truly free opnesource ERP solutions, even more free than DOlibarr.
the challenge. is that, since is equall to SAP, there is alot to learn.

transocean · October 20, 2024, 2:31pm

Joke:

Do you know what SAP means in German?

Writing on paper (Schreiben auf Papier).

giacomo · October 21, 2024, 12:24pm

There are some info collected and sent to the remote server.
I’m not sure to get what is the missed opportunity.
If you’re speaking about public stats, we are working on it.
@Tbaile is trying to setup some dashboard but it’s a low-priority work that we do on the spare time.

Tbaile · October 21, 2024, 2:32pm

Thank you for the through roughly inspection Andy! These don’t usually come by.

As Giacomo stated I’m working on a full Grafana dashboard for all the info of the phone home that will be public, a first release is almost ready. Still it’s a low prio, so it gets the time it gets

At the moment, here’s the dashboard:

I’m thinking on adding an additional section for hardware statistics and to add a whole section on what firewalls do (features activated, controller status, and more) but it will wait.

Also, throwing the bone out there, I’m looking in my free time to do some time-statistics over on how installations change over time, this will help a lot to figure out the trendings in both hardware used and how software is used.

Andy_Wismer · October 21, 2024, 3:03pm

Hi Tommaso

A good idea, and many thanks for the feedback.

I did say - a while ago, in the sense of NS8 Final - that I do trust our devs!
I still do!

My 2 cents
Andy

Great Sofftware can be like a great City…

When Romulus and Remus thought they were finished with Rome, it did not have a Metro.
It does now (actually for about offhand 50 years now…).
At least being an “eternal” City also entails “eternal maintenence” and upgrading.
And always well worth a visit!

izuky · October 23, 2024, 8:29am

I respect your expertise in OPNsense,
I’ve been installing and maintaining NethSecurity firewalls for 5 years now, across nearly 200 diverse environments, so I feel my experience gives me a solid foundation to express an informed opinion on this product.

From version 1.5.0 to 6, 7, and now 8, I’ve seen it all.

Like you, I am not paid by Nethesis, OPNsense, or any other vendor. My opinions are my own.

I also read the other thread where you were talking about system stability.

I can guarantee that in terms of system stability, I have never had any problems other than hardware.
The APU Alix boxes that you also use, it happened in some models that one of the network cards would burn out or the power supply, but I never had a system crash.
One of the many firewalls:

Screenshot 2024-10-23 092851

NICs Bridge:

If you try to install the system on a server rather than in a box, you will realize that the order of the network cards (previously CentOS, now OpenWRT) is not always correct. So you find yourself searching for the correct network card at first startup. With all the cards in a bridge, you connect the cable to any port, log in, and identify them by their MAC addresses. Next you can delete the bridge easly.
I find this very useful.

Domain .lan:

You are absolutely right; I agree, they should change the default domain. It’s also true that changing it takes half a second, but if it had a different name by default, maybe less experienced users would leave it as is. (But do newbies really install firewalls? I hope not.)

Threat Shield:

Community lists do not have confidence levels. But excuse me, how can you give a confidence level to a community list? It seems more than risky to do so. Would you trust a list of public IPs made by third parties for free?
If you subscribe to the paid lists, there is indeed a confidence level.

This is to say that if you want free lists, obviously they can never tell you if they will be well maintained or not. If you buy them, there are companies paid to do this job. Who would bother to do it otherwise?

Wireguard:
They have added WireGuard to the image, but if you read in the “to do” it is under development the GUI to support it.

LetsEncrypt
Let’s Encrypt doesn’t work on .lan?
If you register a subdomain that points to the public IP of the firewall and request the certificate, it works perfectly. I don’t understand the connection to the FQDN."

LDAP o AD
Am I missing something, or are you perhaps judging without knowing the product?

DNS
You’re talking about split-DNS. But in large companies, do you really entrust this task to the firewall? I don’t work with any large companies that need split-DNS and don’t have Active Directory.
And no, they don’t use the firewall as a DNS server.

Or, since you mention TXT, CNAME, etc., are you perhaps talking about hosting infrastructures?
Forgive me, but Nethsecurity may not be the right product for these kinds of infrastructures, but neither is OPNsense… We’re talking about different levels here. I sincerely hope they use truly serious products, like Palo Alto (look at who Google NGFW or Amazon NGFW rely on… Palo Alto, of course).

Security by Obscurity
I don’t understand the example of changing the ssh port.
They give you the possibility to change the port but this does not mean that they are telling you to do it for security reasons.
In fact you can set a key or set the 2fa.
Also, the threat shield module monitors bruteforce requests to the ssh service as you can see:

I’ll stop here because the conversation would become a debate and that’s not what I want to do. Do you want to know what I think instead?

I think what is really MISSING, and as far as I’m concerned I consider it fundamental for a firewall distro are:

The possibility to configure 2fa ALSO for ssh (I would not leave it open from the outside anyway but in very particular situations it might be necessary, for example for an ssh tunnel)
Threatshield control (ex failban in NS7) also for the Nethesis UI, not just for luci or ssh
A LOCAL IPS, which does not depend on external services such as the threat shield, a package which in my opinion is fundamental. From what I understand they are implementing it
keepalived or ucarp. I know that keepalived is in development and will be released soon. For me it is essential to be able to make a physical firewall redundant in critical installations

giacomo · October 23, 2024, 12:32pm

First, thanks for great feedback @Andy_Wismer and @izuky !

To access the feature you need a subscription.
Like on many Open Source product, connection with remote LDAP/AD is often not enabled on free version: it’s usually a feature required for companies and we hope they can pay few bucks a month to support the project

Sadly dropbear does not have it. And to enable it on OpenSSH, I think you need a PAM module which is not present inside OpenWrt.
As an alternative, I can suggest to open SSH only from the VPN zone then use OpenVPN as first factor and SSH itself as second one.
Still, nor clean nor easy

Yes, we hope it will be part of next major release.

Even this one is planned, but not in the upcoming months.
Maybe we can think to include first the required daemons without UI nor configuration scripts.

Already released: NethSecurity project milestone 8.3

Andy_Wismer · October 23, 2024, 6:35pm

Hi @izuky

Thanks for your valuable input.

NICs Bridge:

NethSecurity was built-in NS7, and swiss SME clients generally prefer a dedicated box, so no real option to use or test this - with a few friends or my own 2nd Lab server.

The APU boxes I use are from PCengines, like the ALIX boards. But they are a bit later / advanced compared to the ALIX.

Except for “Lab” tests, I never use an “old” server as a firewall. First, it’s “old”. It’s no more energy efficient, generates too much noise and heat and other issues. I do not trust such boxes anymore as firewall. Additionally, we have fairly high Internet speeds, so using old hardware just doesn’t “cut” it.

A firewall box, or at least typical boards are, compared to servers, fairly strict for NICs and BIOS, they are initialized according to connection on the board…
As OPNsense handles it, the LAN NIC is per default ALWAYS on the first NIC, and is easily seen or discovered as it also provides DHCP out of the box. I much prefer this, than setting up an unneeded bridge, than having to remvoe NICs from the bridge to be able to assign NICs for a specific purpose. I do not need to “identify” a NIC by it’s MAC, I already know the MAC in advance.
All such boards use consequetive MACs for their 2, 3 or 4 NICs. As the first is LAN, the other MACs are clear.
The first time seeing this behaviour, I was shocked - pre-allocating all NICs is just not done - and certainly not on first setup!

Threat Shield:

Well, my point if there IS a community, the Community can provide feedback on the quality of the Community lists…

Second point is on the intro of NethSecurity, the Screenshot could be from an subscription setup, it doesn’t help building trust in a product.

Third point is there should be an Info on the GUI about confidense level only with paid lists. (Subscription).

Wireguard:

If I am forced to use a CLI, why should I install NethSecurity 8 instead of OpenWRT?

LetsEncrypt

Read your own post!

A valid “subdomain” of “.lan” would be for example “ns8.lan” - this is NEVER accessible from the Internet, very simple.

If you mean a valid domain (Which .lan never is!), sure, but it’s not a subdomain of .lan…

LDAP or AD

LDAP or AD are ONLY available for paid subscriptions…

And almost ALL Home Users I know use AD (only 2 of 20 use LDAP) - none want to replicate password changes over serveral systems manually…

DNS

I know and have worke for several large companies using Split DNS. In such cases, external DNS is always handeled externally, internal DNS are usually dedicated or combined with AD, and are redundant.

In smaller companies the Firewall does this task (sometimes) if a real DNS server is not available.

Any decent TCP/IP network will / should implement TCP/IP correctly. This means that DNS is correctly set, forward and backward (PTR records, aka as lookup records. This is what all internal hosts will use when reporting in logs where DNS is concerned.

Example:

Now, let’s say we have a NS8 node in the network. calling this host ns8.mydomain.com would not be wrong… The nextcloud component could use an Alias (CNAME) of cloud.mydomain.com.
If using just two A-Records, both names could be returned for a query, and both could be contained in logs. So far OK.
If there issues with say the web server, but not wth Nextcloud, or vica-versa, with Nextcloud, but not with the Webserver, the indication in the log file “cloud.mydomain.com” instead of “ns8.mydomain.com” would help by pointing me to look more at Nextcloud, instead of the Webserver itself…

As to your opinion of “hosting infrastructures” I concur, neither NethSecurity, OPNsense, PFsense or such are laid out for this special use case.

I did write this:

DPI is part of IPS…

I would prefer Fail2ban as an addition, rather than CroudSec. CroudSec seems to be a “fad” across distros at the moment. Fail2ban works without any internet access - like locked down classrooms without Internet.

I do not think 2fa is really needed, SSH external access to a firewall is by default a clear No-Go.
For people for need to administrate their frewall externally should know how to use VPN.

Being available would just induce more people to be careless with shell access!

Programmerscan try to make apps idiot proof.
But idiots also can be upgraded to “greater idiots”…

I was referring to the dozens of issues with NS7 - where the only problem was crappy Threatshield lists… The whole server down, no Internet, all due to a crappy list - which has actually nothing to do with the server part…?

An absolute No-Go.
And a real reason for a dedicated box.

The only APU box with a NIC issue I had is one for a german client, direct lightning hit on tree and Power Infrastructuer, including Telefon (DSL) line.

Modem/Bridge really blackened, including Powerbrick.

OPNsense still working, except for blackened WAN NIC, reassigned thet NIC to another (OPT1), OPNsense continued working…

They do. They also set up “servers” in the cloud - really great paying for servers for spammers and bot-nets to use…

Trust has become a major problem with Open Source, as recently evidenced by the source code manipulation introduced by a somewhat dubious contributer. That library is something used by many major corporations in the world of IT. And it added a rootkit behind the scenes…

The contributer - almost like a dormant “sleeper” in spy stories, applied for commit permissions several years ago. The actual maintener and only contributer was heavily overworked and underpaid in his daily work, so a helping hand was welcome. After several attempts, this malicious contributor inserted malicous code - but this was, luckily, discovered early…

Now, in software, a supply side chain attack is something really new, on par with the Isreali pager attacks. This has never been done before!

My feeedback to yours, I hope this is comprehensible.

My 2 cents
Andy