Is there a way to modify local subnets in ntop for NethServer?

tchenier · September 17, 2016, 5:18am

I love that Ntop is included in Nethserver. Ability in Ntop to filter hosts between local and remote is a key feature of Ntop that I cannot figure out how to configure for this Nethserver based Ntop instance. Any help? If not, I have a feature request.

Ntop config directs us to define local networks / subnets in the config file ntop.conf

What I find in my conf file is this:

of course it directs me not to modify this file directly since Nethserver will overwrite it. (p.s. I tried to follow the link to the developer’s guide, but didn’t work. Is there maybe another manual config file I missed that also feeds into the one show from /etc/ntopng/ntopng.conf ?)

Config file network apparently comes from the interface network(s). This also auto-populates trusted networks:

But why not auto-populate the ntop file with ALL trusted networks? That would be the cat’s meow! (or is there a way to do this, that I haven’t figured out?) I use a subnet for servers that is distinct from client subnets, so only have servers listed as “local” in Ntop, but I really would like to see all connected users as “local”. Otherwise Ntop makes much less sense it seems. Let me explain where I’m coming from. In the Hosts view, there is an option for selecting (upper right hand corner of the following image) All hosts, Local Only, or Remote Only. If I want to see local hosts that are not included as local, I would need to sort through a rather extensive list of Remote hosts.

Then there is the Networks view. I will not have chart options, aggregate traffic data, etc of my other local networks without these being included as local subnets. And that would be super useful!!!

If all trusted networks became local networks defined in ntop, I could make this more useful by showing individual vlan/subnet sections of client networks. I would do that by breaking up my trusted network entry from:

10.20.0.0/16
to
10.20.0.0/24
10.20.110.0/24
10.20.120.0/24
10.20.150.0/24

If I could define trusted networks that way in Nethserver and have it copy over into the config file for ntop, that would be VERY nice. (again, or is there a way to do that or an alternate manual config file that I missed?)

(Regarding talk of maybe moving away from Ntop, I think bandwidthd is quite nice, but imo one advantage of Ntop is viewing current traffic at the time of a network slowdown. This is helpful in low bandwidth environments to help determine from where and what type of traffic the Internet resources are being sapped. This is the very type of environment I am working to help at the moment.) Thanks for any help or comments.

clinton · September 17, 2016, 7:56am

@tchenier have you tried bandwidthd…

tchenier · September 17, 2016, 4:27pm

Indeed! But I am not capturing traffic correctly in bandwidthd, so I may not have a full view of all it is capabilities.
Perhaps the main problem is I have setup Nethserver as an intercept proxy on a different subnet (using MikroTik policy based routing - would be happy to share how-to of my setup if there is interest.

I am actually not capturing traffic in bandwidthd that I KNOW is going through the proxy server. I can show you below, and would be very happy for tips. Here is my computer with bandwidthd showing no traffic in the last 24 hours.

Here’s an image showing
a) Policy route for http traffic
b) Nethserver filter (and showing matching IP)
c) Web site being blocked in browser

ntop at least sees my traffic, just lists it as remote.

Ideally, I’d love to see the option of pulling in netflow from another source (i.e. my main router wan interface). That way I’d get to see all the traffic, even that which doesn’t pass through or close to the NethServer. I think there would be a real big market for that :D. Can I help build it?

clinton · September 17, 2016, 6:25pm

Yes please, I have just starrted using the mickrotiks, I have been very ubiquiti orientated. So the Mikrotik is a much better, To be honest, when looking at it, its almost like comparing windows to nethserver… No chance…

I would love to see … thanks

clinton

tchenier · September 18, 2016, 12:26am

Copying spreadsheet notes into here doesn’t work. This is where I have my full setup notes that I have repeated successfully multiple times… so full notes on NethServer settings as well as MikroTik settings will have to wait until I have more time for formatting… but here is the simple policy route setup on the Mikrotik side. In this run, I had a single interface Nethserver on green interface. I’ve also done 2 interface Nethservers. This is for Nethserver on transparent proxy. You can also use the MikroTik proxy with Nethserver as the second proxy, but you loose client identity to the Nethserver with taht setup.

/ip firewall address-list add name=mgmt-net address=10.20.0.0/24
/ip firewall address-list add name=proxy-dst-exclude address=10.20.0.1 comment=“router config”
/ip firewall address-list add name=proxy-dst-exclude address=10.21.0.0/16 comment=“all 10.21.x.x for servers”
/ip firewall address-list add name=proxy-src-include address=10.20.0.0/24
/ip firewall address-list add name=proxy-src-include address=10.20.110.0/24
/ip firewall address-list add name=proxy-src-include address=10.20.120.0/24
/ip firewall address-list add name=proxy-src-include address=10.20.150.0/24
/ip firewall mangle add chain=prerouting protocol=tcp dst-port=80,8080 src-address-list=proxy-src-include dst-address-list=!prox-dst-exclude action=mark-routing new-routing-mark=route-proxy
/ip route add dst-address=0.0.0.0/0 gateway=10.21.0.2 routing-mark=route-proxy comment=“routing for proxy”

giacomo · September 19, 2016, 7:48am

Honestly, I don’t recall the reason of not listing all trusted networks in ntopng configuration…

By the way, you can change the configuration like this:

mkdir -p /etc/e-smith/templates-custom/etc/ntopng/ntopng.conf
touch /etc/e-smith/templates-custom/etc/ntopng/ntopng.conf/10base

Write your config inside etc/e-smith/templates-custom/etc/ntopng/ntopng.conf/10base, then apply the configuration:

signal-event nethserver-ntopng-update

tchenier · September 19, 2016, 4:18pm

giacomo, you’re a lifesaver! I’ll try this out. Do you happen to know if there is a similar way to configure bandwidthd? I’m getting no useful data from it, probably because my server is on a separate subnet from any of my users I’m guessing, so nethserver is the only traffic listed.

I’d actually be interested in helping modify the interface so we could select inclusion as local network into Bandwidthd and ntop for each of our trusted networks. A couple check boxes on the trusted network setup and a little script to include these into the two programs, maybe. Seems like wouldn’t be too hard… I’m pretty rusty at this type of thing, so I’d have to look at it before deciding if I could help. I’ll look for any howtos on this after I’m done with this current project push.

As long as I’m talking wish list… if Nethserver had a config page to enable ntop to accept external netflows (or maybe even just unlocked the ntop config for this), that could have pretty broad appeal. I believe many if not most companies do not want to have all their traffic flow through a server (for high availability, speed, or whatever reasons). However, ntop has broad appeal and is not simple to setup. Of course it works with Cisco, Mikrotik and other Netflow generating routers/devices. If Nethserver became the go-to server for ntop server, that could be big.