Integration of external collectd data

capote · May 4, 2021, 2:21pm

Hello,
on my OPNSense collectd is running as an agent. It can deliver the data to another collector host. Is it possible to integrate such data into the Nethservers collectd (-web)?
Sincerely, MArko

mrmarkuz · May 4, 2021, 6:27pm

I’m testing opnsense and had similar idea. Yes, it’s possible.

Add collectd network plugin to the Nethserver by creating /etc/collectd.d/networkserver.conf with following content (192.168.2.22 is the IP of my Nethserver hosting the collectd server):

LoadPlugin "network"

<Plugin "network">
  Listen "192.168.2.22"
</Plugin>

For adding another Nethserver create /etc/collectd.d/networkclient.conf on the collectd client Neth:

LoadPlugin "network"

<Plugin "network">
  Server "192.168.2.22"
</Plugin>

Restart collectd to apply config:

systemctl restart collectd

https://collectd.org/wiki/index.php/Networking_introduction

For opnsense you need to configure hostname AND port.

capote · May 4, 2021, 8:20pm

Thank you for the tip, I adapted it. But there are no changes within the presented monitors.

mrmarkuz · May 4, 2021, 10:44pm

You may click on the title “Collectd Graph Panel” to show the the collectd main page listing the clients or just delete the hostname part (?h=hostname) from the URL.

EDIT:

I forgot that you need to open UDP port 25826 on the collectd server. I did it by adding a network service:

capote · May 5, 2021, 7:50am

This was the trick to get the Collectd Graph Panel presented. I have never before consciously noticed the menu bar

you need to open UDP port 25826

I added the rule:

But the collector bar does not present the OPNsense

Thank you very much.

capote · May 5, 2021, 4:20pm

@mrmarkuz: Do you have an additional idea?
I have…

the collectd service restarted,
Nethserver restarted,
OPNsense-Agent restarted,
configured OPNsense-Agent with and w/o credentials.
No changes.

mrmarkuz · May 5, 2021, 4:41pm

Check logfiles for errors after restarting the services.

Is Nethserver listening on UDP port 25826?

nmap 192.168.3.20 -sU -pU:25826

Does opnsense send packets to the Neth? Run from opnsense (re0 is my internal interface):

tcpdump -i re0 -p -n -s 1500 udp port 25826 -vv

Here is my /usr/local/etc/collectd.conf from the opnsense

Hostname    "keepout"
FQDNLookup    true
Interval    10

LoadPlugin syslog
<Plugin syslog>
  LogLevel err
</Plugin>

LoadPlugin contextswitch
LoadPlugin cpu
LoadPlugin aggregation
LoadPlugin disk
LoadPlugin df
LoadPlugin interface
LoadPlugin load
LoadPlugin memory
LoadPlugin swap
LoadPlugin network
LoadPlugin processes
LoadPlugin uptime
LoadPlugin users

##############################################################################
# Plugin configuration                                                       #
#----------------------------------------------------------------------------#
# In this section configuration stubs for each plugin are provided. A desc-  #
# ription of those options is available in the collectd.conf(5) manual page. #
##############################################################################

<Plugin network>
  <Server "192.168.2.22" "25826">
  </Server>
</Plugin>


<Plugin cpu>
  ValuesPercentage true
</Plugin>

<Plugin "aggregation">
  <Aggregation>
    Plugin "cpu"
    Type "percent"
    GroupBy "Host"
    GroupBy "TypeInstance"
    CalculateNum false
    CalculateSum true
    CalculateAverage true
    CalculateMinimum false
    CalculateMaximum false
    CalculateStddev false
  </Aggregation>
</Plugin>

capote · May 5, 2021, 5:51pm


[root@DAHO-Nethserver ~]# nmap 192.168.3.20 -sU -pU:25826

Starting Nmap 6.40 ( http://nmap.org ) at 2021-05-05 19:44 CEST Nmap scan report for DAHO-Nethserver.lan.home (192.168.3.20) Host is up (0.00018s latency). PORT STATE SERVICE 25826/udp closed unknown

root@DAHO-OPNsense:~ # tcpdump -i igb0 -p -n -s 1500 udp port 25826 -vv

tcpdump: listening on igb0, link-type EN10MB (Ethernet), capture size 1500 bytes
19:47:31.275194 IP (tos 0x0, ttl 64, id 9429, offset 0, flags [none], proto UDP (17), length 1338)
    192.168.3.1.34626 > 192.168.3.20.25826: [udp sum ok] UDP, length 1310
19:47:31.276554 IP (tos 0x0, ttl 64, id 20693, offset 0, flags [none], proto UDP (17), length 1347)
    192.168.3.1.34626 > 192.168.3.20.25826: [udp sum ok] UDP, length 1319
19:47:31.280547 IP (tos 0x0, ttl 64, id 29152, offset 0, flags [none], proto UDP (17), length 1347)
    192.168.3.1.34626 > 192.168.3.20.25826: [udp sum ok] UDP, length 1319




root@DAHO-OPNsense:~ # cat /usr/local/etc/collectd.conf
Hostname    "DAHO-OPNSense"
FQDNLookup    true
Interval    10

LoadPlugin syslog
<Plugin syslog>
  LogLevel err
</Plugin>

LoadPlugin contextswitch
LoadPlugin cpu
LoadPlugin aggregation
LoadPlugin disk
LoadPlugin df
LoadPlugin interface
LoadPlugin load
LoadPlugin memory
LoadPlugin swap
LoadPlugin network
LoadPlugin processes
LoadPlugin uptime
LoadPlugin users

##############################################################################
# Plugin configuration                                                       #
#----------------------------------------------------------------------------#
# In this section configuration stubs for each plugin are provided. A desc-  #
# ription of those options is available in the collectd.conf(5) manual page. #
##############################################################################

<Plugin network>
  <Server "192.168.3.20" "25826">
    Username "root"
    Password "****"
  </Server>
</Plugin>


<Plugin cpu>
  ValuesPercentage true
</Plugin>

<Plugin "aggregation">
  <Aggregation>
    Plugin "cpu"
    Type "percent"
    GroupBy "Host"
    GroupBy "TypeInstance"
    CalculateNum false
    CalculateSum true
    CalculateAverage true
    CalculateMinimum false
    CalculateMaximum false
    CalculateStddev false
  </Aggregation>
</Plugin>

capote · May 5, 2021, 6:07pm

On proxmox level the port is not blocked either

mrmarkuz · May 5, 2021, 6:50pm

It should be 25826/udp open|filtered unknown.
Please recheck the collectd_fw service.

capote · May 5, 2021, 10:44pm

I deleted and recreated the service. No Changes.

mrmarkuz · May 5, 2021, 10:48pm

I have no idea. Maybe check /var/log/firewall.log if there are entries…or try it with firewall rules.

capote · May 5, 2021, 10:50pm

I don’t have installed firewall. I will try to install it.

cat /var/log/firewall.log | grep 25826
May  5 15:11:52 DAHO-Nethserver kernel: Shorewall:loc2fw:REJECT:IN=eth0 OUT= MAC=2a:d0:95:1b:3e:62:00:0d:b9:5a:60:1c:08:00 SRC=192.168.3.1 DST=192.168.3.20 LEN=1348 TOS=0x00 PREC=0x00 TTL=64 ID=38557 PROTO=UDP SPT=33729 DPT=25826 LEN=1328 MARK=0x5100

capote · May 6, 2021, 9:07am

I installed the firewall, created a service object and a local role:

It doesn’t change anything.

Strange…

mrmarkuz · May 6, 2021, 10:52am

You may disable the firewall for testing:

shorewall clear

capote · May 6, 2021, 2:51pm

no difference

capote · May 6, 2021, 3:04pm

How I can remove collectd completely to start with a fresh installation w/o artifacts?

mrmarkuz · May 6, 2021, 3:25pm

Did you check if collectd listens?

netstat -lnp | grep collectd

See this page for more troubleshooting.

Collectd is a base part of Nethserver, removing it will also remove other modules. I recommend a reinstall instead of removing.

Remove the previously added /etc/collectd.d/networkserver.conf and do a

yum reinstall collectd

capote · May 6, 2021, 7:11pm

[root@DAHO-Nethserver ~]# netstat -lnp | grep collectd
raw        0      0 0.0.0.0:1               0.0.0.0:*               7           6483/collectd       
unix  2      [ ACC ]     STREAM     LISTENING     170405   6483/collectd        /var/run/collectd.sock
[root@DAHO-Nethserver ~]#

but still:

[root@DAHO-Nethserver ~]# nmap 192.168.3.20 -sU -pU:25826

Starting Nmap 6.40 ( http://nmap.org ) at 2021-05-06 21:09 CEST
Nmap scan report for DAHO-Nethserver.home.dargels.de (192.168.3.20)
Host is up (0.00023s latency).
PORT      STATE  SERVICE
25826/udp closed unknown

reinstallation doesn’t help, but there was not /etc/collectd.d/networkserver.conf to delete

[root@DAHO-Nethserver collectd.d]# ll
total 28
-rw-r--r--. 1 root root  699 Oct  6  2020 00plugins.conf
-rw-r--r--. 1 root root   81 Oct  6  2020 df.conf
-rw-r--r--. 1 root root 1814 May  1 14:41 filter.conf
-rw-r--r--. 1 root root  333 May  1 19:30 ping.conf
-rw-r--r--. 1 root root  133 Oct 24  2018 rrdtool.conf
-rw-r--r--  1 root root 2062 May  6 12:14 threshold.conf
-rw-r--r--. 1 root root  154 Oct  6  2020 unixsock.conf

mrmarkuz · May 6, 2021, 7:13pm

Hm, collectd does not listen.

It should look like this:

[root@server2 ~]# netstat -lnp | grep collectd
udp        0      0 192.168.2.22:25826      0.0.0.0:*                           15981/collectd
raw        0      0 0.0.0.0:1               0.0.0.0:*               7           15981/collectd
unix  2      [ ACC ]     STREAM     LISTENING     122250492 15981/collectd       /var/run/collectd.sock

Please check messages log for collectd entries:

cat /var/log/messages | grep collectd

I use it without username/password, maybe that’s an issue?

EDIT:

Hm, did you forget to put a config file to collectd to make the network plugin work?

mrmarkuz:

Add collectd network plugin to the Nethserver by creating /etc/collectd.d/networkserver.conf with following content (192.168.2.22 is the IP of my Nethserver hosting the collectd server):
LoadPlugin "network"

<Plugin "network">
  Listen "192.168.2.22"
</Plugin>