Out of 5 servers the netdata status doesn't appear optimal and 1 keeps crashing

Support
1

NethServer Version: 7.9

Can anyone look at their netdata and see if they’re “having issues”.
I have one server that netdata crashes on constantly and the others having “errors?” with failed config lookups, etc.
The first is the one that crashes. I can restart it and it last for 5 min or so before crashing.

● netdata.service - Real time performance monitoring
   Loaded: loaded (/usr/lib/systemd/system/netdata.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/netdata.service.d
           └─cleanup.conf
   Active: activating (auto-restart) (Result: signal) since Tue 2023-04-04 21:18:27 MST; 16s ago
  Process: 16244 ExecStart=/usr/sbin/netdata -P /run/netdata/netdata.pid -D (code=killed, signal=BUS)
  Process: 16241 ExecStartPre=/usr/bin/find /var/cache/netdata/ -type f -empty -delete (code=exited, status=0/SUCCESS)
  Process: 16238 ExecStartPre=/bin/chown -R netdata /run/netdata (code=exited, status=0/SUCCESS)
  Process: 16236 ExecStartPre=/bin/mkdir -p /run/netdata (code=exited, status=0/SUCCESS)
  Process: 16234 ExecStartPre=/bin/chown -R netdata /var/cache/netdata (code=exited, status=0/SUCCESS)
  Process: 16230 ExecStartPre=/bin/mkdir -p /var/cache/netdata (code=exited, status=0/SUCCESS)
 Main PID: 16244 (code=killed, signal=BUS)

Apr 04 21:18:27  servercrashes.adomain.com systemd[1]: Unit netdata.service entered failed state.
Apr 04 21:18:27  servercrashes.adomain.com systemd[1]: netdata.service failed.

============================================

● netdata.service - Real time performance monitoring
   Loaded: loaded (/usr/lib/systemd/system/netdata.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/netdata.service.d
           └─cleanup.conf
   Active: active (running) since Sat 2023-04-01 09:32:06 MST; 3 days ago
 Main PID: 1269 (netdata)
   CGroup: /system.slice/netdata.service
           ├─ 1269 /usr/sbin/netdata -P /run/netdata/netdata.pid -D
           ├─ 1332 /usr/sbin/netdata --special-spawn-server
           ├─ 3768 /usr/bin/python3 /usr/libexec/netdata/plugins.d/python.d.plugin 1
           ├─14588 /usr/bin/bash /usr/libexec/netdata/plugins.d/charts.d.plugin 1
           └─21038 /usr/bin/bash /usr/libexec/netdata/plugins.d/tc-qos-helper.sh 1

Apr 01 09:32:06   anotherserver.adomain.com systemd[1]: Starting Real time performance monitoring...
Apr 01 09:32:06   anotherserver.adomain.com systemd[1]: Started Real time performance monitoring.
Apr 01 09:32:06   anotherserver.adomain.com netdata[1269]: CONFIG: cannot load cloud config '/var/lib/netdata/cloud....lts.
Apr 01 09:32:06   anotherserver.adomain.com netdata[1269]: 2023-04-01 09:32:06: netdata INFO  : MAIN : CONFIG: canno...lts.
Apr 02 08:43:36   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:50] Failed to parse...FMON
Apr 02 08:43:36   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:69] Unknown lvalue ...ice'
Apr 03 05:50:33   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:50] Failed to parse...FMON
Apr 03 05:50:33   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:69] Unknown lvalue ...ice'
Apr 04 03:43:40   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:50] Failed to parse...FMON
Apr 04 03:43:40   anotherserver.adomain.com systemd[1]: [/usr/lib/systemd/system/netdata.service:69] Unknown lvalue ...ice'
Hint: Some lines were ellipsized, use -l to show in full.
2

The cloud.conf and parse errors are quite common but the service shouldn’t stop.

Are there errors in /var/log/netdata/error.log-* ?

Maybe related:

1 Like
3

:rofl: so many logs, one was over a hundred MB.
I stopped the service, deleted the latest huge logs and then did a start/stop service.
I see connection failures and many journal files which doesn’t match up with other installs I have compared. @mrmarkuz hope that’s not too much

2023-04-11 18:40:18: netdata INFO : MAIN : SIGNAL: Not enabling reaper
2023-04-11 18:40:18: netdata INFO : MAIN : resources control: allowed file descriptors: soft = 1024, max = 4096
2023-04-11 18:40:18: netdata INFO : MAIN : Adjusted my Out-Of-Memory (OOM) score from 0 to 1000.
2023-04-11 18:40:18: netdata INFO : MAIN : Adjusted netdata scheduling policy to idle (5), with priority 0.
2023-04-11 18:40:18: netdata INFO : MAIN : Running with process scheduling policy ‘idle’
2023-04-11 18:40:18: netdata INFO : MAIN : netdata started on pid 26509.
2023-04-11 18:40:18: netdata INFO : MAIN : Initializing spawn client.
2023-04-11 18:40:18: netdata INFO : MAIN : Executing /usr/libexec/netdata/plugins.d/system-info.sh
Spawn server is up.
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_NAME=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_ID_LIKE=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_VERSION_ID=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_OS_DETECTION=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_CONTAINER_IS_OFFICIAL_IMAGE=false
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_NAME=NethServer
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_ID=nethserver
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_ID_LIKE=rhel fedora centos
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION=7.9.2009
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_VERSION_ID=7
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_OS_DETECTION=/etc/os-release
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_HOST_IS_K8S_NODE=false
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_NAME=Linux
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_KERNEL_VERSION=3.10.0-1160.88.1.el7.x86_64
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_ARCHITECTURE=x86_64
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_VIRTUALIZATION=kvm
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_VIRT_DETECTION=systemd-detect-virt
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER=none
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CONTAINER_DETECTION=systemd-detect-virt
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_LOGICAL_CPU_COUNT=4
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_VENDOR=GenuineIntel
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_MODEL=Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_FREQ=3399000000
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_CPU_DETECTION=lscpu
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_RAM=3972845568
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_RAM_DETECTION=procfs
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_TOTAL_DISK_SIZE=103079215104
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_SYSTEM_DISK_DETECTION=sysfs
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_INSTANCE_CLOUD_TYPE=unknown
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_INSTANCE_CLOUD_INSTANCE_TYPE=unknown
2023-04-11 18:40:19: netdata INFO : MAIN : NETDATA_INSTANCE_CLOUD_INSTANCE_REGION=unknown
2023-04-11 18:40:19: netdata INFO : MAIN : SQLite database /var/cache/netdata/netdata-meta.db initialization
2023-04-11 18:40:19: netdata INFO : MAIN : metadata database version is 7 (no migration needed)
2023-04-11 18:40:19: netdata INFO : MAIN : SQLite database initialization completed
2023-04-11 18:40:19: netdata INFO : MAIN : SQLite database /var/cache/netdata/context-meta.db initialization
2023-04-11 18:40:19: netdata INFO : MAIN : context database version is 1 (no migration needed)
2023-04-11 18:40:19: netdata INFO : MAIN : CONFIG: cannot load user config ‘/etc/netdata/stream.conf’. Will try stock config.
2023-04-11 18:40:19: netdata INFO : MAIN : Initializing dbengine…
2023-04-11 18:40:19: netdata INFO : MAIN : Found 39 files in path /var/cache/netdata/dbengine
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002508.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Matched file “/var/cache/netdata/dbengine/datafile-1-0000002508.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002509.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Matched file “/var/cache/netdata/dbengine/datafile-1-0000002509.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002510.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Matched file “/var/cache/netdata/dbengine/datafile-1-0000002510.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002511.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Matched file “/var/cache/netdata/dbengine/datafile-1-0000002511.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002512.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Matched file “/var/cache/netdata/dbengine/datafile-1-0000002512.ndf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/datafile-1-0000002513.ndf”

2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002521.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002522.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002523.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002524.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002525.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/journalfile-1-0000002526.njf”
2023-04-11 18:40:19: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine/metadatalog-00000-00001.mlf”
2023-04-11 18:40:19: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002508.ndf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002508.ndf” initialized (size:13504512).
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002508.njf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002508.njf” using MMAP.
2023-04-11 18:40:19: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002508.njf” loaded (size:1019904).
2023-04-11 18:40:19: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002509.ndf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002509.ndf” initialized (size:13434880).
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002509.njf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002509.njf” using MMAP.
2023-04-11 18:40:19: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002509.njf” loaded (size:987136).
2023-04-11 18:40:19: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002510.ndf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002510.ndf” initialized (size:13471744).
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002510.njf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002510.njf” using MMAP.
2023-04-11 18:40:19: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002517.ndf” initialized (size:13463552).
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002517.njf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002517.njf” using MMAP.
2023-04-11 18:40:19: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002517.njf” loaded (size:1269760).
2023-04-11 18:40:19: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002518.ndf”.
2023-04-11 18:40:19: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002518.ndf” initialized (size:13434880).
2023-04-11 18:40:20: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002524.njf” loaded (size:1204224).
2023-04-11 18:40:20: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002525.ndf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002525.ndf” initialized (size:13430784).
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002525.njf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002525.njf” using MMAP.
2023-04-11 18:40:20: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002525.njf” loaded (size:1056768).
2023-04-11 18:40:20: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine/datafile-1-0000002526.ndf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine/datafile-1-0000002526.ndf” initialized (size:4706304).
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002526.njf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine/journalfile-1-0000002526.njf” using MMAP.
2023-04-11 18:40:20: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine/journalfile-1-0000002526.njf” loaded (size:368640).
2023-04-11 18:40:20: netdata INFO : MAIN : DBENGINE: tier 0: 6 pages had a different number of points compared to their timestamps (latest: 9947543 secs ago; these page have been loaded)
2023-04-11 18:40:20: netdata INFO : MAIN : Found 4 files in path /var/cache/netdata/dbengine-tier1
2023-04-11 18:40:20: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine-tier1/datafile-1-0000000001.ndf”
2023-04-11 18:40:20: netdata INFO : MAIN : Scanning file “/var/cache/netdata/dbengine-tier2/journalfile-1-0000000001.njf”
2023-04-11 18:40:20: netdata INFO : MAIN : Initializing data file “/var/cache/netdata/dbengine-tier2/datafile-1-0000000001.ndf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Data file “/var/cache/netdata/dbengine-tier2/datafile-1-0000000001.ndf” initialized (size:393216).
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine-tier2/journalfile-1-0000000001.njf”.
2023-04-11 18:40:20: netdata INFO : MAIN : Loading journal file “/var/cache/netdata/dbengine-tier2/journalfile-1-0000000001.njf” using MMAP.
2023-04-11 18:40:20: netdata INFO : MAIN : Journal file “/var/cache/netdata/dbengine-tier2/journalfile-1-0000000001.njf” loaded (size:77824).
2023-04-11 18:40:20: netdata INFO : MAIN : Starting metadata sync thread with 1024 entries command queue
2023-04-11 18:40:20: netdata INFO : MAIN : SQLite metadata sync initialization complete
2023-04-11 18:40:20: netdata INFO : MAIN : Host ’ server.adomain.com ’ (at registry as ’ server.adomain.com ‘) with guid ‘137657c4-374f-11eb-8ce5-0800270273d8’ initialized, os ‘linux’, timezone ‘replaced’, tags ‘’, program_name ‘netdata’, program_version ‘v1.37.1’, update every 1, memory mode dbengine, history entries 0, streaming disabled (to ‘’ with api key ‘’), health disabled, cache_dir ‘/var/cache/netdata’, varlib_dir ‘/var/lib/netdata’, health_log ‘(null)’, alarms default handler ‘’, alarms default recipient ‘’
2023-04-11 18:40:20: netdata INFO : MAIN : SQLite aclk sync initialization
2023-04-11 18:40:20: netdata INFO : MAIN : SQLite aclk sync initialization completed
2023-04-11 18:40:20: netdata INFO : MAIN : Unable to load ‘/var/lib/netdata/cloud.d/claimed_id’, setting state to AGENT_UNCLAIMED
2023-04-11 18:40:20: netdata ERROR : MAIN : PROCFILE: Cannot open file ‘/etc/netdata/.environment’ (errno 2, No such file or directory)
2023-04-11 18:40:20: netdata INFO : PLUGIN[timex] : thread created with task id 26628
2023-04-11 18:40:20: netdata INFO : PLUGIN[timex] : set name of thread 26628 to PLUGIN[timex]
2023-04-11 18:40:20: netdata INFO : GLOBAL_STATS : thread created with task id 26629
2023-04-11 18:40:20: netdata INFO : REPLICATION : thread created with task id 26638
2023-04-11 18:40:20: netdata INFO : REPLICATION : set name of thread 26638 to REPLICATION
2023-04-11 18:40:20: netdata INFO : PLUGIN[tc] : thread created with task id 26639
2023-04-11 18:40:20: netdata INFO : PLUGINSD[python.d] : thread created with task id 26643
2023-04-11 18:40:20: netdata ERROR : PLUGINSD : cannot open plugins directory ‘/etc/netdata/custom-plugins.d’ (errno 2, No such file or directory)
2023-04-11 18:40:20: netdata INFO : EXPORTING : thread created with task id 26634
2023-04-11 18:40:20: netdata INFO : EXPORTING : set name of thread 26634 to EXPORTING
2023-04-11 18:40:20: netdata INFO : WEB_SERVER[static1] : To use encryption it is necessary to set “ssl certificate” and “ssl key” in [web] !
2023-04-11 18:40:20: netdata INFO : WEB_SERVER[static1] : You are running an OpenSSL older than 1.1.0, web server will not enable multithreading.
2023-04-11 18:40:20: netdata INFO : PLUGIN[proc] : thread created with task id 26641
2023-04-11 18:40:20: netdata INFO : WEB_SERVER[static1] : POLLFD: LISTENER: listening on ‘tcp:127.0.0.1:19999’
started with taskid 26648
2023-04-11 18:40:20: netdata INFO : PLUGIN[proc netdev] : thread created with task id 26653
2023-04-11 18:40:20: netdata INFO : PLUGIN[proc netdev] : set name of thread 26653 to PLUGIN[proc net
2023-04-11 18:40:20: netdata INFO : STATSD_COLLECTOR[1] : POLLFD: LISTENER: listening on ‘udp:127.0.0.1:8125’
2023-04-11 18:40:20: netdata INFO : STATSD_COLLECTOR[1] : POLLFD: LISTENER: listening on ‘tcp:127.0.0.1:8125’
2023-04-11 18:40:20: tc-qos-helper.sh: WARNING: Cannot find file ‘/etc/netdata/conf.d/tc-qos-helper.conf’.
2023-04-11 18:40:20: tc-qos-helper.sh: WARNING: Cannot find file ‘/etc/netdata/tc-qos-helper.conf’.
2023-04-11 18:40:20: charts.d: ERROR: libreswan: module’s ‘libreswan’ check() function reports failure.
2023-04-11 18:40:20: charts.d: WARNING: nut: command ‘upsc’ is not found in /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin.
2023-04-11 18:40:20: charts.d: ERROR: nut: module’s ‘nut’ check() function reports failure.
2023-04-11 18:40:20: charts.d: WARNING: opensips: command ‘opensipsctl’ is not found in /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin.
2023-04-11 18:40:20: python.d INFO: plugin[main] : using python v3
2023-04-11 18:40:20: charts.d: ERROR: opensips: module’s ‘opensips’ check() function reports failure.
2023-04-11 18:40:20: charts.d: FATAL: main: No charts to collect data from.
2023-04-11 18:40:20: netdata INFO : PLUGINSD[charts.d] : PLUGINSD: plugin called DISABLE. Disabling it.
2023-04-11 18:40:20: netdata ERROR : PLUGINSD[charts.d] : ‘/usr/libexec/netdata/plugins.d/charts.d.plugin’ (pid 26645) disconnected after 0 successful data collections (ENDs). (errno 9, Bad file descriptor)
2023-04-11 18:40:20: netdata INFO : PLUGINSD[charts.d] : ‘/usr/libexec/netdata/plugins.d/charts.d.plugin’ (pid 26645) does not generate useful output but it reports success (exits with 0). Will not start it again - it is now disabled…
2023-04-11 18:40:20: netdata INFO : MAIN : /usr/libexec/netdata/plugins.d/anonymous-statistics.sh ‘CRASH’ ‘-’ ‘-’
2023-04-11 18:40:21: netdata INFO : ANALYTICS : thread created with task id 26842
2023-04-11 18:40:21: netdata INFO : ANALYTICS : set name of thread 26842 to ANALYTICS
2023-04-11 18:40:21: netdata INFO : PLUGIN[proc] : Using now_boottime_usec() for uptime (dt is 4 ms)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/pressure/cpu’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /proc/pressure/cpu.
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/pressure/memory’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /proc/pressure/memory.
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/pressure/io’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : Cannot read pressure information from /proc/pressure/io.
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/net/sctp/snmp’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/net/ip_vs_stats’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/net/stat/synproxy’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : DEVICE-MAPPER (‘sda’, 8:0): Cannot open directory ‘/dev/disk/by-label’. (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : DEVICE-MAPPER (‘sda’, 8:0): Cannot open directory ‘/dev/vx/dsk’. (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : DEVICE-MAPPER (‘sr0’, 11:0): Cannot open directory ‘/dev/disk/by-label’. (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : DEVICE-MAPPER (‘sr0’, 11:0): Cannot open directory ‘/dev/vx/dsk’. (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/net/rpc/nfsd’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/net/rpc/nfs’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : PROCFILE: Cannot open file ‘/proc/spl/kstat/zfs/arcstats’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : Cannot read directory ‘/proc/spl/kstat/zfs’ (errno 2, No such file or directory)
2023-04-11 18:40:21: netdata ERROR : PLUGIN[proc] : BTRFS: Cannot open directory ‘/sys/fs/btrfs’. (errno 2, No such file or directory)
2023-04-11 18:40:21: python.d INFO: plugin[main] : [adaptec_raid] is disabled by default, skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [alarms] is disabled by default, skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [am2320] built 1 job(s) configs
2023-04-11 18:40:22: python.d WARNING: plugin[main] : [anomalies] error on loading source : ModuleNotFoundError(“No module named ‘requests’”,), skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [beanstalk] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [bind_rndc] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [boinc] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [ceph] built 1 job(s) configs
2023-04-11 18:40:22: python.d WARNING: plugin[main] : [changefinder] error on loading source : ModuleNotFoundError(“No module named ‘numpy’”,), skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [dockerd] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [dovecot] built 5 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [nvidia_smi] is disabled by default, skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [openldap] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [oracledb] built 1 job(s) configs
2023-04-11 18:40:22: python.d WARNING: plugin[main] : [pandas] error on loading source : ModuleNotFoundError(“No module named ‘pandas’”,), skipping it
2023-04-11 18:40:22: python.d INFO: plugin[main] : [postfix] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [proxysql] built 6 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [puppet] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [tomcat] built 3 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [tor] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [traefik] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [uwsgi] built 4 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [varnish] built 1 job(s) configs
2023-04-11 18:40:22: python.d INFO: plugin[main] : [w1sensor] built 1 job(s) configs
2023-04-11 18:40:22: python.d WARNING: plugin[main] : [zscores] error on loading source : ModuleNotFoundError(“No module named ‘requests’”,), skipping it
2023-04-11 18:40:22: python.d ERROR: am2320[am2320] : Could not find the adafruit-circuitpython-am2320 package.
2023-04-11 18:40:22: python.d INFO: plugin[main] : am2320[am2320] : check failed
2023-04-11 18:40:22: python.d ERROR: beanstalk[beanstalk] : ‘beanstalkc’ module is needed to use beanstalk.chart.py
2023-04-11 18:40:22: python.d INFO: plugin[main] : beanstalk[beanstalk] : check failed
2023-04-11 18:40:22: python.d ERROR: bind_rndc[bind_rndc] : Can’t locate “rndc” binary or binary is not executable by netdata
2023-04-11 18:40:22: python.d INFO: plugin[main] : bind_rndc[bind_rndc] : check failed
2023-04-11 18:40:22: python.d INFO: plugin[main] : boinc[boinc] : check failed
2023-04-11 18:40:22: python.d ERROR: ceph[ceph] : rados module is needed to use ceph.chart.py
2023-04-11 18:40:22: python.d INFO: plugin[main] : ceph[ceph] : check failed
2023-04-11 18:40:22: python.d ERROR: dockerd[local] : ‘docker’ package is needed to use dockerd module
2023-04-11 18:40:22: python.d INFO: plugin[main] : dockerd[local] : check failed
2023-04-11 18:40:22: python.d ERROR: dovecot[localhost] : Failed to connect to “127.0.0.1”, port 24242, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : dovecot[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: dovecot[localipv4] : Failed to connect to “127.0.0.1”, port 24242, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : dovecot[localipv4] : check failed
2023-04-11 18:40:22: python.d ERROR: dovecot[localipv6] : Failed to connect to “::1”, port 24242, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : dovecot[localipv6] : check failed
2023-04-11 18:40:22: python.d INFO: plugin[main] : dovecot[localsocket] : check failed
2023-04-11 18:40:22: python.d INFO: plugin[main] : dovecot[localsocket_old] : check failed
2023-04-11 18:40:22: python.d ERROR: exim[local] : Can’t locate “exim -bpc” binary
2023-04-11 18:40:22: python.d INFO: plugin[main] : exim[local] : check failed
2023-04-11 18:40:22: python.d ERROR: fail2ban[fail2ban] : /var/log/fail2ban.log is not readable
2023-04-11 18:40:22: python.d INFO: plugin[main] : fail2ban[fail2ban] : check failed
2023-04-11 18:40:22: python.d ERROR: gearman[localhost] : Failed to connect to “127.0.0.1”, port 4730, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : gearman[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: haproxy[haproxy] : URL is not defined or type is not
2023-04-11 18:40:22: python.d INFO: plugin[main] : haproxy[haproxy] : check failed
2023-04-11 18:40:22: python.d ERROR: hddtemp[localhost] : Failed to connect to “127.0.0.1”, port 7634, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : hddtemp[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: hddtemp[localipv4] : Failed to connect to “127.0.0.1”, port 7634, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : hddtemp[localipv4] : check failed
2023-04-11 18:40:22: python.d ERROR: hddtemp[localipv6] : Failed to connect to “::1”, port 7634, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : hddtemp[localipv6] : check failed
2023-04-11 18:40:22: python.d ERROR: icecast[localhost] : Url: http://localhost:8443/status-json.xsl. Error: HTTPConnectionPool(host=‘localhost’, port=8443): Max retries exceeded with url: /status-json.xsl (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x7f6a96599f98>: Failed to establish a new connection: [Errno 111] Connection refused’,))
2023-04-11 18:40:22: python.d INFO: plugin[main] : icecast[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: icecast[localipv4] : Url: http://127.0.0.1:8443/status-json.xsl. Error: HTTPConnectionPool(host=‘127.0.0.1’, port=8443): Max retries exceeded with url: /status-json.xsl (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x7f6a965db630>: Failed to establish a new connection: [Errno 111] Connection refused’,))
2023-04-11 18:40:22: python.d INFO: plugin[main] : icecast[localipv4] : check failed
2023-04-11 18:40:22: python.d ERROR: ipfs[localhost] : Url: http://localhost:5001/api/v0/stats/bw. Error: HTTPConnectionPool(host=‘localhost’, port=5001): Max retries exceeded with url: /api/v0/stats/bw (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x7f6a965dbc88>: Failed to establish a new connection: [Errno 111] Connection refused’,))
2023-04-11 18:40:22: python.d ERROR: ipfs[localhost] : Url: http://localhost:5001/api/v0/swarm/peers. Error: HTTPConnectionPool(host=‘localhost’, port=5001): Max retries exceeded with url: /api/v0/swarm/peers (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x7f6a965dbf28>: Failed to establish a new connection: [Errno 111] Connection refused’,))
2023-04-11 18:40:22: python.d ERROR: ipfs[localhost] : _get_data() returned no data or type is not
2023-04-11 18:40:22: python.d INFO: plugin[main] : ipfs[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: litespeed[localhost] : “/tmp/lshttpd/” has no “rtreport” files or dir is not readable
2023-04-11 18:40:22: python.d INFO: plugin[main] : litespeed[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: memcached[localhost] : Failed to connect to “127.0.0.1”, port 11211, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : memcached[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: memcached[localipv4] : Failed to connect to “127.0.0.1”, port 11211, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : memcached[localipv4] : check failed
2023-04-11 18:40:22: python.d ERROR: memcached[localipv6] : Failed to connect to “::1”, port 11211, error: [Errno 111] Connection refused
2023-04-11 18:40:22: python.d INFO: plugin[main] : memcached[localipv6] : check failed
2023-04-11 18:40:22: python.d ERROR: mongodb[local] : Pymongo package v2.4+ is needed to use mongodb.chart.py
2023-04-11 18:40:22: python.d INFO: plugin[main] : mongodb[local] : check failed
2023-04-11 18:40:22: python.d ERROR: monit[localhost] : Url: http://localhost:2812/_status?format=xml&level=full. Error: HTTPConnectionPool(host=‘localhost’, port=2812): Max retries exceeded with url: /_status?format=xml&level=full (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x7f6a965dbd68>: Failed to establish a new connection: [Errno 111] Connection refused’,))
2023-04-11 18:40:22: python.d ERROR: monit[localhost] : _get_data() returned no data or type is not
2023-04-11 18:40:22: python.d INFO: plugin[main] : monit[localhost] : check failed
2023-04-11 18:40:22: python.d ERROR: nsd[local] : Can’t locate “nsd-control stats_noreset” binary
2023-04-11 18:40:22: python.d INFO: plugin[main] : nsd[local] : check failed
EOF found in spawn pipe.
Shutting down spawn server event loop.
Shutting down spawn server loop complete.

4

You may try to clear the cache as explained here: Netdata 1.35.1 and EOF found in spawn pipe – Trond's place

4 Likes
5

That was it. Thank you so much.

Active: active (running) since Thu 2023-04-20 18:22:15 MST; 4 days ago
Main PID: 5298 (netdata)

2 Likes