"Zombie User" after deleting one user

NethServer Version: 7.8
Module: webserver, mail, nextcloud, webtop, firewall, ips

I deleted one user (Provider: LDAP) and his email account.
Now I found a lot of error logs like:

08:40 imap: Error: Authenticated user not found from userdb, auth lookup id=3045195777 (auth connected 1 msecs ago, handshake 1 msecs ago, request took 1 msecs, client-pid=11844 client-id=1) dovecot


08:40 auth: Error: plain(USERxyz,127.0.0.1,<FdwvZMqyKNV/AAAB>): user not found from any userdbs  dovecot 

I checked that no client has configured such an email account.

How can I fix this?
best regards, Marko

Please do a backup before, I don’t test it!!!

You could try to recreate the user, delete his mailbox with:

doveadm expunge -d -u user mailbox '*' all
doveadm -v purge -u user 

and delete the user again.

1 Like

Thank you for this idea.
I actually expect that the reapplied user will get a different internal ID and all manual delete actions will refer to it. Therefore I still hesitate to test such an operation on the production system.

Aren’t the users created somewhere and the user (or its remaining ID) could be deleted manually from the directory or database?

Does anyone have any experience with this? I won’t be the first user who has to delete a user…

@capote

Hi Marko

If you’re using AD, again Stephdl’s PHPLDAPadmin Module can help!

https://wiki.nethserver.org/doku.php?id=phpldapadmin

Run in “Kamikaze” mode, you can easily clean up your AD - I think there is still the user there.
You would also need to check if the user-home (/var/lib/nethserver/users) is still there (delete if yes).

My 2 cents
Andy

unfortunately I use LDAP :frowning:

Is it possible tp migrate?

/var/lib/nethserver/home contains no suspicious user directory.

Sincerely, Marko

Hi Marko,

Such a very long shot that I hesitated before posting:
Since it is a so strange number (FdwvZMqyKNV/AAAB), is it possible that it comes from a vhost that you created with Cockpit then deleted it ?

Michel-André

@capote

Hi Marko

PHPLDAPadmin also works if you’re just using LDAP!

My 2 cents
Andy

1 Like

Hi Andy, I checked it - no artifacts.
I will investigate the idea from @michelandre

best regards, Marko

@capote

Hi Marko

Michel-André idea is also a good possibility…

Andy

I checked all DBs: no hit
I ckecked
/etc/: no hit
/opt/: no hit
/usr/: no hit
/var/:

[root@ns-srv01 /]# grep -rnwi '/var/' -e 'FdwvZMqyKNV/AAAB'
/var/log/maillog:79715:Oct 29 08:40:17 ns-srv01 dovecot: auth: Error: plain(USER01,127.0.0.1,<FdwvZMqyKNV/AAAB>): user not found from any userdbs
/var/log/imap:109394:Oct 29 08:40:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=11844 id=1) (internal failure, 1 successful auths): user=<USER01>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=11845, secured, session=<FdwvZMqyKNV/AAAB>
[root@ns-srv01 /]#

this kind of string changes permanently:

/var/log/imap:131854:Oct 30 00:20:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=2079 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2080, secured, session=<k7Ljhdey0t5/AAAB>
/var/log/imap:131894:Oct 30 00:22:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=2896 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2897, secured, session=<ebQKjdeyat9/AAAB>
/var/log/imap:131942:Oct 30 00:24:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=3425 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3426, secured, session=<vTIylNeyDuB/AAAB>
/var/log/imap:131990:Oct 30 00:26:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=3965 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=3966, secured, session=<BtNYm9eyqOB/AAAB>
/var/log/imap:132030:Oct 30 00:28:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=4456 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=4457, secured, session=<ViiAoteyPOF/AAAB>
/var/log/imap:132087:Oct 30 00:30:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=5000 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=5001, secured, session=<gxmnqdey7uF/AAAB>
/var/log/imap:132124:Oct 30 00:32:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=5547 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=5548, secured, session=<ldzNsNeyhuJ/AAAB>
/var/log/imap:132157:Oct 30 00:34:17 ns-srv01 dovecot: imap-login: Internal login failure (pid=6024 id=1) (internal failure, 1 successful auths): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=6025, secured, session=<UMr1t9eyEuN/AAAB>

So there is still a session with that ID active? Pragmatic solution: find and kill the session… (I am probably oversimplifying things now)

Could it be a SOGo cronjob that is still running? (every 2 minutes)

SOGO I uninstalled, but surprisingly a CRON job is still running

-rw-r–r–. 1 root root 128 Aug 9 2019 0hourly
-rw-r–r-- 1 root root 316 Oct 8 20:27 backup-data
-rw-r–r-- 1 root root 1398 Feb 23 2020 clamav-unofficial-sigs
-rw------- 1 root root 203 Jul 17 18:15 clamav-update
-rw-r–r-- 1 root root 116 Oct 12 17:06 fail2ban-statistics
-rw-r–r-- 1 root root 249 Oct 10 14:20 getmail
-rw-r–r-- 1 root root 438 Oct 18 10:29 nethserver-blacklist
-rw-r–r-- 1 root root 111 Oct 16 15:40 nextcloud
-rw-r–r-- 1 root root 455 May 25 16:35 ntopng
-rw-r–r–. 1 root root 159 May 22 2019 ptrack_purge
-rw-r–r–. 1 root root 108 Nov 27 2019 raid-check
-rw-r–r-- 1 root root 350 Oct 10 19:25 runmysqlbackup
-rw-r–r–. 1 root root 61 Oct 7 10:58 shorewall-update-dst
-rw-r----- 1 root root 1231 Oct 8 22:54 sogo

[root@ns-srv01 cron.d]# cat sogo
# ================= DO NOT MODIFY THIS FILE =================
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at NethServer official site: https://www.nethserver.org
#
#
# Sogod cronjobs

# Vacation messages expiration
# The credentials file should contain the sieve admin credentials (username:passwd)
0 0 * * *      sogo	/usr/sbin/sogo-tool update-autoreply -p /etc/sogo/sieve.creds

# Session cleanup - runs every minute
#   - Ajust the nbMinutes parameter to suit your needs
# Example: Sessions without activity since 60 minutes will be dropped:
#* * * * *      sogo	/usr/sbin/sogo-tool expire-sessions 60

# Email alarms - runs every minutes
# If you need to use SMTP AUTH for outgoing mails, specify credentials to use
# with '-p /path/to/credentialsFile' (same format as the sieve credentials)
* * * * *      sogo	/usr/sbin/sogo-ealarms-notify > /dev/null 2>&1

# Daily backups
#   - writes to ~sogo/backups/ by default
#   - will keep 31 days worth of backups by default
#   - runs once a day by default, but can run more frequently
#   - make sure to set the path to sogo-backup.sh correctly
30 0 * * * sogo /usr/share/doc/sogo-*/sogo-backup.sh
[root@ns-srv01 cron.d]#

I have checked all other crons: no one with */2 * * * *

find and kill the session

I don’t have an unique PID, because they change randomly too.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 5727 netdata   20   0   56224   5988   1808 S   2.0  0.1   1:13.43 apps.plugin
    1 root      20   0  199872   4220   2364 S   1.3  0.1   3:25.69 systemd
 5410 root      20   0 3108932  38348  22828 S   1.3  0.5   1:08.19 f2b/server
 1313 ntopng    20   0 1690256 256780  16708 S   0.7  3.1 117:50.80 ntopng
 5588 netdata   20   0  357016 103276   3504 S   0.7  1.3   0:22.49 netdata
 8929 root      20   0  563300   6660   3892 S   0.7  0.1   0:12.46 cockpit-bridge
27550 suricata  20   0  911516 383708   4864 S   0.7  4.7  10:58.95 Suricata-Main
  761 dbus      20   0   70776   2428   1692 S   0.3  0.0   1:52.75 dbus-daemon
 1221 unbound   20   0  149816  21864   3324 S   0.3  0.3   1:04.22 unbound
 5723 netdata   20   0  167852  22052   4288 S   0.3  0.3   0:10.40 python
 8739 cockpit+  20   0  530104   7992   4448 S   0.3  0.1   0:07.40 cockpit-ws
27343 root      20   0   75444   4864   3636 S   0.3  0.1   1:36.99 openvpn
28068 root      20   0  162272   2500   1580 R   0.3  0.0   0:00.02 top
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.04 kthreadd
    4 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
    6 root      20   0       0      0      0 S   0.0  0.0   0:18.86 ksoftirqd/0
    7 root      rt   0       0      0      0 S   0.0  0.0   0:01.54 migration/0
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.75 rcu_bh

I proceeded yum autoremove

Removed:
  nethserver-memcached.noarch 0:1.1.0-1.ns7 php-pear-Mail-Mime.noarch 0:1.10.2-1.el7   php-pear-Net-IDNA2.noarch 0:0.1.1-10.el7 php-pear-Net-SMTP.noarch 0:1.7.3-1.el7
  php-pear-Net-Sieve.noarch 0:1.3.4-4.el7   python2-gflags.noarch 0:2.0-5.el7          sogo-activesync.x86_64 0:5.0.0-1.ns7     sogo-ealarms-notify.x86_64 0:5.0.0-1.ns7
  sogo-tool.x86_64 0:5.0.0-1.ns7            sope49-gdl1-mysql.x86_64 1:4.9-5.0.0.1.ns7

Dependency Removed:
  gnustep-base.x86_64 0:1.24.9-1.el7              gnustep-base-libs.x86_64 0:1.24.9-1.el7              lasso.x86_64 0:2.5.1-5.el7
  libobjc.x86_64 0:4.8.5-39.el7                   libwbxml.x86_64 0:0.11.2-4.el7.centos                memcached.x86_64 0:1.4.15-10.el7_3.1
  php-pear.noarch 1:1.9.4-21.el7                  php-pear-Auth-SASL.noarch 0:1.0.6-5.el7              php-pear-Net-Socket.noarch 0:1.0.14-1.el7
  sogo.x86_64 0:5.0.0-1.ns7                       sope49-appserver.x86_64 1:4.9-5.0.0.1.ns7            sope49-cards.x86_64 0:5.0.0-1.2.ge470e57.ns7
  sope49-core.x86_64 1:4.9-5.0.0.1.ns7            sope49-gdl1.x86_64 1:4.9-5.0.0.1.ns7                 sope49-gdl1-contentstore.x86_64 0:5.0.0-1.2.ge470e57.ns7
  sope49-ldap.x86_64 1:4.9-5.0.0.1.ns7            sope49-mime.x86_64 1:4.9-5.0.0.1.ns7                 sope49-sbjson.x86_64 1:2.3.1-5.0.0.1.ns7
  sope49-xml.x86_64 1:4.9-5.0.0.1.ns7             xmlsec1.x86_64 0:1.2.20-7.el7_4                      xmlsec1-openssl.x86_64 0:1.2.20-7.el7_4

Complete!

Hopefully the old artifacts are gone and do not cause me new problems
[root@ns-srv01 cron.d]# ll
-rw-r----- 1 root root 1231 Oct 8 22:54 sogo.rpmsave

the error persists :sob:

Did you try to remove sogo.rpmsave?

Yes, but it don’t solve the problem. I did not expect this either, because it is not a 2-minute cron.
Unfortunately there is no 2-minute cron at all to convict the culprit.

My next suspicion would be WebTop, if WebTop would manage known users somewhere.

The user USER01 appears there in the suggested user list.
Unfortunately, https://docs.nethserver.org/en/v7/webtop5.html has no advice to the used Data Base.

But the log gives a 2 minute recurring event. If that isn’t trhough cron, there must be something else doing 2 minute recurring events.

@webtop_team
Can somebody help?

I did it…
After doveadm expunge -d -u user01 mailbox '*' all

comes an error:
Error: Deleting mailbox 'INBOX' failed: INBOX can't be deleted.

…and the error log entry persists.

Sincerley, Marko

Something at the logs while trying to delete the mailbox?