Pop3 connector - emails re-dowloaded

Just a blind shot:

Could it be you have a full mailbox or are over quota?

1 Like

Indeed there was something strange last days on the same topic

i deleted the mails from the provider and set the retrieval time to 60 minutes

i have done a backup of the server configuration before the update. so i’m not so sure what caused the problem anymore, because the error always had a latent after changes.

https://community.nethserver.org/t/pop3-connector-no-longer-deletes-mails/11474/2

It is not a problem of the quota. no user exceeds 30%

Can you identify duplicates in the logs (msgid)?
Do /var/lib/getmail/oldmail* files get updated?

I see the following in one of the files

oldmail-mail.dominio.com-993-ttintoayo@dominio.com-INBOX
1240220693/41528^@1543332001
1240220693/41529^@1543397401
1240220693/41522^@1542893701
1240220693/41523^@1542899102
1240220693/41520^@1542879602
1240220693/41521^@1542887101
1240220693/41526^@1543312201
1240220693/41527^@1543320301
1240220693/41524^@1542966301
1240220693/41525^@1543231802
1240220693/41548^@1544565901
1240220693/41549^@1544603402
1240220693/41540^@1543917302
1240220693/41541^@1544018101
1240220693/41542^@1544018101
1240220693/41543^@1544176501
1240220693/41544^@1544433002
1240220693/41545^@1544434502
1240220693/41546^@1544527201

I have configured the pop3 service to delete messages immediately upon download.

neither does it eliminate them from the mail provider.

Sorry for my bad english.

After update to 7.6.1810, I have the same problem, emails are re-downloaded with a dovecot message error delivery in logs. If I set disk space to unlimited the problem disappears, but I have no quota anymore. Is there any solution ?

1 Like

This seems a Bug. I’ve done some tests, and as @kunstlust and @samuel_grenier found, when a quota is set under Email → Mailboxes → disk space: apply quota, we get a warning and email messages are not deleted or are fetched again. (what follows is an excerpt):

Global setting mail_plugins won’t change the setting inside an earlier filter at /etc/dovecot/dovecot.conf

Problems might come from declaration of mail_plugins global setting in dovecot.conf…:

# Enable acl plugin for shared mailboxes, 
# and listescape to extend allowable characters in mailbox names
mail_plugins = $mail_plugins acl listescape fts fts_lucene
protocol imap {
  mail_plugins = $mail_plugins imap_acl
}

#
# Quota configuration is enabled
#
mail_plugins = $mail_plugins quota   # <----

protocol imap {
  mail_plugins = $mail_plugins quota imap_quota
}
[root@server ~]# dovecot -n
# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.24 (124e06aa)
doveconf: Warning: /etc/dovecot/dovecot.conf line 276: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/dovecot.conf line 202 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/dovecot.conf line 202)
doveconf: Warning: /etc/dovecot/dovecot.conf line 276: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/dovecot.conf line 202 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/dovecot.conf line 202)
doveconf: Warning: service auth { client_limit=2000 } is lower than required under max. load (2400)
# OS: Linux 3.10.0-957.1.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core)  
...

'Re: delete command' - MARC
there is some sort of problem suddenly happened error when getmail hands over to dovecot deliver and it reports error which stops getmail sending delete command I guess

Yes. If your MDA (delivery program) fails, getmail doesn’t tell the server to delete the message, so it can try again next time.

'SOLVED: getmail 5.6: No deletion of getmail-ed emails anymore' - MARC
As dovecot-lda send these warnings to STDERR, getmails seems to treat them as error and DOES NOT DELETE the getmail-ed msgs anymore.

Similar to this older one: https://www.dovecot.org/list/dovecot/2016-April/103995.html

1 Like

thank a lot for diving, card created https://github.com/orgs/NethServer/projects/1#card-15934419

1 Like

ok for what I saw this should be related to dovecot, at least it is the major bump in the rpm version

[root@prometheus ~]# rpm -qa dovecot getmail
getmail-5.6-1.el7.noarch
dovecot-2.2.10-8.el7.x86_64

#upgraded to centos7.6

[root@ns7loc12 ~]# rpm -qa dovecot getmail
getmail-5.6-1.el7.noarch
dovecot-2.2.36-3.el7.x86_64

we have also this in /var/log/messages

Dec 20 22:01:47 ns7loc12 systemd: PID file /var/run/dovecot/master.pid not readable (yet?) after start.

ok at least it clear now, when you trigger getmail manually

  msg    20/10669 (18537 bytes), delivery error (command dovecot-lda 6685 wrote to stderr: doveconf: Warning: /etc/dovecot/dovecot.conf line 262: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/dovecot.conf line 188 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/dovecot.conf line 188))
Delivery error (command dovecot-lda 6689 wrote to stderr: doveconf: Warning: /etc/dovecot/dovecot.conf line 262: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/dovecot.conf line 188 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/dovecot.conf line 188))

in dovecot.conf

service auth {
-    client_limit = 2000
+    client_limit = 2500
}


- mail_plugins = $mail_plugins acl listescape fts fts_lucene 
+ mail_plugins = $mail_plugins acl listescape fts fts_lucene quota

protocol imap {
-  mail_plugins = $mail_plugins imap_acl
+ mail_plugins = $mail_plugins imap_acl quota imap_quota
}


#
# Quota configuration is enabled
#
- mail_plugins = $mail_plugins quota

- protocol imap {
-  mail_plugins = $mail_plugins quota imap_quota
- }

/etc/systemd/system/dovecot.service.d/limits.conf

[Service]
-  LimitNOFILE=2000
+ LimitNOFILE=2500
+ Group=vmail

systemctl daemon-reload
systemctl restart dovecot

in /etc/cron.d/getmail you have the command line to start manually getmail and check if no error

now here : msg 26/10669 (1707 bytes) delivered, deleted

/usr/bin/flock -n -E 0 /var/lib/getmail/stephane.xxx@xxx.xx.cfg -c "/usr/bin/getmail --getmaildir /var/lib/getmail/ --rcfile /var/lib/getmail/stephane.xxx@xxx.xx.cfg "

I will try to patch tomorrow

a patch to test for those are concerned with this bug

yum install http://packages.nethserver.org/nethserver/7.6.1810/autobuild/x86_64/Packages/nethserver-mail-server-2.4.2-1.3.pr99.gd760317.ns7.noarch.rpm

to go back at the original

yum downgrade nethserver-mail-server

Patch accepted, we need to verify it…Xmas excuses are not accepted :smiley:

Really thank you in advance.

https://github.com/NethServer/dev/issues/5679

1 Like

On the QA note, is nethserver-getmail needed or it’s nethserver-mail-getmail?
After a quick test it seems to work, mail is deleted from server and not duplicated.

1 Like

yep nethserver-mail-getmail, sorry for the mistake and tk a lot @dnutan you are my hero

just a question you upgraded nethserver-mail-server from nethserver-testing

Yes, upgraded all nethserver-mail-* packages from testing repo.

1 Like

Just updated nethserver-mail-* through the update manager and tested. It works, no more emails re-downloaded. Thanks for all.

1 Like