Nethserver-rspamd a new module

stephdl · February 21, 2018, 9:17am

working smooth on my side, try to restart redis-rspamd and rspamd

systemctl restart rspamd-redis rspamd

as a side note, we agreed with @davidep to release by the nethserver repository the version of rspamd, obviously we need to test them

filippo_carletti · February 21, 2018, 9:23am

Same timeouts here:

grep "got IO timeout" /var/log/maillog | wc -l
2199

stephdl · February 21, 2018, 9:45am

@hucky @filippo_carletti sorry for my mistake…still in holiday

the IO error is relative to fuzzy check database of rspamd.com (the main website)

according to this

https://groups.google.com/forum/#!topic/rspamd/OtizdoxmLFs
https://groups.google.com/forum/#!topic/rspamd/5_Zc2Ynm4Dc

it could be a firewall issue (UDP 11335) or a routage problem, please try it, I cannot reproduce it.

PS…time to play with my daughter

hucky · February 21, 2018, 10:03am

i see that the error comes from the controller:

21.2.2018, 11:00:03 controller 17793 csession c2ac55 got IO timeout with server rspamd.com(88.99.142.95), after 3 retransmits

but in any case it seems not to be every time it makes the check.

i will try to figure out more.

PS Good Reason to do a break, wish you a nice time with your daughter

davidep · February 21, 2018, 11:28am

Just added p3scan module to the mail2 bundle.

New test cases 1.4 and 2.4, added here /cc @quality_team

https://github.com/NethServer/dev/issues/5394#issuecomment-367295232

Updated docs

http://docs.nethserver.org/en/latest/mail2.html#from-pop3-proxy-module

This issue requires QA tests too:

davidep · February 21, 2018, 11:57am

A post was split to a new topic: Builtin block port 25

hucky · February 21, 2018, 1:43pm

so have figured out the problem with the error of the controller, seems that a synch over the proxy not work.
so i do bypass the proxy for the rspamd domain and it seems to work now

hucky · February 21, 2018, 1:46pm

hmm, do we have to install the pop3 proxy from software-center?

guess not… it is implemented, right?

davidep · February 21, 2018, 1:49pm

During QA it’s not available from Software Center. Required installation/upgrade commands are documented here

https://github.com/NethServer/dev/issues/5394#issuecomment-367295232

stephdl · February 21, 2018, 5:08pm

@davidep, do we need to think to bypass the proxy for rspamd.com, rspamd is permanently asking request to this domains, but it does it also for many Blacklist websites also

filippo_carletti · February 21, 2018, 5:14pm

The traffic from NethServer itself is not proxied.
AFAIU, the service running on port 11335 on rspamd.com is down.

[root@n ~]# host rspamd.com
rspamd.com has address 88.99.142.95
rspamd.com mail is handled by 10 mail.highsecure.ru.
[root@n ~]# nc -v 88.99.142.95 11335
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: No route to host.

filippo_carletti · February 22, 2018, 5:27pm

My mistake: the rspamd.com service runs on udp:

[root@n ~]# nc -u -v 88.99.142.95 11335
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 88.99.142.95:11335.

Using tcpdump on the firewall I can see traffic going back and forth:

[root@n ~]# tcpdump -nn -p -i en2 host 88.99.142.95 -s 1500 -A
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en2, link-type EN10MB (Ethernet), capture size 1500 bytes
18:13:31.055404 IP 93.57.48.69.52416 > 88.99.142.95.11335: UDP, length 160
E....F@.?...]90EXc._..,G..F.rsfe...k...z..l.e2.
...M.7.p:g..\..y.`$..rz.z.8.r....j.(lq.......s_....b}.....AI.c...TG....D...D.n.`i......So..6.....m...`.P.Mz..H.if`..!.F.@T..C.FggH..6..I.4.\
18:13:31.089727 IP 88.99.142.95.11335 > 93.57.48.69.52416: UDP, length 56
E..T..@.4..HXc._]90E,G...@...i........_....._..s.B..?(c.AB...I..Z....$!.u-..%|....>/

But the timeout errors are still there in the logs.

sharpec · February 26, 2018, 5:47pm

Hi all,
the desire to try your work is always stronger than the stability of the system.

after i try to update existing enviroment pop3 proxy to testing mail2-p3scan

i found this in /var/log/message

systemd: Starting p3scan proxy...
Feb 26 18:03:01 proxy p3scan[27849]: P3Scan Version 2.3.2
Feb 26 18:03:01 proxy p3scan[27849]: Selected scannertype: basic (Basic file invocation scanner)
Feb 26 18:03:01 proxy p3scan[27849]: Listen now on 0.0.0.0:8110
Feb 26 18:03:01 proxy systemd: Started p3scan proxy.
Feb 26 18:03:01 proxy p3scan: /bin/chown: invalid user: ‘p3scan:p3scan’
Feb 26 18:03:01 proxy p3scan[27850]: Changing uid (we are root)
Feb 26 18:03:01 proxy p3scan[27850]: ERR: Unknown User 'p3scan'
Feb 26 18:03:01 proxy p3scan[27850]: ERR: Exiting now...
Feb 26 18:03:01 proxy p3scan: ERR: Unknown User 'p3scan'

i have manually add user
useradd -d /var/spool/p3scan -g p3scan -M -s /sbin/nologin p3scan

and fix permission (old amavis)
chown p3scan:p3scan -R /var/spool/p3scan
chown p3scan:p3scan -R /var/run/p3scan

now everything works fine
PS: in a clean setup i havent this problem

davidep · February 26, 2018, 5:46pm

Hi @sharpec, happy to see you again!

Thank you very much for your test I’ll check out p3scan package again tomorrow!

As the package is still in testing I move this discussion under the rspamd thread.

sharpec · February 26, 2018, 8:32pm

in this period I could not read much of the forum, but this rspamd story made my “fingers prick”.

allow me to report during the upgrade

Feb 26 21:05:52 proxy systemd: Reloading.
Feb 26 21:05:52 proxy esmith::event[5283]: [INFO] service postfix is disabled: skipped

config setprop postfix status enabled
signal-event nethserver-mail-server-save

correct?

davidep · February 26, 2018, 9:25pm

Postfix disabled in DB?

It’s really strange: I’m not aware of any code that alters that prop.

stephdl · February 26, 2018, 10:07pm

something in logs messages or maillog ???

sharpec · February 27, 2018, 2:01pm

ok, i have found this at time of yum swap

messages

Feb 26 17:20:25 proxy esmith::event[2992]: expanding /etc/postfix/main.cf
Feb 26 17:20:25 proxy esmith::event[2992]: expanding /etc/postfix/sasl_passwd
Feb 26 17:20:25 proxy esmith::event[2992]: expanding /etc/postfix/tls_policy
Feb 26 17:20:28 proxy postfix: /usr/sbin/postconf: warning: /etc/postfix/main.cf: unused parameter: submission_smtpd_milters=unix:/var/run/rspamd/worker-proxy, unix:/var/run/opendkim/milter
Feb 26 17:20:28 proxy esmith::event[2992]: [INFO] service postfix is disabled and has been stopped
Feb 26 17:21:59 proxy esmith::event[6388]: expanding /etc/postfix/access.cidr
Feb 26 17:21:59 proxy esmith::event[6388]: expanding /etc/postfix/main.cf
Feb 26 17:21:59 proxy esmith::event[6388]: expanding /etc/postfix/master.cf
Feb 26 17:21:59 proxy esmith::event[6388]: expanding /etc/postfix/mynetworks.cidr
Feb 26 17:21:59 proxy esmith::event[6388]: expanding /etc/postfix/transport
Feb 26 17:22:00 proxy esmith::event[6388]: [INFO] service postfix is disabled: skipped

maillog

Feb 26 17:20:28 proxy postfix/postfix-script[3032]: stopping the Postfix mail system
Feb 26 17:20:28 proxy postfix/master[18577]: terminating on signal 15
Feb 26 17:30:17 proxy postfix/postqueue[11960]: warning: Mail system is down -- accessing queue directly

stephdl · February 27, 2018, 8:17pm

this was during the installation of what packages ?

maybe you could past a gist of the full log please

davidep · March 8, 2018, 5:17pm

A post was split to a new topic: Email 2 (Beta) released