Nethserver-rspamd a new module

mail
v7

(Stéphane de Labrusse) #1

Hi

Just a teaser now, I started to work on rpsamd I have something basic with still a lot more things to do. However I would be interested if some of you know it, use it, and have tips of course on features to introduce and others to flee.


Sophos vs Clamav: who's the best antivirus?
Email 2 (Beta) released
Server-side mail filtering?
Unknown User p3scan after mail2-p3scan update
(Markus Neuberger) #2

Seems to be a full featured spam filter, looking forward to first test release…

https://rspamd.com/features.html
https://rspamd.com/comparison.html


(Davide Principi) #3

https://rspamd.com/doc/integration.html

Postfix milter :yum:


(Markus Neuberger) #4

I like this spam filter idea so much I installed it to try it and it’s working but I didn’t test much now,
If someone likes to test, here is a quick and dirty install copy/paste just to get into it.

curl https://rspamd.com/rpm-stable/centos-7/rspamd.repo > /etc/yum.repos.d/rspamd.repo
rpm --import https://rspamd.com/rpm-stable/gpg.key
yum -y update
yum -y install nethserver-mail-server rspamd nethserver-redis
cat > /etc/e-smith/templates/etc/postfix/main.cf/90rspamdmilter << EOL
#
# 90rspamdmilter
#
#smtpd_milters = unix:/var/lib/rspamd/milter.sock
# or for TCP socket
smtpd_milters = inet:localhost:11332
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
# skip mail without checks if something goes wrong
milter_default_action = accept
EOL
cat > /etc/httpd/conf.d/rspamd.conf << EOL
<Location /rspamd>
        Order allow,deny
        Allow from all
</Location>
RewriteRule ^/rspamd\$ /rspamd/ [R,L]
RewriteRule ^/rspamd/(.*) http://localhost:11334/$1 [P,L]
EOL
expand-template /etc/postfix/main.cf
signal-event nethserver-mail-server-update
systemctl enable rspamd
systemctl start rspamd
systemctl restart httpd

Using the gtube pattern in mail text

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

and roundcube doesn’t send my “bad” mail anymore…I could send the mail with spamassassin but I don’t know if it’s filtering locally.

The logfile is /var/log/rspamd/rspamd.log:

2017-12-08 01:07:20 #18975(normal) <6f669f>; task; accept_socket: accepted connection from 127.0.0.1 port 50928, task ptr: 000000000244C780
2017-12-08 01:07:20 #18975(normal) <6f669f>; task; rspamd_check_gtube: <2afed2d2804c4607418b0c3bac7635af@domain.local>: gtube pattern has been found in part of length 70
2017-12-08 01:07:20 #18975(normal) <6f669f>; task; rspamd_message_parse: loaded message; id: <2afed2d2804c4607418b0c3bac7635af@domain.local>; queue-id: <CFC6F30A6AF7>; size: 625; checksum: <7be1974381651e59cd09135164e0339d>
2017-12-08 01:07:20 #18975(normal) <6f669f>; task; rspamd_task_write_log: id: <2afed2d2804c4607418b0c3bac7635af@domain.local>, qid: <CFC6F30A6AF7>, ip: 127.0.0.1, from: <admin@domain.local>, (default: S (reject): [15.00/15.00] [GTUBE(0.00){}]), len: 625, time: 0.000ms real, 0.213ms virtual, dns req: 0, digest: <7be1974381651e59cd09135164e0339d>, rcpts: <markus@domain.local>, mime_rcpt: <markus@domain.local>
2017-12-08 01:07:20 #18975(normal) <6f669f>; task; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 0 regexps matched, 163 regexps total, 0 regexps cached, 0B bytes scanned using pcre, 0B bytes scanned total

I’ll get more into rspamd and then I’d like to compare it to spamassassin when filtering some of my mails and maybe find out some interesting features.


(Stéphane de Labrusse) #5

Take a look to my github , clone the repo, build it with make-rpms, play a bit

I know a bug with the reverse proxy


(Markus Neuberger) #6

I tried it now. Looks really nice, I have to learn about the features…

I put the RPM on my webhost so others can try or do you have it online somewhere? If it’s ok for you?

curl https://rspamd.com/rpm-stable/centos-7/rspamd.repo > /etc/yum.repos.d/rspamd.repo
rpm --import https://rspamd.com/rpm-stable/gpg.key
yum install http://markusneuberger.at/download/nethserver-rspamd-0.1.0-1.ns7.noarch.rpm

Spamassassin Bayesian filter issues
(Stéphane de Labrusse) #7

congratulation, a developer is born :slight_smile:

It is a bit soon to share it, I would go to redis database instead of sqlite (needed if we want the antivirus feature for example)

My aim is to replace nethserver-mail-filter and it is a lot of work. This rpm will be available by the software-center since it is part of a work for nethesis


Howto install Zabbix 3.4
(Matthieu Gaillet) #8

I’m ready to help if needed.


(Stéphane de Labrusse) #9

(Alessio Fattorini) #10

:cake: :baby:

Give this man a prototype :slight_smile:


(Stéphane de Labrusse) #11

let me few days, I’m testing it on my own server, it is an alpha, seems workable but you know the music :smiley:


(Stéphane de Labrusse) #12

wait a bit, just a bit…I would love to have something workable for the end of the month…I hope so


(Matthieu Gaillet) #13

Je t’attends de pied ferme ! :slight_smile:


(Stéphane de Labrusse) #14

still things to do :smiley:


(Alessio Fattorini) #15

:scream: gorgeous


(Stéphane de Labrusse) #16

@davidep and @giacomo, please comment

##dovecot-antispam

rspamd can learn like spamassassin by moving the email to the inbox or the junk email box by something really easy
This is the piece of code, sure you know it

protocol imap {
  mail_plugins = $mail_plugins antispam
}


plugin {
    antispam_backend = mailtrain
    antispam_spam    = Junk
    antispam_trash   = Trash
    antispam_mail_sendmail = /usr/bin/rspamc
    antispam_mail_spam     = learn_spam
    antispam_mail_notspam  = learn_ham
    antispam_mail_sendmail_args = -h;localhost:11334;-P;TheRspamdPassword
}

the original code comes from

[root@ns7dev5 ~]# rpm -qf /etc/e-smith/templates/etc/dovecot/dovecot.conf/50spam_training 
nethserver-mail-server-1.10.18-1.ns7.noarch

I would know how I can add my code and replace the code you can see below

plugin {
    antispam_backend = MAILTRAIN
    antispam_spam = Junk
    antispam_trash = Trash
    antispam_mail_sendmail = /usr/libexec/nethserver/spam-training.sh
    antispam_mail_spam = spam
    antispam_mail_notspam = ham
    antispam_mail_sendmail_args = %u 
}

I could propose some solutions, not yet really look after, I’m thinking loudly

  1. Move /etc/e-smith/templates/etc/dovecot/dovecot.conf/50spam_training to nethserver-mail-filter (or any other rpm) and create also in nethserver-rspamd a 50spam_training. Anyway I think that we need to set nethserver-mail-filter as a conflict of rspamd (I don’t see interest to get two spam/antivirus filters)

  2. modify /usr/libexec/nethserver/spam-training.sh and detect if /usr/bin/rspamc is installed, then use this binary to learn if it is a spam or not

I would tend for the solution 2

##Amavisd-new

For now I did not modify the rpm nethserver-mail-common with the amavisd-new, it works like that

[root@ns7dev5 ~]# cat /etc/e-smith/templates/etc/postfix/main.cf/90rspamd 
#smtpd_milters = unix:/var/lib/rspamd/milter.sock
# or for TCP socket
smtpd_milters = inet:localhost:11332
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
# skip mail without checks if something goes wrong
milter_default_action = accept

I simply did not install nethserver-mail-filter. I prefer a modular approach and let the sysadmin to remove nethserver-mail-filter and install manually nethserver-rspamd.

I’m not a big fan to remove/bypass the amavisd if it works like that but do we have to go in the direction to deactivate amavisd if rspamd is installed ?


(Davide Principi) #17

:thumbsup: yes

I’d add an “Obsoletes nethserver-mail-filter” and “Requires nethserver-mail-common > xyz” to nethserver-rspamd. We should provide just one solution to fight spam that works (and amavisd/spamassassin has many issues today).

One of these issues is the disclaimer feature, implemented by nethserver-mail-common (IIRC): amavisd relay a message through spam checks, AV checks and altermime too…


(Stéphane de Labrusse) #18

I suppose you speak of this

Dec 12 00:52:02 prometheus spam-training.sh/stephdl@de-labrusse.fr[32324]: Message enqueued as spam
Dec 12 00:52:02 prometheus postfix/qmgr[26835]: DA9D51807E91B: from=<root@de-labrusse.fr>, size=101219, nrcpt=1 (queue active)
Dec 12 00:52:06 prometheus postfix/pipe[32325]: DA9D51807E91B: to=<stephdl@de-labrusse.fr+spam@spamtrain.nh>, relay=spamtrain, delay=4.3, delays=0.53/0.05/0/3.7, dsn=2.0.0, status=sent (delivered via spamtrain service)
Dec 12 00:52:06 prometheus postfix/qmgr[26835]: DA9D51807E91B: removed

(Davide Principi) #19

That is the “spam training” feature, implemented on mail-server by the Dovecot plugin. I see no problem on replacing its backend from spam-training.sh to rspamc


(Filippo Carletti) #20

The antispam dovecot plugin is now deprecated:
https://wiki2.dovecot.org/Plugins/Antispam
We should replace it with IMAPSieve:
https://wiki2.dovecot.org/HowTo/AntispamWithSieve