How to fight the incredible amount of spam?


(Dave Land) #1

This may or may not be related, but something got corrupted in the shorewall database and essentially kept me from adding or removing any more machines from the domain. That plus the incredible amount of spam (over 500 per day in my junkmail on T-Bird), forced me to take the server down and put my power-hungry, but working, IBM x-345 back to work with SME server. After a couple updates from being down so long, everything works again, and my junk mail has dropped back down to about 30 or 40 per day. I’ll research it more when I have time, but things are pretty crazy right now business-wise, and I’m here by myself. :stuck_out_tongue:


Shorewall slows down shuttdown
(Filippo Carletti) #2

I can’t think of a relation between shorewall and domain join. shorewall logs to firewall.log, clues could be found there.

I’m running an enhanced antispam since a couple of days and it seems really effective. I’m going to test your spam against it and let you know how it goes.


(Dave Land) #3

There appears to be some relation between shorewall and a lot of other operations, because it updates the rules each time you add or remove a new machine in DHCP. The error I was getting was: Error at line 175 in /etc/shorewall/?? (can’t remember the name of the file).

The machine is offline right now, and I’ll have to set it up with a sandboxed network so I can get back into the web interface and replicate the error message. There was something about a missing destination point that caused it. I tried editing the file to remove the offending entry/machine since it’s no longer on the network anyway, but as soon as I restarted shorewall service, it regenerated the original file of course, and then it was refusing to hand out IP addresses to any new machine I tried to add via DHCP.
Anyway, it will be a few days minimum before I have time to set it back up in a sandbox to diagnose further…

Dave L.


(Gabriel GHEORGHIU) #4

Hi Dave,

I don’t think is about shorewall.
I think it’s about a “Spam Attack”.
There are alternate massive attacks.
Look at my registrations.

Best regards,
Gabriel


(Filippo Carletti) #5

I bet it’s /etc/shorewall/rules.
We had a bug for a few days in rules generated for portforwards when you had the ips enabled.
Every rules has a comment that helps understanding where it come from and how to fix it.


(Dave Land) #6

Exactly! That is the file. When I tried to do port forwarding for a machine I just put back on line is when it first popped up the error message, and although it appeared to let me add the machine, it wasn’t handing out an IP address to it specifically via the hardware address (i.e. 20:10:75:6e:32:4b = 192.168.0.2) Instead the machine would just be assigned an arbitrary address in the DHCP range (192.168.0.243) Anyway, as soon as I get some free time, I’ll look into it further.

In regards to the spam attack, yes we always get a lot of spam on this domain, but SpamAssassin seems to work much better on SME for some reason, since it cut the junk mail down to less than 1/4 of what it was before. I don’t know what the differences are between how it’s configured in NethServer vs. SME server, that would cause that much of a difference.

Dave L.


(Filippo Carletti) #7

NethServer antispam relies heavily on user training, it will behave much better over time.
As I said, I’m experimenting with black lists, I’ll let you know.


(Gabriel GHEORGHIU) #8

Hello,

How blacklist module looks in Endian - SMTP proxy (linking with the comparison between NS and Endian):


(Artem Fedai) #9

How about Dnsbl database, there are lot of them and it reduse spam wery well


(Filippo Carletti) #10

NethServer would need a “full” dns server to properly use URIBL (they block queries coming from popular dns).
I have unbound dns running since a couple of weeks, I’ll post results when I have enough stats about spam.


(Filippo Carletti) #11

@xmechanic provided me 38 unrecognized spam samples.
Some of them didn’t have spamassassin tags, so I suspect they were not received by NethServer. You can spot them below because of the missing X-Spam-Score line.
However, only one mail was not marked as spam by my setup (and it got 4.8 points).
Legend: msg number, optional original score, my score.

1
Content analysis details:   (21.0 points, 5.0 required)
2
Content analysis details:   (16.7 points, 5.0 required)
3
X-Spam-Score: 3.787
Content analysis details:   (18.3 points, 5.0 required)
4
Content analysis details:   (19.2 points, 5.0 required)
5
Content analysis details:   (22.7 points, 5.0 required)
6
Content analysis details:   (4.8 points, 5.0 required)
7
Content analysis details:   (8.7 points, 5.0 required)
8
X-Spam-Score: 2.232
Content analysis details:   (19.6 points, 5.0 required)
9
X-Spam-Score: 2.676
Content analysis details:   (8.7 points, 5.0 required)
10
Content analysis details:   (21.0 points, 5.0 required)
11
X-Spam-Score: 3.927
Content analysis details:   (9.7 points, 5.0 required)
12
Content analysis details:   (18.3 points, 5.0 required)
13
Content analysis details:   (7.0 points, 5.0 required)
14
Content analysis details:   (15.7 points, 5.0 required)
15
Content analysis details:   (15.7 points, 5.0 required)
16
X-Spam-Score: 3.294
Content analysis details:   (15.4 points, 5.0 required)
17
X-Spam-Score: 4.939
Content analysis details:   (12.4 points, 5.0 required)
18
X-Spam-Score: 3.54
Content analysis details:   (13.7 points, 5.0 required)
19
Content analysis details:   (15.7 points, 5.0 required)
20
X-Spam-Score: 2.784
Content analysis details:   (20.9 points, 5.0 required)
21
X-Spam-Score: 2.95
Content analysis details:   (32.4 points, 5.0 required)
22
Content analysis details:   (13.5 points, 5.0 required)
23
X-Spam-Score: 4.672
Content analysis details:   (11.4 points, 5.0 required)
24
X-Spam-Score: 4.51
Content analysis details:   (23.6 points, 5.0 required)
25
Content analysis details:   (15.7 points, 5.0 required)
26
Content analysis details:   (17.6 points, 5.0 required)
27
X-Spam-Score: 2.228
Content analysis details:   (10.2 points, 5.0 required)
28
X-Spam-Score: 2.812
Content analysis details:   (18.3 points, 5.0 required)
29
Content analysis details:   (8.0 points, 5.0 required)
30
Content analysis details:   (18.3 points, 5.0 required)
31
Content analysis details:   (10.2 points, 5.0 required)
32
Content analysis details:   (8.7 points, 5.0 required)
33
Content analysis details:   (24.7 points, 5.0 required)
34
Content analysis details:   (15.7 points, 5.0 required)
35
X-Spam-Score: 3.74
Content analysis details:   (15.7 points, 5.0 required)
36
Content analysis details:   (9.3 points, 5.0 required)
37
Content analysis details:   (8.7 points, 5.0 required)
38
X-Spam-Score: 2.676
Content analysis details:   (13.2 points, 5.0 required)

(Dave Land) #12

I like the looks of the blacklist module in Endian. Very configurable, and straightforward control of blacklists and whitelists. Thanks GG_jr for the pictures of the interface. I wonder how much trouble it would be to incorporate that into NethServer? I’m going to try to get my machine back up tonight (MST), in a sandbox of sorts, and maybe give filippo_carletti access, so he can look directly at what is going on (if you’re interested) :slight_smile: I’ll e-mail you and give you the login info at that time…

Dave L.


(Alessio Fattorini) #13

That’s great! Please, could you keep us in touch opening a new topic? :smiley: I’m curious about your results!


(Stefano) #14

I need a clarification…

IIUC, NS doesn’t use (ATM) any kind of DNSBL/RHSBL to fight spam…

does it mean that every mail is received and filtered and, if spammy, discarded?

if so, the [DNS|RHS]BL is a must

moreover, remember that using (by design or as default value) Google’s DNS will break use of URIBL (see other related topic)


(Artem Fedai) #15

Hi all so workaround is :

mkdir -p /etc/e-smith/templates-custom/etc/postfix/main.cf

cp /etc/e-smith/templates/etc/postfix/main.cf/01filter_strict_checks  /etc/e-smith/templates-custom/etc/postfix/main.cf/01filter_strict_checks 

edit those section

vi /etc/e-smith/templates-custom/etc/postfix/main.cf/01filter_strict_checks 

# insert reject_non_fqdn_recipient before address verification
@smtpd_recipient_restrictions = map { $_ eq 'reject_unverified_recipient' ? ('reject_non_fqdn_recipient', $_) : $_ } @smtpd_recipient_restrictions,
'reject_rbl_client ips.backscatterer.org',
'reject_rbl_client dnsbl.proxybl.org',
'reject_rbl_client b.barracudacentral.org',
'reject_rbl_client zen.spamhaus.org';

signal-event nethserver-mail-common-update


(Filippo Carletti) #16

I’m not sure I’m understanding the question. We have two levels for spam:

  1. mark (tag)
  2. discard

2 has to be higher than 1. Suggested value for mark is 5, discard is optional (I usually set it to 10 or 12).

URIBL could be made to work using a cache dns (I’m using unbound, I plan to find time to finish implementation next week).


(Filippo Carletti) #17

@Nas, I think we could enable some RBL following the manual (http://docs.nethserver.org/projects/nethserver-devel/en/latest/email.html#rbl-server-list), no need for a template-custom, do you agree?

db configuration setprop postfix RblStatus enabled RblServers zen.spamhaus.org,b.barracudacentral.org,dnsbl.proxybl.org,ips.backscatterer.org

(Artem Fedai) #18

i think it should be field in WebUI where we could input RBL servers


(Stefano) #19

I try to explain better myself :smile:

using DNSBL/RHSBL spam email are blocked at the first stage of email transaction… it means that, de facto, a blocked email will never be received at all, there’s no data session
withous this kind of filtering, using only spamassassing or similar approach, the whole smtp transaction has place, and the whole mail is analyzed and eventually filetered/tagged/discarded…

there’s a big difference in terms of traffic and bandwidth


(Stefano) #20

just be aware that barracuda is quite aggressive and the guys are not so reactive in maintaining their list…