MrE
(Enrique D)
November 18, 2022, 9:53pm
1
Maybe during my tests learning how to configure the filters, I did something wrong; I really don’t remember if from the beginning these errors have appeared.
Where could I check and correct this problem?
Thank you in advance.
mrmarkuz
(Markus Neuberger)
November 19, 2022, 8:48pm
2
It seems a top filter is configured for a filter type that doesn’t support it.
I’d search for filter = "top";
in the rspamd config, usually the files /etc/rspamd/local.d/multimap.conf
and /etc/rspamd/modules.d/multimap.conf
, to find the possibly wrong filter.
Here you can find the supported filters for the filter types:
https://rspamd.com/doc/modules/multimap.html#map-filters
Similar issue:
opened 10:46AM - 03 Oct 18 UTC
closed 08:19AM - 08 Oct 18 UTC
### Classification (Please choose *one* option):
* [ ] Crash/Hang/Data loss
… * [ ] WebUI/Usability
* [ ] Serious bug
* [ ] Ordinary bug
* [x] Feature
* [ ] Enhancement
### Reproducibility (Please choose *one* option):
* [X] Always
* [ ] Sometimes
* [ ] Rarely
* [ ] Unable
* [ ] I didn’t try
* [ ] Not applicable
### Rspamd version:
1.8.0, 1.7.4
### Operation system, CPU:
Ubuntu 18.04.1 LTS
### Description (Please provide a descriptive summary of the issue):
Multimap module logging 'bad search filter: text' when using filter = "text".
All works perfectly but this information in log is some confusing.
version 1.8.0
lua; multimap.lua:273: bad search filter: text
version 1.7.4
lua; multimap.lua:262: bad search filter: text
### Compile errors (if any):
### Relevant logs (see details [here](https://rspamd.com/doc/faq.html#how-to-debug-some-module-in-rspamd)):
2018-10-03 06:26:46 #6382(normal) <2ee678>; task; accept_socket: accepted connection from 192.168.150.5 port 37968, task ptr: 00007FB1CC843C80
2018-10-03 06:26:46 #6382(normal) <2ee678>; task; rspamd_task_load_message: loaded message from zstd compressed stream; compressed: 2870; uncompressed: 5631
2018-10-03 06:26:46 #6382(normal) <2ee678>; task; rspamd_message_parse: loaded message; id: <20181003032639.2C0A51425DF@mx.domain.ru>; queue-id: <2C0A51425DF>; size: 5631; checksum: <4ff3257223bf0ec6dbf577afc9ec10f4>
2018-10-03 06:26:46 #6382(normal) <2ee678>; lua; settings.lua:358: check for settings
2018-10-03 06:26:46 #6382(normal) <2ee678>; task; rspamd_mime_part_detect_language: detected part language: en
2018-10-03 06:26:46 #6382(normal) <2ee678>; lua; multimap.lua:273: bad search filter: text
2018-10-03 06:26:47 #6382(normal) <2ee678>; lua; greylist.lua:356: greylisted until "Wed, 03 Oct 2018 03:31:47 GMT", new record
2018-10-03 06:26:47 #6382(normal) <2ee678>; task; lua_task_set_pre_result: <20181003032639.2C0A51425DF@mx.domain.ru>: set pre-result to soft reject: 'Try again later'
2018-10-03 06:26:47 #6382(normal) <2ee678>; lua; neural.lua:293: SHORT ann score: 0.976
2018-10-03 06:26:47 #6382(normal) <2ee678>; lua; neural.lua:489: cannot learn ANN tSHORTE26FFBE6233D815E260: too many spam samples: 84
2018-10-03 06:26:47 #6382(normal) <2ee678>; task; rspamd_task_write_log: id: <20181003032639.2C0A51425DF@mx.domain.ru>, qid: <2C0A51425DF>, ip: 180.125.253.119, from: <wdduqwgj@atft.com>, (default: F (soft reject): [0.00/6.00] [BAYES_SPAM(4.00){100.00%;},RBL_SPAMHAUS_XBL(4.00){119.253.125.180.zen.spamhaus.org : 127.0.0.4;},HFILTER_HOSTNAME_UNKNOWN(2.50){},RBL_SENDERSCORE(2.00){119.253.125.180.bl.score.senderscore.com;},RBL_SPAMHAUS_PBL(2.00){119.253.125.180.zen.spamhaus.org : 127.0.0.11;},RBL_VIRUSFREE_BOTNET(2.00){119.253.125.180.bip.virusfree.cz : 127.0.0.2;},NEURAL_SPAM_SHORT(1.95){0.976;0;},SUBJ_EXCESS_BASE64(1.50){},FAKE_REPLY(1.00){},RDNS_NONE(1.00){},UNPRECISE_RCPT_DETAIL_FROM_SPAMMY(0.50){},FORGED_SENDER(0.30){zhanfansi305@126.com;wdduqwgj@atft.com;},MANY_INVISIBLE_PARTS(0.20){3;},MIME_HTML_ONLY(0.20){},DMARC_POLICY_SOFTFAIL(0.10){126.com : No valid SPF, No valid DKIM;none;},MIME_BASE64_TEXT(0.10){},RCVD_NO_TLS_LAST(0.10){},ARC_NA(0.00){},ASN(0.00){asn:4134, ipnet:180.96.0.0/11, country:CN;},FREEMAIL_REPLYTO(0.00){126.com;},FROM_NEQ_ENVFROM(0.00){zhanfansi305@126.com;wdduqwgj@atft.com;},GREYLIST(0.00){greylisted;Wed, 03 Oct 2018 03:31:47 GMT;new record;},HAS_REPLYTO(0.00){zhanfansi305@126.com;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_TWO(0.00){2;},RCVD_VIA_SMTP_AUTH(0.00){},R_DKIM_NA(0.00){},R_SPF_NA(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 5631, time: 935.223ms real, 35.962ms virtual, dns req: 18, digest: <4ff3257223bf0ec6dbf577afc9ec10f4>, rcpts: <info@domain.ru>, mime_rcpt: <info@domain.ru>, subject: RE: Comfortable Beanbag Chir
2018-10-03 06:26:47 #6382(normal) <2ee678>; task; rspamd_protocol_http_reply: regexp statistics: 69 pcre regexps scanned, 4 regexps matched, 175 regexps total, 12 regexps cached, 17.61k bytes scanned using pcre, 17.61k bytes scanned total
2018-10-03 06:26:47 #6382(normal) <2ee678>; protocol; rspamd_protocol_http_reply: writing compressed results: 6371 bytes before 2181 bytes after
### Expected results:
No warning text 'bad search filter: text'
### Actual results:
### Debugging information (see details [here](https://rspamd.com/doc/faq.html#how-to-figure-out-why-rspamd-process-crashed)):
### Configuration (e.g. `rspamadm configdump module`):
rspamd.conf.local:
multimap {
local_bl_subject_regexp { type = "header"; header = "Subject"; filter = "text"; map = "$CONFDIR/local_bl_subject_regexp.map.inc"; symbol = "LOCAL_BL_SUBJECT_REGEXP"; description = "Local subject blacklist with header and regexp only"; regexp = true;}
}
metric {
name = "default";
group {
name = "local";
symbol {
weight = 30;
description = "Sender subject listed in local subject blacklist with header and regexp only";
name = "LOCAL_BL_SUBJECT_REGEXP";
}
}
}
local_bl_subject_regexp.map.inc:
^.+\d{2}_\d{2}_\d{4}\s+\d{2}_\d{2}\s+\d+$
### Additional information:
2 Likes
MrE
(Enrique D)
November 19, 2022, 9:34pm
3
Thank you @mrmarkuz
Found it in the file “etc/rspamd/local.d/multimap.conf
”, there I created the template that you suggest in a post:
I wikified it, should be possible now.
To block tlds:
Create a custom template:
mkdir -p /etc/e-smith/templates-custom/etc/rspamd/local.d/multimap.conf
Create /etc/e-smith/templates-custom/etc/rspamd/local.d/multimap.conf/90tld with following content:
#
# configure tld list
#
{
if ($rspamd{SpamCheckStatus} eq 'enabled') {
$OUT .= << 'EOF'
FROM_BLACKLIST_TLD {
type = "from";
map = [
"${CONFDIR}/blacklist_from_tld.map",
];
action = "reject";
filter = "top";
symb…
And it’s working great, with those messages mentioned.
Reading the docs, I don’t know which type and/or filter to change it for, I found several options.
How to correct it without losing this protection?
#
# configure tld list
#
FROM_BLACKLIST_TLD { #change to SOMETHING_BLACKLIST_TLD ?
type = "from"; # change to: "helo" or "hostname" ?
map = [
"${CONFDIR}/blacklist_from_tld.map", #blacklist_something_tld.map ?
];
action = "reject";
filter = "top";
symbol = "FROM_BLACKLIST_TLD"; # SOMETHING_BLACKLIST_TLD ?
regexp = true;
description = "Refused list of FROM TLD";
It’s an idea, and I could be very wrong:
FIXED_BLACKLIST_TLD {
type = "helo"; # or "hostname" ?
map = [
"${CONFDIR}/blacklist_fixed_tld.map",
];
action = "reject";
filter = "top";
symbol = "FIXED_BLACKLIST_TLD";
regexp = true;
description = "Refused list of FIXED TLD";
}
Reading all the configured maps and documents, I’m already getting an idea but I can’t find the correct solution.
Regards
mrmarkuz
(Markus Neuberger)
November 19, 2022, 11:00pm
4
I’d suggest to change the filter and not the type:
filter = "email:domain:tld"
instead of
filter = "top"
to block the mail address domain tld and not the mailserver tld so the “From” type is needed.
3 Likes
MrE
(Enrique D)
November 19, 2022, 11:11pm
5
I need to ask you:
Will the change continue to block these spam cases (with viruses)?
Lately, this is the most dangerous type of mail we have managed to block, as they pretend to be from “legitimate” domains and it is not possible to block them with the “from” address.
Luckily, when they try to impersonate my server, with the symbol DMARC_POLICY_REJECT (20) they can no longer get in (from your tips).
mrmarkuz
(Markus Neuberger)
November 19, 2022, 11:39pm
6
Yes as other filters or customized symbols shouldn’t be affected by the change.
Please test if the tld reject really works, I didn’t try it.
2 Likes
MrE
(Enrique D)
November 20, 2022, 12:20am
7
Filter it’s working but the message still appears:
mrmarkuz
(Markus Neuberger)
November 20, 2022, 12:50am
8
Did you run
signal-event nethserver-mail-filter-update
to write the config files and restart rspamd?
1 Like
MrE
(Enrique D)
November 20, 2022, 4:58am
9
Yes @mrmarkuz , I forgot to tell.
(10:03pm) No, wait a minute. I edited the WRONG .conf file and the changes were not saved.
(10:14pm) I reverted the changes…
MrE
(Enrique D)
November 20, 2022, 5:19am
10
This day I have a lot of attention lapses… I start all over again:
cat /etc/e-smith/templates-custom/etc/rspamd/local.d/multimap.conf/90tld
#
# configure tld list
#
{
if ($rspamd{SpamCheckStatus} eq 'enabled') {
$OUT .= << 'EOF'
FROM_BLACKLIST_TLD {
type = "from";
map = [
"${CONFDIR}/blacklist_from_tld.map",
];
action = "reject";
#filter = "top";
filter = "email:domain:tld"
symbol = "FROM_BLACKLIST_TLD";
regexp = true;
description = "Refused list of FROM TLD";
}
EOF
}
}
(10:22pm) I am no longer getting errors, but I have to wait for spam.
…
(11:15pm) @mrmarkuz It is working and no error messages!
Thank you!
1 Like