Rspamd 2.1 Update request

Feature
41

I installed on a production server and found a puzzling log entry. It saids 89 learned, however the dashboard says Bayes are fully trained.

I guess some rspamd internal counters are not aligned as I expect :thinking: /cc @stephdl

2 Likes
42

I suspect that the bayes counter has changed, where the dashboard takes its counter ?

on my server it seems relevant between what rspamc stat and what I can find in logs

/var/log/maillog-20191124:122128:Nov 24 08:16:30 prometheus rspamd[27027]: <6dd2d0>; proxy; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_SPAM: not enough learns 116; 200 required
/var/log/maillog-20191124:122129:Nov 24 08:16:30 prometheus rspamd[27027]: <6dd2d0>; proxy; rspamd_redis_finalize_process: cannot retreive stat tokens from Redis: skip obtaining bayes tokens for BAYES_HAM: not enough learns 57; 200 required

Once you have reached the minimal score (200) this message does not appear in log anymore

43

maybe I will need some logs evidence and output of rspamc stat to report to upstream

44

Well during the weekend the learns counter grew rapidly and reached 200.

Dec 15 00:19:12 nethservice rspamd[15843]: <5fb59c>; proxy; rspamd_redis_connected: skip obtaining bayes tokens for BAYES_SPAM of classifier bayes: not enough learns 199; 200 required
Dec 15 00:54:32 nethservice rspamd[15843]: <bc0c60>; proxy; rspamd_redis_connected: skip obtaining bayes tokens for BAYES_HAM of classifier bayes: not enough learns 199; 200 required

Then started the “skip learning” messages:

Dec 15 01:02:06 nethservice rspamd[15843]: <ebbbe5>; proxy; rspamd_task_process: skip learning: <NethServer/nethserver-httpd/pull/64/push/4397709277@github.com> is skipped for bayes classifier: already in class ham; probability 100.00%

This is the rspamc stat output:

[root@here ~]# rspamc stat
Results for command: stat (0.034 seconds)
Messages scanned: 1457081
Messages with action reject: 421653, 28.93%
Messages with action soft reject: 3528, 0.24%
Messages with action rewrite subject: 16, 0.00%
Messages with action add header: 40105, 2.75%
Messages with action greylist: 6790, 0.46%
Messages with action no action: 984989, 67.60%
Messages treated as spam: 465302, 31.93%
Messages treated as ham: 991779, 68.06%
Messages learned: 25210
Connections count: 0
Control connections count: 12
Pools allocated: 18691
Pools freed: 18658
Bytes allocated: 21.87MiB
Memory chunks allocated: 128
Shared chunks allocated: 17
Chunks freed: 0
Oversized chunks: 1547
Fuzzy hashes in storage "local": 0
Fuzzy hashes in storage "rspamd.com": 905848192
Fuzzy hashes stored: 905848192
Statfile: BAYES_SPAM type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 3261; users: 39; languages: 0
Statfile: BAYES_HAM type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 11001; users: 1251; languages: 0
Total learns: 14262

The Total learns counter here is really big. And the one used by the Dashboard UI, Messages learned is even bigger:

curl --connect-timeout 2 'http://localhost:11334/auth'

{
  "version": "2.2",
  "auth": "ok",
  "uptime": 248605,
  "clean": 985008,
  "probable": 40121,
  "greylist": 6790,
  "reject": 421653,
  "soft_reject": 3528,
  "scanned": 1457100,
  "learned": 25210,
  "read_only": false,
  "config_id": "token"
}
2 Likes
45

I report upstream why we have all counters with different numbers

2 Likes
46

I tested also the behavior if ClamAV is not responding. All seems fine and ready for a production release!

Edit: update to rspamd 2.2 released.

1 Like
49

This topic was automatically closed after 5 days. New replies are no longer allowed.