Using Neth as a mail server for Discourse

Support
, ,
1

NethServer Version: 7.8
Module: mail server

tl;dr: I’d like to use my Neth installation as a mail server for my Discourse installation, and don’t want to miss anything.

I’ve been running a small Discourse installation for the last year or so, handling a group I moved from Yahoo before the great purge. Because most of the users had been using it as a mailing list (it goes back almost 20 years, originating on egroups.com, and was an outgrowth of an AOL listserv that predated that by 5 years or so), I set everyone to mailing list mode by default. That, of course, means it handles a fair bit of email traffic, even with as small a forum as it is. Mailgun isn’t all that expensive, all things considered, but if I’m already running a Neth server, it ought to be able to handle the mail, and maybe save me a few bucks.

My Neth server and the Discourse installation are on separate Contabo VPSs. Discourse is at one domain:


…and my Neth server is another. I’d like Discourse to be able to send mail through the Neth server, but handle any incoming mail itself–it’s already configured to do that.

So the question is, how to configure this? Obviously I need to create a user and give it a nice, long password, so I can use that user for sending mail. Configure Discourse to use mail.mydomain, that user and password, port 587. Easy enough.

Where I’m a little unclear is what else needs to be done:

  • Do I need to add spare-oom.com as a mail domain on my Neth server?
  • I assume I need to update the SPF record for spare-oom.com?
  • Do I need to do anything with DKIM?
  • Anything else?
2

For me I use this little tool to help me see where I have missed things https://www.mail-tester.com/

Depends on how you setup your NS install, but maybe.

Yes, this must match the settings for your setup. The above website will let you know what values it is seeing and what they expect it to be.

Yes. Adding DKIM and DMARC records are in my opinion a must in 2020 and easy/free to do,

Check your IP reputation to make sure you the address is not on a blacklist to prevent delivery of your outbound emails.

1 Like
3

Interesting. Sending mail is failing. The Neth log has:

Oct  2 16:27:57 neth postfix/smtpd[18542]: connect from spare-oom.com[144.91.75.139]
Oct  2 16:27:57 neth rspamd[11516]: <00eea5>; proxy; proxy_accept_socket: accepted milter connection from /var/run/rspamd/worker-proxy port 0
Oct  2 16:27:57 neth postfix/smtpd[18542]: lost connection after AUTH from spare-oom.com[144.91.75.139]
Oct  2 16:27:57 neth postfix/smtpd[18542]: disconnect from spare-oom.com[144.91.75.139]

But the discourse-doctor script is reporting this error, apparently received from my Neth box:

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Or just send a test message to yourself.
Email address for mail test? ('n' to skip) []: (redacted)@srv1.mail-tester.com
Sending mail to (redacted)@srv1.mail-tester.com. . . 
Testing sending to (redacted)@srv1.mail-tester.com using mail.familybrown.org:587.
======================================== ERROR ========================================
                                    UNEXPECTED ERROR

503 5.5.1 Error: authentication not enabled


====================================== SOLUTION =======================================
This is not a common error. No recommended solution exists!

Please report the exact error message above to https://meta.discourse.org/
(And a solution, if you find one!)
=======================================================================================
4

Salut Dan,

After a chat with Stéphane at the end of last year I spent quite some time writing an howto on NethServer & Discourse.

You can look at Chapter V - SparkPost.
https://www.micronator.org/affaires/produit/nethserver-201-cahier-10forum-discourse/

I do not remember eactly why or where I took that, but I started this chapter with:

“Since it is very difficult to get nethserver-mail-server to work with Discourse, it is highly recommended
(and this, even by the Discourse team) to use an external service to take care of the electronic messaging.”

Freely use SparkPost if you have lest than 30 mails/day.

- You supply the DKIM text, etc. and SparkPost take care of it all. (A few screen-shots in Chapter-V).
- There is also a “Set Up For Bounce”.

Michel-André

5

Email throughput for my forum is pretty sporadic–I sent 6500 messages over the past month, which would only average 22/day, but as many as 1400 in one day. It doesn’t send messages every day, but on days it does, it’s usually not less than 100.

I’m aware of the Discourse devs’ recommendation, but their stated reason for that is the complexity in installing, securing, and maintaining your own mail server–Neth handles all this anyway. Granted, Mailgun is only costing me about $5/mo, but if I could save that, it would be nice. I’m also thinking it might improve deliverability, as I’m getting a number of bounces that I think just result from Mailgun’s IP not being trusted.

6

did this recently https://wiki.nethserver.org/doku.php?id=discourse

maybe it can help

3 Likes
7

Ok… that triggers me a bit more than a little.
@danb35 you’re certainly more skilled and experienced than me, but in my little time on devices i learned that… i can easily find little imperfections.

You are talking about “public” hosts, would you like to share a bit more of your current setup for

  • NethServer as mailserver
  • Discourse as email client
    please?

Also… i only take a run on Discourse as a user, not as admin, therefore a lot of things are unknown to me. But…

Discourse talks in SMTP to the mail server (i suppose) so most of the job should be:

  • add the discourse domain to NethServer
  • create the DKIM key for the domain
  • add a new hostname to NethServer (with the same domain of Discourse installation)
  • add all the data to DNS panel of the domain (A Record, MX Record, SPF Record, DMARC record with the DKIM data of the new domain)
  • create a user for Discourse
  • configure Discourse to use a SmartHost

Or am I too much optimistic?

1 Like
8

Hum my module uses only the smtp to send email, not sure it is able to receive email

9

But in any case, unless the public ip of Discourse is added to the list of addresses allowed for relay, the application server must authentiticate to SMTP for sending messages…
Assuming that VPS may have multiple virtual hosts on the same IPv4 (or 6) address, i’d feel much more comfortable having credentials (and a strong password) instead of relay without authentication.
And with TLS connection too.

10

Yes, this exactly. And even more fundamentally, Discourse requires SMTP authentication anyway–it won’t work without a username and password to authenticate to.

It really should; Discourse needs to be able to deal with bounces, new topics by email, replies by email, etc. In my installation, Discourse handles that directly; no incoming mail to the Discourse installation should ever touch the Neth (or other remote mail server) box. But it’s also capable of polling a mailbox if needed.

Glad to, but I’m not sure what information would be relevant here. Neth has the mail server module installed, and has been acting as my private family mail server since I installed it. I haven’t done anything fancy to its configuration. In preparation for trying to use Neth for this, I’ve

  • Created a user called spareoom on my Neth server, and given that user a nice, long password
  • Added spare-oom.com as a mail domain on the Neth server

I think that’s all I should need to do on the Neth end. I haven’t made any changes to any of the authentication or security settings for email that I’m aware of.

Discourse, to the best of my knowledge, only supports sending email via SMTP (I mention this because many of the bulk mail hosts have APIs for message submission as well, but Discourse doesn’t use those)–it won’t use a local sendmail binary or its equivalent, it won’t send without authentication, it doesn’t act as its own SMTP host, etc. It’s designed to connect as an authenticated (and encrypted, by default) client to an SMTP host and submit messages that way. My installation is on a Ubuntu VPS following their instructions (discourse/docs/INSTALL-cloud.md at main · discourse/discourse · GitHub). Currently, it’s configured to use Mailgun, and that configuration consists of entering their mail server name, username, password, and port. TLS is enabled by default. This configuration has worked perfectly from Day 1–and on Day 1, Mailgun had a free plan for up to 10k messages/month, which handled everything I needed. That plan has now gone away; you’re now paying after about 1000 messages/month.

So, changing the Discourse configuration means changing four lines in app.yml to set the mail host, username, password, and port (and optionally set TLS, but it’s enabled by default), and then running ./launcher rebuild app. And all of that works fine, generating no errors. But actually sending the message doesn’t work, generating the error noted above.

11

Maybe is the dumbest question in the world but… did you configured an email client for use the “discourse account” of your nethserver for sending and receiving emails and checking that everything work as expected?
Are there any IPS rules on your NethServer installation? Fail2Ban?

12

It’s a pretty obvious troubleshooting step, but I hadn’t tried it. Now I have, and it works–I can send emails as that user with a return address of @talk.spare-oom.com. I can also receive emails there, though that would never happen in production.

I don’t have IPS installed.

It’s running, but the IP for spare-oom.com isn’t blocked. I can connect to the mail server from inside the Discourse container (run ./launcher enter app followed by telnet neth.mydomain 587 succeeds), so I think that rules out one set of issues.

13

Could you give us the yaml configuration lines relative to the email part please

14 
  DISCOURSE_SMTP_ADDRESS: mail.familybrown.org
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: postmaster@talk.spare-oom.com
  DISCOURSE_SMTP_PASSWORD: "redacted"
15

I put more settings to use the smtp of nethserver

16

The only additional setting you have is to use TLS–which defaults to true anyway.

17

It is a yaml file, the indentation is important, could you check it

18

Explicitly enabling TLS made no difference, still getting the same error. And yes, it’s indented properly–two spaces, no tabs.

Edit: I’m able to duplicate this error when I try authenticating manually via Telnet following these instructions:
https://www.ndchost.com/wiki/mail/test-smtp-auth-telnet

Not much of a surprise, as permitting authentication via Telnet would have allowed the credentials to travel unencrypted. Trying to do it via openssl s_client isn’t working because the base64 password starts with R, and that makes it renegotiate.

1 Like
19

We need maillog

20 
Oct  4 14:34:35 neth postfix/smtpd[21777]: connect from spare-oom.com[144.91.75.139]
Oct  4 14:34:35 neth rspamd[24203]: <0cb37d>; proxy; proxy_accept_socket: accepted milter connection from /var/run/rspamd/worker-proxy port 0
Oct  4 14:34:35 neth postfix/smtpd[21777]: lost connection after AUTH from spare-oom.com[144.91.75.139]
Oct  4 14:34:35 neth postfix/smtpd[21777]: disconnect from spare-oom.com[144.91.75.139]
Oct  4 14:34:35 neth rspamd[24203]: <0cb37d>; milter; rspamd_milter_process_command: got connection from 144.91.75.139:53674
Oct  4 14:34:35 neth rspamd[24203]: <0cb37d>; proxy; proxy_milter_finish_handler: finished milter connection