How to automatically configure Thunderbird and Outlook email with Nethserver

[danb35]

Content moved to the wiki.

[dnutan]

I’ve been reading up a bit on how to automate, or at least simplify, configuration of email clients to connect to Nethserver. Unfortunately, there doesn’t appear to be any one standard–Thunderbird, Outlook, and iOS/Mac OSX Mail all use different methods to accomplish this.

Great! There were some community members interested in having this for their email client of choice (like No autodiscover, MailServer Configuration (autodiscover, SPF, DKIM)).
Having a clear and simple howto for Thunderbird is a good starting point, as there are many Thunderbird users here.

[danb35]

I’ve added information that should work for Outlook, though not having a copy of MS Outlook on hand, I can’t test it myself. But it looks like you gave links to a number of useful tools in the “no autodiscover” thread; might need to check those out.

iOS is going to be the tricky one. To do it right, you need a .mobileconfig profile which should be signed by a trusted TLS certificate. There’s a tool to create these here, but it seems to depend on a number of packages that aren’t installed in Neth by default, and it would need to run as root in order to read the server’s private key.

Edit: Oh, automx looks like it could be very handy indeed.

Edit 2: Using Microsoft’s connectivity analyzer, I checked my instructions for Outlook, and had to make a small change (the Autodiscover directory, and Autodiscover.xml filename, both need to be capitalized), but with that change, it’s working. Though I still think I may want to play with automx, as it would be a one-stop solution if it works as advertised.

[danb35]

Well, I’ve played with it a little bit, and automx really looks like the better tool to use here–static files like I’ve shown above don’t require any software installation, and only minimal configuration, but they don’t do the job for iOS, and they aren’t very flexible.

OTOH, automx only needs a few packages to be installed (mod_wsgi, python-ipaddr, and m2crypto, all of which are in the standard repos), takes a single config file, and can pull user information out of LDAP (so the user only needs to give email address and password–though I haven’t tested this feature on Neth yet). It will create configs for Thunderbird, Outlook, and iOS, and will sign the configs for iOS allowing them to be imported without warnings. It takes a few tweaks to the virtual host configuration, but nothing too bad.

It ships with a simple little HTML form that will take email address, full name, and password, and generate an iOS .mobileconfig profile. The ideal use of this would probably be to integrate a similar form into the server manager and have it accessible to all users (like the change password panel).

/etc/automx.conf needs to be templated, but that shouldn’t be too tough of a chore. That would cover server names, LDAP configuration, organization name (to attach to the account), TLS cert/key paths, etc. And really, the whole thing probably should be packaged into an RPM.

Problem is that, in order to sign the iOS configuration, the script needs to be able to read the server’s private key, which is ordinarily only readable by root. Since it isn’t a daemon, (I assume) it can’t do the trick that Apache itself does to start up as root, read the key, then shed privileges. So either the script needs to run as root, or the private key needs to be readable by apache–neither sounds like an attractive proposition. I guess a third option would be to generate a separate cert just for the config signing–straightforward enough if using Let’s Encrypt, though rate limits can be a concern.

Edit: It does look like the last “official” release of automx, 0.10.2, is nearly four years old, while there’s a v1.1.2 that hasn’t been tagged as an official release on github but seems to make some significant changes.

[mrmarkuz]

Great work, great howto!
Client autoconfig support for TB, OL and iOS is a killer feature!

And really, the whole thing probably should be packaged into an RPM.

It’s on my list.

[Stefano_Zamboni]

About automx, ask @stephdl, he already has something workable

[stephdl]

yes @mrmarkuz, stefano did some work that you can browse at http://mirror.de-labrusse.fr/NethDev/

I have not much time now on this but if you can do it, I’m fine

[danb35]

you can browse at http://mirror.de-labrusse.fr/NethDev/

That’s blocked at work as a porn site (stupid internet filtering), so I’ll have to take a look when I get home. Looking at the release notes for automx, it sounds like they’ve dropped the need for m2crypto, so that’s one less package that would need to be added. OTOH, we’d also need to add python-ldap to be able to pull user information from there, which I’d think we’d want to do.

How are you planning to handle access to the server private key, or have you gotten that far yet?

Edit: An implementation detail I’d suggest is to keep the DNS SRV record I mention above, rather than to use separate autoconfig and autodiscover hosts as recommended by the (less-than-fantastic) Automx docs. It does put one more step on the user, but it’s going to simplify the server-side configuration (one VirtualHost can handle everything), and have one less hostname needed on the TLS cert.

[alefattorini]

Great job so far Dan! Many members will benefit of such work.
I love that

[danb35]

you can browse at http://mirror.de-labrusse.fr/NethDev/

Well, my work computer can’t get it, but my tablet can VPN to my home network, and… Just a few things I’m noticing on a quick look:

• The filename (nethserver-automx-0.0.1.tar.gz) indicates that it’s gzipped, but it isn’t. Slight issue in extracting the file.
• I see that this will depend on automx and automx-web. Will you be building separate packages for those, or do those packages already exist somewhere I don’t know about?
• Even though it raises the issues of script access to the server’s private key that I’ve mentioned above, it really should enable signing of iOS .mobileconfig files–unsigned profiles are not worthless, but are much less useful than signed ones.
• Looks like the configuration will be calling for IMAPS/993 rather than IMAP/STARTTLS/143–based on the manual, it sounds like the latter would be preferable.

[stephdl]

@Stefano_Zamboni can advice you, I’m not the author. In tar.gz you have all the mandatory files to build the rpm, I just verified it

[Stefano_Zamboni]

you can use the src.rpm for automx you’ll find here:
https://build.opensuse.org/package/show/home:wrosenauer:devel/automx

[danb35]

I’m circling back to playing with automx, having put the initial write-up from this thread into the wiki. Getting it to respond appropriately to config queries for Thunderbird/Outlook/Apple is straightforward enough. Even building my own web form to work with it is simple enough, if I don’t care too much about validating the input (so my users can get the .mobileconfig file from my own page, rather than the automx-branded page). I haven’t yet figured out how to make that form validate the password before generating the .mobileconfig, but that’s a problem for a later time.

The part I’m scratching my head about is pulling the users’ full names out of LDAP. No doubt my near-complete ignorance of both Perl and LDAP is contributing to this problem, but the templates from the very pre-alpha nethserver-automx RPM will pull the LDAP connection parameters from the system configuration, and when I turn on debugging, it appears that automx is in fact connecting (it doesn’t say that it is, but it doesn’t give an error saying it isn’t).

The automx.conf manpage makes it sound like I should be able to set display_name = ${displayName}, and automx will pull in the Display Name out of LDAP. But when I do that, the account name just shows the literal ${displayName}. A thread from Univention makes it look like it should be escaped: display_name = \${displayName}. But when I do that, the result is the literal \${displayName}. Still working on this…

Edit: A little playing around with ldapsearch makes it look like the appropriate field would be cn rather than displayName:

[root@neth-automx ~]# ldapsearch -xLLL -H ldaps://localhost -b 'dc=directory,dc=nh' '(uid=ginger)'
dn: uid=ginger,ou=People,dc=directory,dc=nh
uidNumber: 1002
gidNumber: 1000
uid: ginger
shadowMax: 99999
shadowWarning: 7
shadowMin: 0
loginShell: /usr/libexec/openssh/sftp-server
homeDirectory: /var/lib/nethserver/home/ginger
shadowInactive: -1
shadowExpire: -1
gecos: Ginger Rogers
shadowLastChange: 17677
shadowFlag: -1
cn: Ginger Rogers
sn: ginger
objectClass: posixAccount
objectClass: shadowAccount
objectClass: inetOrgPerson
mail: ginger@familybrown.org

…but using ${cn} or \${cn} in automx.conf still has the same result: the name ends up as that literal string, rather than the contents of that field.

[stephdl]

The perl api to browse ldap and samba4 is not so hard you could take some example with phpladadmin and sogo.

[danb35]

OK, so I’ve done some more tinkering. I’m giving up (for the time being) on pulling the user’s full name out of the LDAP server, though something tells me I’m missing something pretty obvious there. It would be ideal, of course, for it to work both with OpenLDAP and AD, but it may take someone smarter than I to figure this out.

I’ve built an RPM for automx using the current code on GitHub–the 0.10.2 version used for the OpenSuSe RPM is now four years old, and there have been some major changes since then, though the project now appears to be all but abandoned. I started with their .spec file, cleaned it up a bit (mainly to remove the multi-distro logic), and adjusted some dependencies.

I’ve also built a nethserver-automx RPM based on @Stefano_Zamboni’s work, with a couple of updates to the templates and dependencies. The templates seem to be doing what I want them to do, and the script seems to be working as intended. Here’s what’s tested and working:

• The script responds properly to Thunderbird-style autoconfiguration requests, with properly-formatted XML consistent with the manual.
• The script generates .mobileconfig files for Apple clients, with the same configuration.
• When the configuration properties are set correctly, and the script is given a certificate and private key, it will generate signed .mobileconfig files, which are much more useful (fewer warnings to click through on installation).

Not tested, but assumed working:

• Responding properly to Outlook-style autoconfiguration requests. Since this uses the same script and configuration file, it should work, but I haven’t tested it yet.

Not working, needs fixing:

• The nethserver-automx RPM adds a virtualhost.conf template fragment and expands that template, but doesn’t reload httpd. Trying to reload httpd puts it in a weird state where the error log is showing repeated segfaults. After restarting httpd, it works fine.
• The nethserver-automx-update event doesn’t expand the automx.conf template.

Signing .mobileconfig files remains an issue of key control. Ordinarily the private key file is only readable by root, and Apache doesn’t run as root. This means the automx script can’t read the private key, and therefore can’t sign anything. There are a few ways I can think of to deal with this:

• Run Apache as root. Generally discouraged for security reasons.
• Make the server’s private key readable by Apache. Also generally discouraged for security reasons.
• Make the automx script setuid root. Again, discouraged for security reasons. Any of these three options raise the possibility of leaking private key data for your main server certificate.
• Create a separate certificate for just autoconfig.yourdomain, and make that readable by the automx script. There’s still the possibility of a key compromise if there’s a vulnerability in that script, but that key would do less damage if compromised.

My RPM would work with options 2 and 4 (since there are config properties for the cert and key paths, you can set them to whatever you want), but I think option 4 is the least bad.

So how can I get these RPMs somewhere where people can bang on them for a bit?

[Stefano_Zamboni]

OK, so I’ve done some more tinkering. I’m giving up (for the time being) on pulling the user’s full name out of the LDAP server, though something tells me I’m missing something pretty obvious there. It would be ideal, of course, for it to work both with OpenLDAP and AD, but it may take someone smarter than I to figure this out.

I tried to make it work too, and I gave up

the main issue I found is that debugging the code is a PITA and something is clearly missing in their code.
I searched for working configuration examples, tested using little python scripts that query ldap (successfully), but there was nothing to do: ldap interface is not working

[danb35]

I tried to make it work too, and I gave up

The frustrating thing is that Univention has a sample config that (they claim) works on their system. But when I’m trying it, not only do I not get the information I expect to see, there’s just no variable substitution going on at all–I put ${displayName} into the config file, I get ${displayName} in my output. And both Python and LDAP are pretty much opaque to me, so beyond confirming that it isn’t working, I’m pretty well out of my depth.

This should work, and it should be pretty straightforward. OTOH, it certainly isn’t critical, I lack the skills to figure out why it isn’t working, and automx seems like it’s 99% abandonware (so there’s no support from its developers). So I’m dropping that and working on what I might be able to understand instead.

[Stefano_Zamboni]

substitution going on at all–I put ${displayName} into the config file, I get ${displayName} in my

did you try with a templated config file or a plain one?

automx seems like it’s 99% abandonware (so there’s no support from its developers). So I’m dropping that and working on what I might be able to understand instead.

well, automx is just a web app that outputs an xml file fetching info from Ldap (optionally)

[Stefano_Zamboni]

replying to mysqlf:

Build software better, together

[danb35]

did you try with a templated config file or a plain one?

With a plain one.

well, automx is just a web app that outputs an xml file fetching info from Ldap (optionally)

The only part that makes it interesting to me, really, is its ability to generate (and sign) .mobileconfig files–the rest is easy to do with static files as I’ve shown here. But the LDAP connection would be a bonus if we could get it to work (it definitely isn’t the end of the world if users have to type in their own names, but it’s kind of frustrating to not be able to make it work when it seems like it should be such a simple thing).