Piler Mailarchiv

stephdl · January 29, 2021, 1:35pm

NethServer is a clone of centos7 however with an engine under the hood to write the configuration files by templates. I did a tour to install it at Piler Mailarchiv

I think the major issue I have was sphinx on centos7, it seems that we should install sphinx3 and there is no rpm.

Docker could be another way if it safe enough to run in production

jsuto · January 29, 2021, 2:11pm

You are right, unfortunately sphinx 3 stopped producing rpm packages. However, they provide a binary tar.gz that runs on centos 7 (http://sphinxsearch.com/files/sphinx-3.3.1-b72d67b-linux-amd64.tar.gz), or I created a somewhat smaller tar.gz with only the sphinx binaries, eg. indexer, searchd (https://download.mailpiler.com/generic-local/sphinx-3.3.1-bin.tar.gz) good enough to use with piler.

stephdl · January 30, 2021, 9:00am

Looking for docker installation I can find a ressource about the enterprise edition, but not much relative to the community
https://mailpiler.com/running-piler-enterprise-in-a-containerized-environment-with-docker-compose/

do you have the same documentation for the community ?

I saw that your development resources are on bitbucket : https://bitbucket.org/jsuto/piler/

EDIT: I found a related issue: https://bitbucket.org/jsuto/piler/issues/875/create-a-docker-version

stephdl · January 30, 2021, 9:46am

Thank to @davidep to make nethserver-docker possible and thank to @mrmarkuz to maintain it …this is really fun

Here a first attempt with docker-compose …it works, of course two billion things to do

[root@ns7loc7 docker]# docker ps
CONTAINER ID   IMAGE                COMMAND                  CREATED          STATUS                      PORTS                                                          NAMES
44b1b4d4af69   sutoj/piler:1.3.10   "/start.sh"              20 minutes ago   Up 19 minutes (unhealthy)   0.0.0.0:25->25/tcp, 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   docker_piler_1
955d0baeef50   mariadb:10.4         "docker-entrypoint.s…"   20 minutes ago   Up 20 minutes               3306/tcp                                                       docker_mysql_1
d1bb0263baa0   memcached:latest     "docker-entrypoint.s…"   20 minutes ago   Up 20 minutes               11211/tcp                                                      docker_memcached_1

Out topic @mrmarkuz but we should think to bundle docker compose…or let maybe the developer bundles it if the version is not accordingly the good wanted one.

obviously we have concurrency between apache and postfix who claim to use 80,443,25

stephdl · January 30, 2021, 9:58am

Will check it a bit more @jsuto thank for your work and to visit us !!!

stephdl · January 30, 2021, 12:05pm

does it is to make some income I saw a redirection to https://yoursite.com/ @jsuto ?

jsuto · January 30, 2021, 2:12pm

You may fix docker-compose.yaml to use other ports or bind addresses, eg. 127.0.0.1:80 or 1.2.3.4:8080 as long as you keep the ports inside the containers.

Also if you have memcached or mariadb installed on nethserver already, piler can use them allowing to simplify the docker-compose file.

stephdl · January 30, 2021, 2:16pm

ok we use a mysql mariadb 5.5, indeed I could use them, I am more concerned by the port 25 we have a workable solution of email server an obviously we use it…I am playing a bit with pilerimport but I read your concern about it, and It doesn’t work well I think with my tls policy, or I need to play a bit with it. However for a matter of test I used it without encryption and it is nice to see the email archived.

jsuto · January 30, 2021, 2:39pm

Just as nethserver (and many more projects) tries to monetize the product, the same is true for piler.
It comes in two flavors: an open source and a commercial edition.

The two sites are connected, however, a link to the commercial edition’s site alone doesn’t produce revenue. I regard its purpose is to raise awareness. I hope, it’s still ok.

jsuto · January 30, 2021, 2:41pm

Try using 127.0.0.1:2525:25 port scheme for piler’s mail server.

stephdl · January 30, 2021, 6:32pm

I know that free software is not enough clear in english, French are better for that

Programme libre != Programme gratuit

We have two different and clear statements on it.

I am a full advocate of sustainable development I have no problem on how a project can raise some incomes.

stephdl · January 31, 2021, 6:29pm

Well tested something

Install piler docker on aqua with docker-compose
bcc all email we receive to archive@archive.yourdomain.com
create a host esmith prop to make archive.yourdomain.com to aqua internal IP here 172.0.0.4
create a https reverse proxy to 172.0.0.4 for piler.example.com

I can see the email listed in the dashboard of piler, either sent by our postfix, or manually imported by piler-import

However I cannot retrieve them in the search function, it seems that the indexer has no errors

To be continued

mrmarkuz · January 31, 2021, 10:30pm

Good idea. It seems it’s used in many projects, see Peertube for instance.
For nethserver-docker we could download the latest docker compose with the update event when a status property is enabled like we do with portainer.
In our dev projects we could point to a specific bundled docker-compose version if needed.

jsuto · February 1, 2021, 10:03am

I also did some research deploying piler inside docker to nethserver. I created two DNS entries:
neth.acts.hu is for nethserver itself, and neth-archive.acts.hu a dedicated vhost for piler gui.

docker run -d
–name piler
-e MYSQL_ROOT_PASSWORD=xxxxxx
-e MYSQL_PILER_PASSWORD=abcde123
-e MYSQL_HOSTNAME=neth.acts.hu
-e PILER_HOSTNAME=neth-archive.acts.hu
-p 127.0.0.1:2525:25
-p 127.0.0.1:8080:80
-v etc-piler:/etc/piler
-v var-piler:/var/piler
sutoj/piler:1.3.10

It works almost just fine, I’ve got one final issue. I’ve decided to use a local ldap to provide user accounts both for emails and for piler. The idea is to let piler gui authenticate users against the common ldap database. However, it seems that eg. ldapsearch authenticates fine when it connects to 127.0.0.1:389, but it fails when you connect to the host IP, eg. 10.1.2.3:389. Unfortunately you can’t connect to 127.0.0.1:389 from inside the docker container.

A not that elegant workaround might be to authenticate against the imap server, but in that case you can’t have multiple email aliases, mailing / distribution list membership assigned to the users. At least not easily.

Can you suggest how to make the ldap authentication from a docker container possible?

Btw. I’m curious if you have any stats on how many percents of the nethserver deployments use local ldap vs. remote ldap or AD?

pike · February 1, 2021, 11:04am

Well… Piler should rely on SSSD, which can be connected to local LDAP, remote LDAP, AD.

mrmarkuz · February 1, 2021, 11:34am

You may add the needed container to the aqua network and setup a firewall rule to reach the host ldap server, see Nethserver Docker documentation.

stephdl · February 1, 2021, 12:15pm

@giacomo could you share us the percentage of ldap vs ad account provider, even remote vs local if you can have

If you look the documentation we allow 3306 from aqua to the firewall, It is the same I think for 389

I bet @jsuto for a remote authentication to ldap you need to enable starttls on 389

giacomo · February 1, 2021, 1:03pm

On Enterprise installations, 21% AD 79% OpenLDAP (including both local and remote)

stephdl · February 1, 2021, 4:52pm

Firstable I would like to propose my help for this work, ask me what you need, or I could even do it for you if you give me the configuration file.

In nethserver you have an helper try

account-provider-test dump

It will print all you need for an authentication, if the bind is local or remote, samba AD or OPENLDAP.

This is an example in perl how to authenticate user in dokuwiki, no matter it is samba or ldap, we detect and write the template accordingly

github.com

stephdl/nethserver-dokuwiki/blob/ns7/root/etc/e-smith/templates/etc/dokuwiki/local.protected.php/10auth

{

use esmith::util;
use NethServer::SSSD;
my $sssd = new NethServer::SSSD();

my $quotedBindPass = $sssd->bindPassword();
$quotedBindPass =~ s/\'/\\'/g;

#specific OpenLDAP
my $server = $sssd->ldapURI() .':'. $sssd->port();
my $usertree = $sssd->userDN();
my $grouptree = $sssd->groupDN();
my $binddn = $sssd->bindDN();
my $startTls = $sssd->startTls() || '0';

#specific Samba4 AD
my $host = $sssd->ldapURI();
my $baseDN = $sssd->baseDN();
$tls = $sssd->startTls();

This file has been truncated. show original

The idea is to do the same in the volume of docker, I assume it is possible in /var/lib/docker/volumes/docker_piler_etc/* we could create a template to write the configuration and restart the container once it is done (or restart only piler if it is possible)

jsuto · February 1, 2021, 8:05pm

I tried with the aqua network, I can reach mysql just fine. I even ran

db dockrules set pilerldap aqua TCPPorts 389,636 status enabled
signal-event firewall-adjust

However, the ldap server still doesn’t like me. I think the problem may be elsewhere. I believe that you must connect to 127.0.0.1:389 otherwise you can’t authenticate even with a correct password.

ldapsearch -x -h 127.0.0.1 -D “uid=piler,ou=People,dc=directory,dc=nh” -b “ou=People,dc=directory,dc=nh” -w piler123 “(&(objectclass=posixAccount)(uid=piler))” mail
piler, People, directory.nh
dn: uid=piler,ou=People,dc=directory,dc=nh
mail: piler@acts.hu

search result

search: 2
result: 0 Success

numResponses: 2

numEntries: 1

ldapsearch -x -h 172.28.0.1 -D “uid=piler,ou=People,dc=directory,dc=nh” -b “ou=People,dc=directory,dc=nh” -w piler123 “(&(objectclass=posixAccount)(uid=piler))” mail
ldap_bind: Invalid credentials (49)

Notice that the only difference is the ldap host IP-address I try to connect.

account-provider-test dump shows that it’s 127.0.0.1 as well:

{
“BindDN” : “cn=ldapservice,dc=directory,dc=nh”,
“LdapURI” : “ldap://127.0.0.1”,
“DiscoverDcType” : “dns”,
“StartTls” : “”,
“port” : 389,
“host” : “127.0.0.1”,
…
}

So I think the solution is to fix the ldap server to accept login attempts on any IP-address. How can we do that?