Matrix - synapse

This is a WIP for creating an install howt for Matrix - Synapse messaging server on NethServer 7.
A startingpoint could be: https://www.vultr.com/docs/create-a-chat-server-using-matrix-synapse-and-riot-on-centos-7
and the official install howto on github: https://github.com/matrix-org/synapse/blob/master/INSTALL.md
please feel free to add in this topic steps to install on NethServer.
This is a wikified post.

The goal is to have a Matrix - Synapse module for NethServer with the option to both have access with a local accountprovider (either LDAP or Samba4 AD) as internal messaging server AND the option to have conversations with other Matrix - Synapse servers through federation

ToDo

  • Voice/video chat with TURN server

Prerequisites

You need a valid letsencrypt cert to let the clients connect. You may use Matrix Synapse with your default domain but for avoiding virtualhost conflicts I recommend to use a separate (sub)domain like matrix.example.org

I tested with Riot client on Windows and Android. Encryption is working.

Installing needed packages and dev tools

yum -y install libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config python-virtualenv libffi-devel openssl-devel postgresql-devel
yum -y groupinstall "Development Tools"

Install postgres 10.0, create DB and user synapse

You may replace SECRET with a more secure password.

yum -y install rh-postgresql10
scl enable rh-postgresql10 bash
postgresql-setup --initdb --port=55433
exit
cat << EOF > /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf
# "local" is for Unix domain socket connections only
local   all         postgres                          ident
local   all         all                               md5
# IPv4 local connections:
host    all         all         127.0.0.1/32          md5
# IPv6 local connections:
host    all         all         ::1/128               md5
EOF
systemctl enable rh-postgresql10-postgresql --now
su - postgres -c "scl enable rh-postgresql10 -- psql --port=55433"
create user synapse with encrypted password 'SECRET';
CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse;
\q
Alternative install of postgres92 via nethserver-postgresql
yum -y install nethserver-postgresql postgresql-devel
su - postgres
createuser synapse
psql -c "alter user synapse with encrypted password 'SECRET';"
psql
CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse;
\q
exit

Install synapse

yum -y install rh-python36
mkdir -p /opt/synapse
scl enable rh-python36 bash
virtualenv -p python3.6 /opt/synapse
source /opt/synapse/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install matrix-synapse
pip install Jinja2
alternative install with python2.7
mkdir -p /opt/synapse
virtualenv -p python2.7 /opt/synapse
source /opt/synapse/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install matrix-synapse
pip install Jinja2

Install postgres client libs

pip install matrix-synapse[postgres]

Create config file

Replace example.org with the domain you want to use:

cd /opt/synapse
python -m synapse.app.homeserver \
    --server-name example.org \
    --config-path homeserver.yaml \
    --generate-config \
    --report-stats=no

Edit homeserver.yaml to use postgresql, search for database:, edit name and add/edit the args as needed:

database:
    name: psycopg2
    args:
        user: synapse
        password: SECRET
        database: synapse
# specify port because rh-postgresql10 doesn't run on default 5432 port
        host: localhost
        port: 55433
        cp_min: 5
        cp_max: 10

Configure httpd

We use a reverse proxy to make synapse accessible at https://example.org. As it has some special options it cannot be added via web UI. Don’t forget to replace example.org with the domain you use.

cat << EOF > /etc/httpd/conf.d/a_synapse.conf
<VirtualHost *:443>
        SSLEngine on
        ServerName example.org

        AllowEncodedSlashes NoDecode
        ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
        ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
</VirtualHost>

Listen 8448

<VirtualHost *:8448>
        SSLEngine on

        AllowEncodedSlashes NoDecode
        ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon
        ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix
</VirtualHost>
EOF

Restart httpd to apply the new config:

systemctl restart httpd

Open port for synapse federation

config set synapse service status enabled TCPPort 8448 access red,green
signal-event firewall-adjust

Create, enable and start systemd service

cat << EOF > /etc/systemd/system/matrix-synapse.service
[Unit]
Description=Matrix Synapse service
After=network.target

[Service]
Type=forking
WorkingDirectory=/opt/synapse/
ExecStart=/opt/synapse/bin/synctl start
ExecStop=/opt/synapse/bin/synctl stop
ExecReload=/opt/synapse/bin/synctl restart
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=synapse

[Install]
WantedBy=multi-user.target
EOF

systemctl enable matrix-synapse --now

Test synapse

Browse to https://example.org/_matrix/key/v2/server/auto and you should get JSON output like

{"old_verify_keys":{},"server_name":"example.org"...

Register local synapse users

To register a new (admin) user:

register_new_matrix_user -c homeserver.yaml http://localhost:8008

Now you are ready to connect to the server with a matrix synapse client like the Riot web app.

Riot web app

wget https://github.com/vector-im/riot-web/releases/download/v1.5.12/riot-v1.5.12.tar.gz
tar -xzf riot-v1.5.12.tar.gz
mv riot-v1.5.12 /var/www/html/riot
cp /var/www/html/riot/config.sample.json /var/www/html/riot/config.json
rm -f riot-v1.5.12.tar.gz

Edit /var/www/html/riot/config.json to match your domain:

    "default_server_config": {
        "m.homeserver": {
            "base_url": "https://example.org",
            "server_name": "example.org"
        },
        "m.identity_server": {
            "base_url": "https://matrix.org"

Browse to https://example.org/riot and login to your web chat app.

Federation test

If you’re able to connect to room #synapse:matrix.org, federation should work.
You may test with federation tester too: https://federationtester.matrix.org/

LDAP/AD

You may use LDAP/AD as userbase. This way both, local synapse users and LDAP/AD users, are possible.

For remote AD/LDAP you need to use LDAPS and port 636 and for AD maybe a cert, I didn’t test remote AD/LDAP…

Install the ldap module:

pip install matrix-synapse-ldap3

Uncomment the password providers settings in /opt/synapse/homeserver.yaml (search for password_providers:) and edit to your needs.

Replace BINDPASSWORD with your AD/LDAP bind user password.

Local LDAP

password_providers:
    - module: "ldap_auth_provider.LdapAuthProvider"
      config:
        enabled: true
        uri: "ldap://localhost:389"
        start_tls: true
        base: "ou=People,dc=directory,dc=nh"
        attributes:
           uid: "uid"
           mail: "email"
           name: "sn"
        bind_dn: "cn=ldapservice,dc=directory,dc=nh"
        bind_password: BINDPASSWORD

Local AD

password_providers:
    - module: "ldap_auth_provider.LdapAuthProvider"
      config:
        enabled: true
        uri: "ldap://nsdc-testserver.ad.domain.local"
        start_tls: true
        base: "dc=ad,dc=domain,dc=local"
        attributes:
           uid: "cn"
           mail: "email"
           name: "samaccountname"
        bind_dn: "ldapservice@ad.domain.local"
        bind_password: BINDPASSWORD

Apply config:

systemctl restart matrix-synapse

Bridges

matrix-appservice-discord - work in progress

Get nodejs 10 and matrix and compile:

cd ~
yum install rh-nodejs10
git clone https://github.com/Half-Shot/matrix-appservice-discord.git
cd matrix-appservice-discord
scl enable rh-nodejs10 bash
npm install
npm run build
cp config/config.sample.yaml ./config.yaml

Edit config.yaml to set your domain:

bridge:
    domain: "example.com"
    homeserverUrl: "https://example.com"

Generate discord-registration.yaml and copy to right place:

node build/src/discordas.js -r -u "http://localhost:9005" -c config.yaml
cp discord-registration.yaml /opt/synapse/

Apply discord bridge to synapse config:

Edit /opt/synapse/homeserver.yaml and add the

app_service_config_files:
  - "discord-registration.yaml"

Now I’m stuck at the docker run.

docker run -v /root/matrix-appservice-discord:/data halfshot/matrix-appservice-discord

I get following error. It seems the container does not get an IP address.

Mar-20 21:34:40.721 [ClientFactory] error: Could not login as the bot user. This is bad! Error: getaddrinfo EAI_AGAIN discordapp.com
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:64:26)

mautrix-telegram - work in progress

This is an example for a simple matrix bridge.

yum -y install http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm
yum -y install ffmpeg
su - postgres -c "scl enable rh-postgresql10 -- psql --port=55433"
create user mautrixtelegram with encrypted password 'SECRET';
CREATE DATABASE mautrixtelegram
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER mautrixtelegram;
\q
mkdir /opt/mautrix-telegram
cd /opt/mautrix-telegram
scl enable rh-python36 bash
virtualenv .
source ./bin/activate
pip install --upgrade mautrix-telegram[all]
cp example-config.yaml config.yaml

Create a telegram bot at https://core.telegram.org/bots to get api id and hash.

Open config.yaml and configure homeserver settings, api_id/api_hash, look for example.org and replace with your domain.

Add postgres settings:

database: postgres://mautrixtelegram:SECRET@localhost:55433/mautrixtelegram

Create registration file with following command:

python -m mautrix_telegram -g

Edit ../synapse/homeserver.yaml (The matrix homeserver config):

app_service_config_files:
  - "discord-registration.yaml"
  - "/opt/synapse-telegram/registration.yaml"

Restart matrix-synapse and start mautrix_telegram

systemctl restart matrix-synapse
python -m mautrix_telegram

Go to your Telegram and talk to your bot and check the output on the screen.

Clean routine

Delete Matrix Synapse to make reinstall possible if something failed badly:

systemctl stop matrix-synapse
rm -Rf /opt/synapse
su - postgres -c "scl enable rh-postgresql10 -- psql --port=55433"
drop database synapse;
drop role synapse;
\q
4 Likes

A big THANK YOU @mrmarkuz for the first steps of this howto.
Would it be possible to add a reverse proxy config so LE certs can be used behind a router on a private subnet?
oops… you already have added that
@mrmarkuz: did you also see the warning that puthon 2.7 will be deprecated:

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support

1 Like

Yes, it’s already on the todo list, I’ll give it a try with python3 asap. Postgres 9.5 is recommended and we use 9.2. but it’s working so far.

i think ill wait for the module…

tried following the instructions multiple times. i cant get past

======
Job for matrix-synapse.service failed because the control process exited with error code. S ee "systemctl status matrix-synapse.service" and "journalctl -xe" for details.

========
Not sure where am going wrong.

looking forward to testing that.

EDIT:
i was able to resolve that one.
issue was

had not been defined properly… thanks for the clean script

You may run synctl start to start synapse manually and check the output for errors.
Or check with systemctl status matrix-synapse -l

I put all lines that need to be performed in terminal between [code] tags. Am I right this will avoid parsing problems?

so far everything has worked, to the point i can login to riot.

But, the testing of synapse returns 404.
while login to riot on local says cannot reach homeserver.

what could be the error?

Is matrix synapse running?
systemctl status matrix-synapse -l
Is rh-postgresql10-postgresql running?
There are log files in /opt/synapse, please check them for errors…
If you copy/paste to terminal be careful that every command runs because systemctl commands cut off following commands.

looks like all is ok…
but still not able to have online test working. that weired

Hm I think I forgot the firewall settings.

Please try:

config set synapse service status enabled TCPPort 8448 access green,red
signal-event firewall-adjust

i have run the command. still getting same behavior

Thanks for testing, the problem was the httpd conf file order, I’ll update the howto…

2 Likes

what is identity provider, i saw a notification about it not having bein genabled.

Please try to change the identity server in riot config /var/www/html/riot/config.json to matrix.org instead of your domain and check if the notification goes away. I assumed that our server is an identity server too but maybe I was wrong…

"m.identity_server": {
            "base_url": "https://matrix.org"
1 Like

now it works great,
i want to test ldap and users created on nethserver if they will be able to automatically login.
then i test if the video and voice call functions work.

2 Likes

@robb I don’t know if parsing is really easier with code tags. Most of the lines are not source code (bash commands/config files) so the syntax highlighting does not have the intended effect.

I had some parsing issues, mostly because of extra spaces in the txt formatting

I am running into a small issue: On a subscription server the repo where Development Tools reside is not configured. What repo do I need to add?

Hm, it’s a default centos yum group of packages.

What do you get?

yum group list

yum groupinfo "Development Tools"

Maybe this works:

yum --setopt=group_package_types=mandatory,default,optional groupinstall "Development Tools"

Source:

It’s a subscription server. Only nethserver repo’s are listed
@davidep @giacomo any suggestions?

Loaded plugins: changelog, fastestmirror, nethserver_events
Loading mirror speeds from cached hostfile
 * sb-base: u2.nethserver.com
 * sb-centos-sclo-rh: u2.nethserver.com
 * sb-centos-sclo-sclo: u2.nethserver.com
 * sb-epel: u2.nethserver.com
 * sb-extras: u2.nethserver.com
 * sb-nethserver-base: u2.nethserver.com
 * sb-nethserver-updates: u2.nethserver.com
 * sb-updates: u2.nethserver.com
Installed Groups:
   Backup
   Backup restore
   Basic firewall
   Dutch language
   Email
   FTP server
   Fail2ban
   Instant messaging
   MariaDB (MySQL) server
   Mattermost
   Netdata
   NethServer subscription
   New Server Manager (Beta)
   Nextcloud
   OpenVPN
   Report (Beta)
   SMTP proxy
   Statistics
   Web hosting
Available Groups:
   Bandwidth monitor
   Dedalo Hotspot
   Deep packet inspection (DPI)
   Fax server
   File server
   French language
   German language
   Greek language
   Hungarian language
   IPsec tunnels
   Intrusion Prevention System
   Italian language
   POP3 connector
   POP3 proxy
   Portuguese language
   Print server
   Reverse proxy
   Romanian language
   Roundcube web mail
   Russian language
   SNMP server
   Serbian language
   Spanish language
   Turkish language
   UPS support
   VoIP PBX
   Web filter
   Web proxy
   WebTop 5 groupware
Done
yum groupinfo "Development Tools"
Loaded plugins: changelog, fastestmirror, nethserver_events
Loading mirror speeds from cached hostfile
 * sb-base: u2.nethserver.com
 * sb-centos-sclo-rh: u2.nethserver.com
 * sb-centos-sclo-sclo: u2.nethserver.com
 * sb-epel: u2.nethserver.com
 * sb-extras: u2.nethserver.com
 * sb-nethserver-base: u2.nethserver.com
 * sb-nethserver-updates: u2.nethserver.com
 * sb-updates: u2.nethserver.com
Warning: group/environment Development Tools does not exist.

has anyone managed to get ldap local users to be able to login into the app?