Yes for sure we want to offer something secure by design, but for now the focus is to create the tool and also the next needed when you use docker : a reverse proxy that I hope it will be traefik
As I said we need inputs of all sysadmin or skilled people, the matter to write the code of a good howto is easy and quick.
My green network is 192.168.122.0/24 (libvirt NAT). I’m looking at
[root@vm5 ~]# config show docker
docker=service
TCPPort=9000
access=green
status=enabled
And in nethserver-portainer_container action
/usr/bin/docker run -d -p 9000:9000 ... portainer/portainer
Also the systemd drop-in /etc/systemd/system/docker.service.d/nethserver.conf
ExecStart=/usr/bin/dockerd --ip 192.168.122.5
…well I want to propose an alternative approach for iptables and port forwards: there can be another solution to access portainer web UI, let’s discuss it!
pass --iptables=false to dockerd: we push iptables rules explicitly, by defining them in templates and from firewall UI (we can define a custom firewall zone in the future). We can also disable shorewall flag for Docker chains with this configuration.
assign a static, well known (from prop?) IP to portainer, like 172.17.0.2 (and expand it to /etc/hosts as “portainer”)
configure an Apache ProxyPass rule on httpd-admin instance, like we did for rspamd (…and use “portainer” host)
access portainer at https://IP:980/portainer
no need to configure https on portainer host: we already have httpd-admin in front of it
With this approach docker cannot allocate IP addresses for us: I need to reason about this a bit…
In this case, when you create a container, then you might need a human intervention to create the firewall stuff with a panel in nethgui…not sure it is nice
Docker runs on all available interfaces of the server, it means 0.0.0.0. Of course I do not want that all containers are reachable by their TCP ports from everywhere, I want to decide on what interface/zone docker launch its containers.
For facility/efficiency we launch dockerd with --ip xxx.xxx.xxx.xxx however to change the ip we use the network service panel, but sometime we have a red (false) nic and it can mislead the sysadmin…hence if we still go in this direction we need a tiny panel.
Another thing that probably needs an UI in the future is the storage configuration. The default devicemapper configuration is not recommended in production, instead a block device has to be allocated for the direct-lvm mode (see Use the Device Mapper storage driver (deprecated) | Docker Docs).
About --ip I’m studying the Docker networking a bit to see if we have an alternative…
My proposal is to disable the default Docker behavior that mangles iptables configuration, and requires a special Shorewall configuration to work properly in NethServer. Instead, all the Firewall plumbing happens with esmith templates and DB values.
Furthermore the prototype above defines a dedicated docker network, associated to a firewall zone: aqua. The first impression from the sysadmin point of view is to act with aqua like a green zone with some hosts in it. Please read the README.rst file for more info.
Another successful experiment is using a dedicated block device, as recommended by Docker official docs.
Now I have to test if traefik is really useful for us. It can auto-configure the reverse-proxy routes by reading them from containers metadata, which is really cool. Furthermore it is a real reverse-proxy, for large sites, with load balancing, health checks and automatic failover… But I can also drop traefik and configure Apache as reverse proxy too
Special thanks to @giacomo for helping with Shorewall and @stephdl for his starting point!
traefik is a huge software, I thought to get two ways to configure it, by the [docker] configuration (by labels) like you spoke, but also by the [file] configuration for vhost we have in apache (we have to change the tcp 80 and 443 port of apache if we want to start traefik)
Bear in mind that you have in the portainer interface, an url link to use the port of the container, it could be nice to get it workable
I mean about this FAQ — Portainer 1.23.2 documentation
Yes, the “label” way works and is really straightforward Just add a label from Portainer UI (Docker --label option from CLI) to instruct Traefik properly
The prototype above relies on firewall port forwarding rules for port 80 and 443 to override the main Apache instance and direct IP packets to Traefik. Surely every application that runs a real backend service, like WebTop, Mattermost, Nextcloud can push their rules into [file] to bypass Apache and optimize HTTP(S) traffic hops.
My impression is we do not need the “Published Ports” link in real cases (see image below). The link works only for HTTP, if the container exposes a web server. But web traffic is already handled by a reverse proxy (like Traefik). If the container exposes some other service (for instance a database server) the link is useless!
What did we achieve so far and where to go? Just some thoughts…
Docker integration with Shorewall. Defining the aqua network leads to a situation that can be easily handled with existing Firewall interface, from Server Manager. We need a wizard procedure that creates the firewall objects automatically.
Docker dedicated block device storage: requires an UI to select it (wizard procedure)
Portainer is a nice web UI, tailored on the docker CLI. I think most docker options are trasposed to the UI, so it is really powerful but quite complex. One must know well Docker to operate Portainer. BUT there’s the “Application templates” feature that is really promising for me!
It is like our Software Center page: the good news are that we can customize it, designing app templates specific to the NethServer environment. For instance we can define a “Redmine” template that connects MariaDB instance on NethServer host when it is started. Portainer templates can be instructed to ask for additional parameters interactively, for instance the virtual host name (see the MS SQL example). We can instruct Portainer to read the App catalog from the local NethServer instance, so it can be generated by a template with local parameters.
Traefik is a real reverse proxy. It’s flexible, it has a configuration file and also an HTTP API for configuration. It can auto-configure itself by reading the container labels (set by a Portainer app-template) from Docker: it’s perfect for it. However, its web UI is read-only, so we are still lacking a complete UI to configure it as reverse proxy for services running in another (LAN) host.
The aqua Shorewall zone is defined and created automatically
Portainer is instantiated and configured automatically
A dedicated storage device can be attached and configured before the docker daemon is started for the first time
traefik has been left behind, as we have the “Reverse proxy” page for that. This is the biggest design change since my post here: What about Docker on NethServer 7?
Just for our experiments, I configured mysql port 3306 open from aqua
Now what I can’t still grasp is the container upgrade and backup/restore lifecycles. I need to do some tests and study the latest Docker features to get an idea for them… ideas are welcome!
I tried caddy and joomla but I always get to the Nethserver default page instead of the portainer apps when browsing to http://nethserver:portainerport.
Dedicated storage seems to work (docker volumes are created).
The portainer app button is missing.
After installation it worked but when I did signal-event nethserver-docker-update more often, it happened that portainer was not reachable.
It can be a symptom of a lack of connectivity. I ran this experiment:
configured nethserver-docker on a green-only host
added red interface later
tried to ping from an alpine container ping 8.8.8.8 wget -qO- www.google.com
The ping was blocked. I found the aqua0 bridge was DOWN. It wasn’t bring up by interface-update event. So we need to add some logic to it, to bring it up again. This command was enough to work around the issue:
ip link set aqua0 up
Please check the aqua0 state: if you find it DOWN take note of it!
just a quick test, and probably i miss something… i tested on a VM with only a green interface.
dedicated storage seems ok, but i cannot reach the exposed port of the container, any hints?
Forgot the Docker facilities that automatically expose a TCP port by altering the packet filtering / nat configuration : they are disabled in NethServer!
You have to define a port forward in the NethServer firewall configuration manually.
Edit: If you want to expose a web site, configure the Reverse Proxy
For developers: I think this can be done automatically (and easily) by an RPM too…