Docker Aqua Network Firewall Rules

Support
, , ,
1

NethServer Version: 7.9.2009
Module: Services, Firewall, Web Server (Reverse Proxy)

My situation is somewhat like this thread.

I currently have multiple services running in docker containers, using the aqua network, and reverse proxy for SSL. Each of these services exposes an HTTP port which the Reverse Proxy maps to a domain and serves the SSL cert.

Two of these services are also using LDAP as enabled by a docker rules entry.

I want to allow WikiJS to use GitLab for auth but I’m having connectivity issues. Due to the way WikiJS works I can’t configure it to refer to GitLab by the internal address on the Aqua network. I can only configure a single URL for WikIJS to connect to a GitLab instance for auth - so this needs to be the public address.

My attempted solution was to allow the Aqua network access to 443, so that it would be able to access GitLab. This does not seem to work at all. On Neth Server itself I can get a response on port 443 through telnet. Within the WikiJS container, I cannot access 443 on the aqua network - but I can access 636 on the aqua network - even though both ports are in the same docker rule.

I bet I have a misunderstanding of how to access the other side of the reverse proxy internally and I’m looking for help.

Here’s my docker rule:

# db dockrules show
jitsiLdap=aqua
    TCPPorts=389,636,443
    status=enabled

Here’s my attempt at the diagram. Everything except the red arrows is working.

NethServer Firewall
NethServer Firewall927×707 38.8 KB

2 Likes
2

Thanks.
For a detailed explaination. For the diagram. For te clear goals.
And most for all… “The big scary Internet” icon in the bottom.
:slight_smile:

1 Like
3

I’ve done some more digging and found that the traffic isn’t even making it to the reverse proxy; it’s being blocked by the firewall:

Shorewall:aqua2fw:REJECT:IN=aqua0 OUT= MAC=<redacted> SRC=172.28.0.4 DST=<public_ip> LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=4050 DF PROTO=TCP SPT=45134 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0
4

After disabling the firewall with shorewall clear the traffic gets through just fine. Is there a recommended way to allow just this traffic? I don’t want to tear bigger holes in the firewall than I need to.

1 Like
5

@mrmarkuz - sorry to ping you, but it looks like you’ve answered a similar question here: Firewall and Aqua Network issue - #5 by JOduMonT. I believe I am within the typical use-case of having a container connect to a localhost port (443) by adding the rule to dockrules, but shorewall still reqjects it. Any ideas?

6

It seems you want to connect a container to the public ip instead of localhost.

It should be possible to create a shorewall rule or a port redirection. I’m going to test the WikiJS and Gitlab connection when I find time, maybe there’s another solution. Do you use NethServer in server (just green interface) or gateway mode (green and red interface)?

No, this was the other way round, connecting from localhost to container.

1 Like
7

My Nethserver is in gateway mode; it has access to the internet. I tried playing with the shorewall rules but couldn’t get anything to just allow that traffic through without completely disabling the firewall.

It seems you want to connect a container to the public ip instead of localhost

I could accept the container being able to connect to aqua:443, because I could manually edit /etc/hosts in the container to point to localhost for the specific domain names required. As long as it can access port 443, this should work fine and be easier than making code changes in containers.

For full disclosure, I’ve also made an alternate solution for my current problem by submitting a PR to WikiJS that allows me to provide it an alternate (internal) URL for checking tokens. When I set that to the container’s Aqua IP I am able to get the containers to talk together. However, this requires me to hard-code the IPs of the containers to ensure the mapping works. Also, I think there may come a time where I am unable to edit the container and still need it to talk to another one through the reverse proxy - so I’d still like to understand how to do this for future cases.

Thank you for your time. :slight_smile:

1 Like
8

I cannot reproduce the issue. After setting dockrules TCPPorts to 443 I can connect to aqua port 443 from the wikijs container.

I don’t have a valid cert in my testing environment but the following works:

docker exec wikijs-wiki-1 curl -k https://172.28.0.1

docker exec wikijs-wiki-1 curl -k https://<public ip>

I can reach the sites from inside the containers:

docker exec wikijs-wiki-1 curl -k https://gitlab.domain.local

docker exec gitlab-gitlab-1 curl -k https://wikijs.domain.local

I tested using just internal addresses for red and green and with pseudo domain names so maybe it’s a wrong testing environment.

My docker-compose.yml files to compare:

gitlab docker-compose.yml 
version: '2.3'

services:
  redis:
    restart: always
    image: redis:6.2.6
    command:
    - --loglevel warning
    volumes:
    - redis-data:/data:Z

  postgresql:
    restart: always
    image: sameersbn/postgresql:12-20200524
    volumes:
    - postgresql-data:/var/lib/postgresql:Z
    environment:
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production
    - DB_EXTENSION=pg_trgm,btree_gist

  gitlab:
    restart: always
    image: sameersbn/gitlab:15.11.2
    depends_on:
    - redis
    - postgresql
    ports:
    - "10080:80"
    - "10022:22"
    volumes:
    - gitlab-data:/home/git/data:Z
    healthcheck:
      test: ["CMD", "/usr/local/sbin/healthcheck"]
      interval: 5m
      timeout: 10s
      retries: 3
      start_period: 5m
    environment:
    - DEBUG=false

    - DB_ADAPTER=postgresql
    - DB_HOST=postgresql
    - DB_PORT=5432
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production

    - REDIS_HOST=redis
    - REDIS_PORT=6379

    - TZ=Asia/Kolkata
    - GITLAB_TIMEZONE=Kolkata

    - GITLAB_HTTPS=false
    - SSL_SELF_SIGNED=false

    - GITLAB_HOST=localhost
    - GITLAB_PORT=10080
    - GITLAB_SSH_PORT=10022
    - GITLAB_RELATIVE_URL_ROOT=
    - GITLAB_SECRETS_DB_KEY_BASE=hidden
    - GITLAB_SECRETS_SECRET_KEY_BASE=hidden
    - GITLAB_SECRETS_OTP_KEY_BASE=hidden

    - GITLAB_ROOT_PASSWORD=
    - GITLAB_ROOT_EMAIL=

    - GITLAB_NOTIFY_ON_BROKEN_BUILDS=true
    - GITLAB_NOTIFY_PUSHER=false

    - GITLAB_EMAIL=notifications@example.com
    - GITLAB_EMAIL_REPLY_TO=noreply@example.com
    - GITLAB_INCOMING_EMAIL_ADDRESS=reply@example.com

    - GITLAB_BACKUP_SCHEDULE=daily
    - GITLAB_BACKUP_TIME=01:00

    - SMTP_ENABLED=false
    - SMTP_DOMAIN=www.example.com
    - SMTP_HOST=smtp.gmail.com
    - SMTP_PORT=587
    - SMTP_USER=mailer@example.com
    - SMTP_PASS=password
    - SMTP_STARTTLS=true
    - SMTP_AUTHENTICATION=login

    - IMAP_ENABLED=false
    - IMAP_HOST=imap.gmail.com
    - IMAP_PORT=993
    - IMAP_USER=mailer@example.com
    - IMAP_PASS=password
    - IMAP_SSL=true
    - IMAP_STARTTLS=false

    - OAUTH_ENABLED=false
    - OAUTH_AUTO_SIGN_IN_WITH_PROVIDER=
    - OAUTH_ALLOW_SSO=
    - OAUTH_BLOCK_AUTO_CREATED_USERS=true
    - OAUTH_AUTO_LINK_LDAP_USER=false
    - OAUTH_AUTO_LINK_SAML_USER=false
    - OAUTH_EXTERNAL_PROVIDERS=

    - OAUTH_CAS3_LABEL=cas3
    - OAUTH_CAS3_SERVER=
    - OAUTH_CAS3_DISABLE_SSL_VERIFICATION=false
    - OAUTH_CAS3_LOGIN_URL=/cas/login
    - OAUTH_CAS3_VALIDATE_URL=/cas/p3/serviceValidate
    - OAUTH_CAS3_LOGOUT_URL=/cas/logout

    - OAUTH_GOOGLE_API_KEY=
    - OAUTH_GOOGLE_APP_SECRET=
    - OAUTH_GOOGLE_RESTRICT_DOMAIN=

    - OAUTH_FACEBOOK_API_KEY=
    - OAUTH_FACEBOOK_APP_SECRET=

    - OAUTH_TWITTER_API_KEY=
    - OAUTH_TWITTER_APP_SECRET=

    - OAUTH_GITHUB_API_KEY=
    - OAUTH_GITHUB_APP_SECRET=
    - OAUTH_GITHUB_URL=
    - OAUTH_GITHUB_VERIFY_SSL=

    - OAUTH_GITLAB_API_KEY=
    - OAUTH_GITLAB_APP_SECRET=

    - OAUTH_BITBUCKET_API_KEY=
    - OAUTH_BITBUCKET_APP_SECRET=
    - OAUTH_BITBUCKET_URL=

    - OAUTH_SAML_ASSERTION_CONSUMER_SERVICE_URL=
    - OAUTH_SAML_IDP_CERT_FINGERPRINT=
    - OAUTH_SAML_IDP_SSO_TARGET_URL=
    - OAUTH_SAML_ISSUER=
    - OAUTH_SAML_LABEL="Our SAML Provider"
    - OAUTH_SAML_NAME_IDENTIFIER_FORMAT=urn:oasis:names:tc:SAML:2.0:nameid-format:transient
    - OAUTH_SAML_GROUPS_ATTRIBUTE=
    - OAUTH_SAML_EXTERNAL_GROUPS=
    - OAUTH_SAML_ATTRIBUTE_STATEMENTS_EMAIL=
    - OAUTH_SAML_ATTRIBUTE_STATEMENTS_NAME=
    - OAUTH_SAML_ATTRIBUTE_STATEMENTS_USERNAME=
    - OAUTH_SAML_ATTRIBUTE_STATEMENTS_FIRST_NAME=
    - OAUTH_SAML_ATTRIBUTE_STATEMENTS_LAST_NAME=

    - OAUTH_CROWD_SERVER_URL=
    - OAUTH_CROWD_APP_NAME=
    - OAUTH_CROWD_APP_PASSWORD=

    - OAUTH_AUTH0_CLIENT_ID=
    - OAUTH_AUTH0_CLIENT_SECRET=
    - OAUTH_AUTH0_DOMAIN=
    - OAUTH_AUTH0_SCOPE=

    - OAUTH_AZURE_API_KEY=
    - OAUTH_AZURE_API_SECRET=
    - OAUTH_AZURE_TENANT_ID=

volumes:
  redis-data:
  postgresql-data:
  gitlab-data:
wikijs docker-compose.yml 
version: "3"
services:

  db:
    image: postgres:11-alpine
    environment:
      POSTGRES_DB: wiki
      POSTGRES_PASSWORD: wikijsrocks
      POSTGRES_USER: wikijs
    logging:
      driver: "none"
    restart: unless-stopped
    volumes:
      - db-data:/var/lib/postgresql/data

  wiki:
    image: ghcr.io/requarks/wiki:2
    depends_on:
      - db
    environment:
      DB_TYPE: postgres
      DB_HOST: db
      DB_PORT: 5432
      DB_USER: wikijs
      DB_PASS: wikijsrocks
      DB_NAME: wiki
    restart: unless-stopped
    ports:
      - "8080:3000"

volumes:
  db-data:

The used reverse proxies:

# db proxypass show
gitlab.domain.local=VhostReverse
    CertVerification=yes
    Description=
    HTTP=no
    HTTPS=yes
    PreserveHost=yes
    SslCertificate=
    Target=http://localhost:10080
    ValidFrom=
    WebSockets=enabled
    WebSocketsPath=
wikijs.domain.local=VhostReverse
    CertVerification=yes
    Description=
    HTTP=no
    HTTPS=yes
    PreserveHost=yes
    SslCertificate=
    Target=http://localhost:8080
    ValidFrom=
    WebSockets=enabled
    WebSocketsPath=

dockrules:

# db dockrules show
gitlabwikijs=aqua
    TCPPorts=389,636,443
    status=enabled

aqua rules in /etc/shorewall/rules:

# grep aqua /etc/shorewall/rules

# 65aqua Accept ping from aqua
Ping/ACCEPT    aqua            $FW
# 65aqua -- Rules for Docker containers
?COMMENT aqua
ACCEPT	aqua	$FW	tcp	3306
ACCEPT	aqua	$FW	tcp	389
ACCEPT	aqua	$FW	tcp	636
ACCEPT	aqua	$FW	tcp	443
2 Likes
9

Thank you very much for taking the time to do all this testing. I went through all my configs and checked that I was getting the same outputs as you. I changed my WikiJS’s gitlab auth back to using the public address and it just started working. I tried clearing cookies and removing the application from my GitLab profile to force the token flow to happen again, and it seemed to work perfectly fine. The only difference is that my proxypass configuration uses 127.0.0.1 where yours has localhost, but since it’s working now I don’t think that’s it.

I really don’t know what changed that it’s suddenly working, or if it’ll stay working.
I still see some entries in the firewall:

Shorewall:aqua2fw:REJECT:IN=aqua0 OUT= MAC=<redacted> SRC=172.28.0.5 DST=172.28.0.1 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=40186 DF PROTO=TCP SPT=8929 DPT=54962 WINDOW=0 RES=0x00 RST URGP=0

This appears to be the gitlab (172.28.0.5) container trying to communicate on port 8929 (used for reverse proxy forwarding) to the aqua network.I’m not sure what it’s for, but I can’t see any adverse effects from it being blocked (yet).

1 Like