Nethserver-discourse, Let's Encrypt, and SSL

Hi all and especially @stephdl on holiday in the canyon,

After 3 weeks of struggle: installation, reinstallation, standard Discourse, nethserver-discourse, Apache disable / Nginx, Nginx only, HAProxxy, Apache only, I finally found the cause of the warning in the URL bar and the unable to reconnect after “force https”.

With standard Discourse:

The warning in the URL bar is due to the installation of the Logo

  1. It is absolutely necessary not to install the logo when connecting to Discourse for the first time.

  2. You must install the logo only after the configuration is complete with Settings → Branding.

  3. To correct the situation:

  • Remove logo in Settings → Branding.
  • Reinstall the logo in Settings → Branding.

Not able to login is due to Apache

In the file: /etc/httpd/conf.d/zzz_discourse.conf, just before the line ProxyPass, add this line:

 RequestHeader set X-Forwarded-Proto "https"

Restart Apache

# systemctl restart httpd

It should be the same with nethserver-discourse and NGINX /etc/nginx/conf.d/zzz_discourse.conf

force https

  • Settings → Security → activate “force https”.
  • Finally reboot to make absolutely sure everything is working perfectly.

image Complete victory, let us raise the banner of victory.

An how-to will follow soon,

Michel-André

Hi all and especially @stephdl on holiday, if he is out of the canyon, :wink:

For Nethserver-Discourse

After a backup snapshot, restart the VM

Wait at least 2-3 minutes to let time for clamd to finish.
Wait at least another 2-3 minutes to let ruby start Discourse (there are 4 ruby running for discourse).
image

FILE: /etc/httpd/conf.d/zzz_discourse.conf

...
ProxyPreserveHost On
ProxyRequests Off

# Add those 2 lines to be always able to login
RequestHeader set X-Forwarded-Proto "https"
AllowEncodedSlashes NoDecode

ProxyPass / unix:/var/discourse/shared/standalone/nginx.http.sock|http://localhost/
ProxyPassReverse  / unix:/var/discourse/shared/standalone/nginx.http.sock|http://localhost/
...

Restart Apache:

# systemctl restart httpd
# systemctl status httpd  |  grep Active

Clear caches

  • Station:

    ipconfig /flushdns

  • Browser:
    History → Clear Recent History…

DO NOT RELOAD THE BROWSER PAGE, USE THE FULL LINK: https://forum.toto.org/.

  • Login

  • Configuration → Parameters → Security → Activate “force https”
    Refresh page - warning is still there => normal because of the “Branding icons”.

  • Configuration → Parameters → Branding
    Delete “logo” -> click the green check mark to activate the suppression
    Delete “small logo” -> click the green check mark to activate the suppression
    Repeat if more logos.
    Make sure there are no more logos.

    Download a new “logo” -> click the green check mark to activate the new logo
    Download a new “small logo” -> click the green check mark to activate the new “small logo”
    Repeat if more logos.

  • Logout
    The page refreshes to display the home page, there is no more warning in the URL bar.

Reboot, clear caches, and login to be absolutely sure that all is running perfectly.

Michel-André

P.S.
Use 2 different browsers to login; if you have problems after activating “force https”, you can use the other browser still logged in to deactivate it.

released in my rpm thank

2 Likes
1 Like
1 Like

About X-Forwarded-Proto, We could release a prop that enables the additional headers. The prop can be enabled by default starting from 7.10. I’d avoid a new UI checkbox because the new default behavior shouldn’t harm in 98% of cases :crossed_fingers:

RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}

Current template:

https://github.com/NethServer/nethserver-httpd/tree/master/default/etc/e-smith/templates/httpd/proxy-extra

Also X-Forwarded-For, X-Forwarded-Host and Forwarded are interesting :thinking:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

1 Like

Thanks to Stephane we have now this new enhancement!

@michelandre, @quality_team: could you test it? You can find QA instructions here: https://github.com/NethServer/dev/issues/6558#issuecomment-919395102

2 Likes

Hi @davidep and @stephdl,

Thank you for this improvement, it is appreciated.

Correct me if I’m wrong:

  • X-Forwarded-Proto => to pass the protocol i.e. https originated from SOURCE (client) to DESTINATION (LOCAL host)
  • X-Forwarded-For => to pass the IP address of the SOURCE to DESTINATION
  • X-Forwarded-Host => to pass the hotname of the SOURCE to DESTINATION

So those directives take care of the “Header” of the packet.

I think I have the same kind of problem - packet received by the Proxy is different from the packet received by DESTINATION. The Proxy “plays” with the packet before relaying it.

In my problem with Matrix-Synapse packet received, the packet is “canonized” before being relayed, meaning that any character %40 is replaced with @ and %3A with :

EDIT: If this packet contains a URL, itself containing the above characters, when it is received by DESTINATION it isn’t what is expected.
REF: https://github.com/matrix-org/synapse/issues/10749, especially the post of richvdh.

To resolve this new problem, you have to give the option nocanon.

      ProxyPass /_matrix http://127.0.0.1:8008/_matrix nocanon

Can you also implement such a directive’s option ?

Michel-André

P.S. Can you please give the CLI command to install/test the new directives.

from mod_proxy - Apache HTTP Server Version 2.4

Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of PATH_INFO. The optional nocanon keyword suppresses this and passes the URL path "raw" to the backend. Note that this keyword may affect the security of your backend, as it removes the normal limited protection against URL-based attacks provided by the proxy.

1 Like

Hi @stephdl,

Martin gave me full acces to his Chat installation and I compared all the configurations with mine.
All are the same except he is running it hosted at some place meaning that it is directly connected to the Internet with nothing in front of his Chat server.

My installation is running on a LOCAL VM and my main NS server redirects all related URL/PORT to the LOCAL server.

As answered on a forum where I asked for any suggestions to resolve the problem:

… your logs showed that something upstream of your server (Apache, presumably, or possibly Cloudflare) was replacing %40 with @ in the request URI of incoming requests (and %3A with : ), hence invalidating the signature made by the sending server.

I suspect that it is my main NS server redirections that are replacing the %40, the %3A, etc… characters but I don’t know if it is the URL or the Port redirections doing it.

I checked /etc/httpd/conf.d/virtualhosts.conf and nowhere there is a nocanon directive parameter.

Since my main server is quite full of all kind of modules/applications which are all working correctly, I don’t want to do a mistake by changing an unappropriate parameter.

Usually, nocanon goes after the retry=30 as in the line below.

ProxyPass  / https://192.168.1.111/ max=3 retry=30 nocanon

I have a few redirected URL.

Where do you think I should insert the nocanon for the corresponding lines?
Should I modify with a custom template and which ones ?
Will it affect all other vhosts I have, like mediawiki, etc ?

Thank you in advance,

Michel-André

in that case I would simply drop a apache reverse proxypass virtualhost in /etc/httpd/conf.d/myproxyCannonNeeded.conf

Hi @stephdl,

Thank you for your reply.

On my main server, the one connected directly to the internet that redirects to LOCAL, already contains entries for each redirection in /etc/httpd/conf.d/virtualhosts.conf.

On the main server, do you mean:

  1. Create a file that comes alphabetically before virtualhosts.conf
  2. Copy the redirection sections from virtualhosts.conf to the new created file.
  3. In the new created file, add the nocanon directive parameter at the end of the ProxyPass lines.

Correct me if I’m wrong:

  • When the httpd daemon will look for the redirected port/ServerName, it will take the directives from the new file as it comes before virtualhosts.conf ?

Michel-André

1 Like

yes it should work, cp and paste in a new configuration file your proxy and remove the old one of the cockpit UI

Salut @stephdl,

Should I also delete the port redirections in Cockpit UI ?

Michel-André

I do not know, you have to provide a full valid httpd configuration in your new configuration file, if needed then yes :smiley:

Salut @stephdl,

I redirect 8008 and 8448 ports and for them there are no entries in /etc/httpd/conf.d/virtualhosts.conf.

I think that I should not delete them in Cockpit UI.

Michel-André

no idea sorry I do not understand

Open a new topic and display your full proxypass configuration and what you want to modify

Hi @stephdl,

  1. I started LOCAL VM.
  2. On main server, I copied /etc/httpd/conf.d/virtualhosts.conf to /etc/httpd/conf.d/u_nocanon.conf.
  3. I kept the original virtualhosts.conf as is.
  4. In u_nocanon.conf I deleted all sections not related to the redirections.
    I added nocanon after retry=30 in all sections.
  5. On main server, I restarted httpd.
  6. On LOCAL VM, I restarted httpd and matrix-synapse.
  • I logged into my LOCAL Chat server with Firefox.
  • I logged into matrix.org with Chrome.
  • On my LOCAL Chat server, I created an encrypted public Room.
  • On my connection to matrix.org, I Search for the created Room on LOCAL Chat server => Join.
  • My connection to matrix.org joined the Room on LOCAL Chat server without problem (which is a Première).
  • Exchange a few chats between my Chat and matrix.org => both were fast to display the exchanges.

CONCLUSION:
It is the web redirections that canonicalise (replacing the characters) %40 , the %3A , etc…

QUESTION:
Is it possible to do the same with this directive parameter as you did with X-Forwarded-Proto ?
Might be quite tricky as it is a parameter and not a full directive… But since you are adding max=3 and retry=30 parameters only in redirection sections, can you also add nocanon ?

Michel-André

2 Likes

Hello michel, yes we added new feature to proxypass, but I am not sure for the nocanon, because it might be risky as the apache documentation said, but you are right to ask for it because developers sometimes are in their ivory tower.

We added for now the X-Forwarded-Proto and soon the AllowEncodedSlashes NoDecode but for now without UI checkbox. Maybe with UI we could add another one to enable nocanon because I know nocanon works with AllowEncodedSlashes NoDecode.

In my dream night I would like to change a bit the UI and allow more complex proxypass feature, for now we just allow to do a reverse proxy from / to another http URL, and I would like to be able to this in the UI if needed

<VirtualHost *:443>
  ServerName collabora.domain.com
  SSLEngine on
  # Encoded slashes need to be allowed
  AllowEncodedSlashes NoDecode
  # keep the host
  ProxyPreserveHost On
  # static html, js, images, etc. served from loolwsd
  # loleaflet is the client part of LibreOffice Online
  ProxyPass           /loleaflet http://127.0.0.1:9980/loleaflet retry=0
  ProxyPassReverse    /loleaflet http://127.0.0.1:9980/loleaflet
  # WOPI discovery URL
  ProxyPass           /hosting/discovery http://127.0.0.1:9980/hosting/discovery retry=0
  ProxyPassReverse    /hosting/discovery http://127.0.0.1:9980/hosting/discovery
  # Main websocket
  ProxyPassMatch "/lool/(.*)/ws$" ws://127.0.0.1:9980/lool/$1/ws nocanon
  # Admin Console websocket
  ProxyPass   /lool/adminws ws://127.0.0.1:9980/lool/adminws
  # Download as, Fullscreen presentation and Image upload operations
  ProxyPass           /lool http://127.0.0.1:9980/lool
  ProxyPassReverse    /lool http://127.0.0.1:9980/lool
  # Endpoint with information about availability of various features
  ProxyPass           /hosting/capabilities http://127.0.0.1:9980/hosting/capabilities retry=0
  ProxyPassReverse    /hosting/capabilities http://127.0.0.1:9980/hosting/capabilities
</VirtualHost>

a textarea should be able to help us to make this

/loleaflet http://127.0.0.1:9980/loleaflet
/hosting/discovery http://127.0.0.1:9980/hosting/discovery
/lool http://127.0.0.1:9980/lool
/hosting/capabilities http://127.0.0.1:9980/hosting/capabilities

cc @davidep @giacomo

Salut @stephdl

I googled for “URL-based attacks” and it is mainly for mail Phishing.

With ClamAV, Fail2ban, etc, I think the dangerousity is quite low. Also, on all stations, I have Avast and Spybot.

But mainly as I am using NethServer, I think it is almost irrelevant.

Michel-André