Improve reverse proxy, automatic handling of websockets

agreed, but the proxypass UI is not maybe the good tool, a rpm with the vhost reverse configuration could be done.

No, on the contrary, there should be NO seperate RPM which magically creates some hidden configrations which intereferes with your manually created other reverse proxy entries. This leads to endles debugging sessions, which I had until I found hidden configuration files left over after package deinstalls.

The ONLY correct way is to enable to reverse proxy UI to reverse proxy all services of a to NS to a backend NS. Possible limitations (like “target must use https” which is actualy a bug in my opinion) and caveats (like "ensure correct public webtop URL in the backend server) should be documented in the documenation.

It is not neccessary to describe the general concecpt of how to reverse proxy at all, but would be a good option to give examples for webtop, nextcloud and matterhost. This includes the URs are mapped correctly (e.g. not webtop/webtop), or that the certificates have to be created in the frontend not in the backend.

1 Like

to recap

  1. the reverse proxy to a local websocket works (tested mattermost, webtop)
  2. the reverse proxy to a remote websocket works for webtop if you add the proxied domain to PublicUrl and DavServerUrl
  3. I cannot make it workable for mattermost, I think it is something internal because I got a 403 error

I tend to think the configuration is valid but some apps cannot works like this, so at this point my mind is split.

  • close the PR because it cannot work in most of 99.9999999999% of cases (I am kidding)
  • Use the SME Server way, make it workable only by a hidden esmith property, at least until we could be sure it is fully workable

Currently there is not a single hint that the option is not working. All problem disussed u ntil now came from wrong configurations or the fact that the NS application are not too easy to reverse proxy at all. This is however another issue which should be solved in the way I outlined before. Currently it work for be with webtop and nextcloud, and it will also work with other applications. So leave the option visible, close the issue.

The other issue that reverse proxing NS applications is not to easy in general should be a seperate task.

We reached an agreement, this feature will be released inside the proxypass UI, maybe we have to test it a bit more

4 Likes

We have a working implementation thanks to @stephdl work!
Probably, the feature will be marked as experimental, but before taking the final decision we need your feedback!

So, please try the packages from the PR:

yum install -y http://packages.nethserver.org/nethserver/7.8.2003/autobuild/x86_64/Packages/nethserver-httpd-3.10.11-1.16.pr82.g70ba85e.ns7.noarch.rpm http://packages.nethserver.org/nethserver/7.8.2003/autobuild/x86_64/Packages/nethserver-httpd-proxypass-3.10.11-1.16.pr82.g70ba85e.ns7.noarch.rpm http://packages.nethserver.org/nethserver/7.8.2003/autobuild/x86_64/Packages/nethserver-httpd-virtualhosts-3.10.11-1.16.pr82.g70ba85e.ns7.noarch.rpm

To revert the changes:

yum downgrade nethserver-httpd nethserver-httpd-proxypass nethserver-virtualhosts 

Please @carsten @royceb @djx test it and report your feedback!

2 Likes

oO that really came just about the time I needed. Thanks all.

Just a question. If websocket listens to more than 1 path (e.g /wss and /websock) is there any way to include them all in the path rule?

Also the yum downgrade seems not to work

yum downgrade nethserver-httpd nethserver-httpd-proxypass nethserver-virtualhosts
Loaded plugins: changelog, fastestmirror, nethserver_events
No Match for available package: nethserver-virtualhosts-1.0.6-1.ns7.noarch
Resolving Dependencies
→ Running transaction check
—> Package nethserver-httpd.noarch 0:3.10.11-1.ns7 will be a downgrade
—> Package nethserver-httpd.noarch 0:3.10.11-1.16.pr82.g70ba85e.ns7 will be erased
—> Package nethserver-httpd-proxypass.noarch 0:3.10.11-1.ns7 will be a downgrade
—> Package nethserver-httpd-proxypass.noarch 0:3.10.11-1.16.pr82.g70ba85e.ns7 will be erased
→ Finished Dependency Resolution
Error: Package: nethserver-httpd-virtualhosts-3.10.11-1.16.pr82.g70ba85e.ns7.noarch (@/nethserver-httpd-virtualhosts-3.10.11-1.16.pr82.g70ba85e.ns7.noarch)
Requires: nethserver-httpd >= 3.10.11-1.16.pr82.g70ba85e.ns7
Removing: nethserver-httpd-3.10.11-1.16.pr82.g70ba85e.ns7.noarch (@/nethserver-httpd-3.10.11-1.16.pr82.g70ba85e.ns7.noarch)
nethserver-httpd = 3.10.11-1.16.pr82.g70ba85e.ns7
Downgraded By: nethserver-httpd-3.10.11-1.ns7.noarch (nethserver-updates)
nethserver-httpd = 3.10.11-1.ns7
Available: nethserver-httpd-3.9.0-1.ns7.noarch (nethserver-base)
nethserver-httpd = 3.9.0-1.ns7
Available: nethserver-httpd-3.10.0-1.ns7.noarch (nethserver-updates)
nethserver-httpd = 3.10.0-1.ns7
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

Btw reverted back by doing the below

yum install -y http://packages.nethserver.org/nethserver/7.8.2003/base/x86_64/Packages/nethserver-httpd-3.9.0-1.ns7.noarch.rpm http://packages.nethserver.org/nethserver/7.8.2003/base/x86_64/Packages/nethserver-httpd-proxypass-3.9.0-1.ns7.noarch.rpm http://packages.nethserver.org/nethserver/7.8.2003/base/x86_64/Packages/nethserver-httpd-virtualhosts-3.9.0-1.ns7.noarch.rpm

yum update

Welcome to NethServer community :slight_smile:

Not from the UI, but if you do not specify the path, the rewrite rule could already do some magic for you.
Just try it and see how it goes :wink:

1 Like

Nobody wants to test this new feature?
I hope that someone would spend few minutes on it during the weekend! :crossed_fingers:

i would love to test, but i am not sure what i would be testing exactly in this case. So ill let those with more experience

1 Like

This is to test to see if the reverse proxy functions with web-services that utilize web sockets (FreeNas/Xen Orchestra are two off the top of my head that I’ve experienced).

Give me some time today @giacomo and I’ll test it out via ns droplets. All my server hardware right now is waiting for their new fiber line to be hooked up by the end of July :confused:

2 Likes

@giacomo

Hi Giacomo

Was the whole week in Germany not far from Frankfurt and had today a big Demo with Proxmox, NethServer and OPNsense. The client group were VERY Happy with the whole thing!

@alefattorini
They liked the whole show that much I might be able to convince them to become Partners of Nethesis, time will show
 They are much bigger than me and my clients - and it could be a real win win for both!

I’m headed home tomorrow, but it’s still a 6 hour trip. But Sunday I can test this!

My 2 cents
Andy

1 Like

Worked as expected with the upgrade instructions; before vs after. Excellent work and super easy to configure and deploy. Thank you for all of the hard work you all put into NethServer.

3 Likes

Thank you @royceb for testing!
Since this was a scenario to cover, I’d proceed with the merge and move the package toward the release! :wink:

Thanks to work of the community, everything is in testing!

I think we can release thanks to Rocye testing, but @carsten would you like to give it the last try?
You’re the one who was proposing the feature :wink:

1 Like

Yes, it works fine, well done. However the extra option for the websocket path does not make any sense and leads to confusion of users and potentional misbehaviour of the proxy. The rewrite rule parser is used to determinate whether a websocket upgrade has been requested and determines the path automatically. There is NO possiblility for the path to ever make sense. It either contains the correct value which the rewrite rule already determined or does not work. So I would suggest removing this option, because it makes no sense.

In theory the rewrite is somewhat slower than ProxyPass, so IF you want to use the path (but keep in mind, that there could be multiple websocket paths, so the single path cannot work here either), then do use Proxypass instead of the rewrite and do no parsing with the rewrite module at all.

So I strongly suggest removing this option!

Another option is missing however to proxy widespread applications: flushpackets=on

It is needed for Guacamole for example as an option to the proxypass statement. So either make another check box or have a “custom proxypass options” field where one can enter the proxypass options completely yourself.

Currently “max=3 retry=30” is added automatically for every reverse proxy without the possiblilty to change it. Having an “custom proxypass option” could add other fields like “flushpackets=on” and also remove or change the default of max and retry (which should also be not set for guacamole).

If “custom proxypass option” is empty, just add the default like now, but when it is not empty append the option to the proxypass statement. With this change,also guacamole could be reversed proxied with the Nethserver GUI. So maybe you change/rename the superflues option to this new one.

Guacamole is currently the only application where I have to use a custom template.

Currently nethserver generates:
ProxyPass /guacamole http://192.168.42.16:8080/guacamole max=3 retry=30

But Guacamole needs:
ProxyPass /guacamole http://192.168.42.16:8080/guacamole flushpackets=on

2 Likes

I simplified the template 30ProxyPass, corrected the order of rewrite and proxypass statements and extended it with the feature “custom proxy pass options”, by misusing the variable, which shoud be renamed. Now all my reverse proxy entries including guacmole with with the UI. For guacamole I have to put “flushpackets=on” on the option “custom-proxypass-opion” (now named WebSocketsPath).

 {
        my $proxypassoptions=$WebSocketsPath;
        if (proxypassoptions eq '') { $proxypassoptions='max=3 retry=30'; }
        if ($WebSockets eq 'enabled' ) {
            my $ws = $Target =~ s/^http/ws/r;
            $OUT .= "    # Websockets proxypass\n";
            $OUT .= "    RewriteCond \%\{HTTP:Upgrade\} websocket [NC]\n";
            $OUT .= "    RewriteCond \%\{HTTP:Connection\} upgrade [NC]\n";
            $OUT .= "    RewriteRule .* ${ws}\%\{REQUEST_URI\} [P,L]\n\n";
        }
        my $target = $Target =~ s|/*$|/|r;
        $OUT .= "    # Reverse Proxy (with exclusion of local Letsencrypt challenge path)\n";
        $OUT .= "    ProxyPassMatch ^/.well-known/acme-challenge/ !\n";
        $OUT .= "    ProxyPass / $target $proxypassoptions\n";
        $OUT .= "    ProxyPassReverse / $target\n";
    }

The feature has been released! :rocket:

Adding an extra option could be cool, but watch out for validation. :wink:
When you have something that could do the job, please open a PR!

Sorry for the late reply, but the option WebProxyPath should be removed. It is totally useless. We are still in testing, so corrections should be possible before this code hits the public.