I found a simpler solution. Just proxy to https instead of http (which is not working currently). This removes the need to change existing templates. However, the files webtop5-webdav.conf, webtop5-zpush.conf, webtop.conf still have to be removed, because the take precedence over the reverse proxy configuration.
Also I improved the reverse proxy to handle WebSockets automatically.