Continuing NS8 Nextcloud + other problems

NethServer Version: 8.4
Module: Nextcloud Version 1.4.1

I have been disappointed in how the migration from NS7 to NS8 has gone. While may talk about how seamless this migration is, I can attest to the fact that this is very complicated and I would suggest, not ready for prime time.

I was pleasantly surprised at how well NS7.9 worked and coming from ClearOS, the community was very welcoming. The community continues to be the strength of this project. Thanks to everyone who has helped me out over the past 2 years.

Post migration about 3 weeks ago, I still can’t get Nextcloud to work properly. Today I did the update to 1.4.1 and got the message that I had to login with the CLI to do the manual updates. :cry: I was hoping it would finally updated properly and fix the issues that I haven’t been able to solve. But my hopes were misplaced. I still have following problems:

  • I can’t get the emails to work for invitations or from admin
  • I can’t get the photos to show up. There is only a blur for each image. When you click on the blurred image, you can see the picture but essentially the preview is broken.
  • Nextcloud office isn’t working properly, usually won’t start, when it starts it crashes (or hangs?) when opening a document. I’m still trying to figure out the following:
Could not establish connection to the Collabora Online server.

Failed to connect to the remote server: The antivirus executable could not be found at /usr/bin/clamscan
  • As anyone who saw my previous post, @mrmarkuz helped me solve the problem with re-attaching samba shared folders (Thank you again @mrmarkuz !!). I think this is a Nextcloud bug but still, it required the better part of three evenings to fix it.
  • Face recognition says “current model not installed”; maximum memory: not configured and “Seems that you still have to configure the assigned memory for image processing.” Nothing in the NS8 documentation about this.

Since migrating, I can’t get my websites to work. I installed the third party app and followed the instructions but what worked perfectly in NS7 doesn’t works at all in NS8. For the site I only get “404 page not found”. I’m astonished that this isn’t part of the core functionality. The basic stuff really needs to work out of the box.

I hope I have time over the next few weeks to spend some evenings to get this going but overall, I made a big mistake to upgrade. I am disappointed.

I’m sorry to read about your bad migration experience. I think it’s possible to fix it.

Did you set up the notification settings in the NS8 settings? See also Email notifications — NS8 documentation

It seems that the app “antivirus for files” is enabled, please disable it and test again. I could reproduce the missing preview by enabling the app but Collabora still worked here.

If the preview still doesn’t work you could try other scale or quality settings, see also Previews configuration — Nextcloud latest Administration Manual latest documentation

What websites is it about?
Which third party app? lamp, wordpress or the webserver app.
What did you already try?

Thank you for your thoughtful and detailed comments. I will look into these this evening after work.

I can answer the one question in the mean time.

What websites is it about?
Which third party app? lamp, wordpress or the webserver app.
What did you already try?

I installed Stephan’s WebServer app from the Software Centre. I believe it is nginx. I followed the documentation instructions. I created a login and password for the sftp user to upload the website content. I uploaded the content, checked that I have DNS records for the sites (still there carried over from the NS7.9), updated the firewall forward from the IP of the old NS7.9 to the new NS8 instance. Since then I’ve checked and rechecked the settings and DNS records. I don’t know what to check next as I have no idea how the networking to enable the website would work through the container set up. I haven’t been able to figure out how it is supposed to work so I can’t figure out where to start on debugging what is wrong.

1 Like

That’s right, webserver uses nginx, but there’s also lamp or specific apps like wordpress.

Did you create a user in sftpgo web admin? This isn’t necessary as required users are created when adding a virtual host, see also WebServer — NS8 documentation

Did you create virtualhosts for your website(s) and uploaded content with the right users? (starting from username 9001)

1 Like

Thank you @mrmarkuz .

Yes, I created two virtual hosts and a user 9001 for the first virtual host and 9002 for the second virtual host. I logged in with the correct user and copied the website from NS7.9 to the the virtualhost in NS8.

These are basically brochure websites, i.e. html, text, a few pictures and 1-3 pages with only links to the sub pages and no database or anything else.

Hm, usually this should just work. The names of the virtualhosts must not be in use for the host or domain name of the NS8 host.

You could check if the page is reachable on the NS8 using curl:

curl -v http://yourvirtualhost.domain.tld

Maybe it’s about permissions? To check it for user 9001, the first virtualhost:

[root@ns8rockytest ~]# runagent -m webserver1 podman exec -ti sftpgo ls -l /srv/sftpgo/data/9001
total 4
-rw-r--r--    1 root     root            21 Mar 14 12:38 info.php

Thanks @mrmarkuz . Here is my progress.

Update Websites.

I believe that may have solved the issue.

When I did the set up, I read that the instructions said to use the the FQDN. So I used domain.tld just like I did in NS7.9. But I didn’t know that if that conflicts with the NS8 host, then what I did won’t work. A note in the docs would be helpful for the newbies like me. I just changed the virtualhost name to www.domain.tld and now it will work.

The second virtualhost domain was domain2.tld and that didn’t work either until I added the www.domain2.tld. Now it is also reachable from the internet.

Next question regarding websites.
In the past I had redirects for domain.tld to www.domain.tld to make sure that if someone just added typed domain.tld that they still got to the website. How do I make sure that someone typing domain.tld ends up at the website www.domain.tld. If I read your comment correctly, I can’t do a redirect from domain.tld or the admin console will not work???

Update on Nextcloud

I also disabled the antivirus in Nextcloud. So now I can actually see the images. That is very strange and unexpected. I like to have clamav running just in case someone uploads something by accident. Running Nextcloud on NS7.9 worked with the antivirus. Is there a way to get clamav running and still see the images? It is a bit scary to have a naked Nextcloud instance.

I will need to keep working tomorrow as I’ve run out of time today.

Thanks again for your help.

Did you add both FQDNs, one without and one including www?

This way I could access the website at ns8test.com and www.ns8test.com and I could reach the cluster-admin at ns8test.com/cluster-admin. (ns8test.com is the NS8 domain name)
IIRC this didn’t work when I tested it the last time, maybe the traefik update made it work.

Maybe unexpected but technically understandable: Clamav isn’t available for Nextcloud so the scan fails (which seems also executed before thumbnail generation) so the previews are missing.

In NS7 clamscan and clamd were available at OS layer so they could be used by other software than mail like Nextcloud.
In NS8 there’s no clamav in the Nextcloud container and the mail server clamav uses socket so it can’t be used by other software.
While it’s possible to install clamav in the Nextcloud container which makes antivirus and previews work, I doubt it’s a good solution as clamav needs a lot of ressources.

Clamav seems intended for mail scanning so there could be issues with false positives when scanning files but for sure it’s better than nothing.

Maybe we should create a clamav container to be used from other apps. :thinking:
See also Threatshield and CroudSec do not prevent / protect from malware (AntiVirus) at ALL! - #5 by stephdl

1 Like

Hi @mrmarkuz. Thanks again for your ongoing help.

Websites update

Seeing that you entered multiple FQDN was unexpected. I didn’t realize that you could do that even though it says “Virtual host names”. There aren’t any instructions to add each FQDN on a line in the Edit virtual host dialog box or in the documentation. Cool!

I just added the domain.tld to the www.domain.tld and tried to access the https://domain.tld from outside my network. Unfortunately it didn’t work. Instead of going to the webpage it ended up at my OPNSense router. So my suspicion is that it isn’t getting forwarded from my router but I’ll have to check. It is weird that the www.domain.tld works but the other doesn’t.

Nextcloud clamav

I’m a bit confused about this. While I understand that clamav may not be enabled in the docker build, I installed the virus scanner from within the Nextcloud Admin portal. I didn’t know that it wasn’t actually installing clamav but rather only accessing the engine that has to be installed separately. Bummer.

After reading through the Threatshield and CrowdSec do not prevent / protect from malware, and the link to info about clamav in a docker container,I understand that it needs some further research however I have to agree that this is an important feature that should be added. As @Andy_Wismer indicated, without any antivirus, you’re potentially exposed. I’m also surprised at the amount of RAM required to run it in docker. The amount of memory usage will become a problem for my home server.

I’m surprised as I’ve been running clamav on Macs forever (at least since 2005) and on ClearOS (back to Clarkconnect & PointClark Networks) and then NS7.9. I don’t ever remember reading that it required so much memory to run. Most of the Macs are RAM limited by design (thanks Apple soldering in memory) and I never remember having a memory or performance issue caused by it. Since I moved some of our home Macs to SophosHome, I have had those performance issues and have considered going back to clamav. Now I’m not sure.

Well enough about Clamav. I have to get onto figuring out how to get Nextcloud office to work.

Thanks again.

Maybe you’re using reverse proxy instead of port forwarding on the OPNsense?

Could it be that you set an internal DNS entry for one domain but not the other? In that case the domain without internal DNS entry will resolve the domain with the public IP and when NAT reflection is disabled on the OPNsense the port forwarding is not done and you end up at the OPNsense.

So either create an internal DNS entry for the domain or enable NAT reflection on the OPNsense, see also Reflection and Hairpin NAT — OPNsense documentation

What I meant is, that it makes no sense to use multiple clamavs for mail, samba, nextcloud etc as it needs some RAM, see also Introduction - ClamAV Documentation

Maybe it helps to save the NS8 Collabora settings to reapply the config.

A quick update.

I have been able to figure out how to get NextCloud to send invitations again. Thank you @mrmarkuz for your suggestions.

I didn’t realize that I need to set up email notifications. Once I did that, I had to check the relay as I already had a mail server running. I had already set to allow relay from internal IP addresses and OpenVPN network segment. However I did set that everyone has to use password even within the network. While this isn’t really necessary, I’ve had my wife and children complain that they couldn’t send email once they left our network as they hadn’t set their smtp password. So if they are forced to do it when they set up the email app on phone or computer, then when they are outside of the network, the email will work. I had to use my userid and pswd for authorization and then the Nextcloud invitations are being sent.

Now onto checking and trying to figure out why Nextcloud office doesn’t work properly.

1 Like

@mrmarkuz , I don’t understand what you mean. Is this settings in the NextCloud Collabora app or the separate Collabora docker container?

It’s about the app settings In the cluster admin:

Please also check the Nextcloud app settings, if the collabora server is selected:

1 Like

Thanks @mrmarkuz .

I was able to get Collabra to start up and edit documents in NextCloud!!! So that has been solved also. Again, thanks for your help.

It is really quite slow compared to the NS7 NextCloud Collabra. I suspect this is due to my Nextcloud & Collabra running on a hard drive rather than the original SSD.

Once my daughter has cleaned up the ~120GB of duplicate photos and videos that he mistakenly uploaded twice prior to the migration, (and was the cause of moving the instance over to the hard drive during the migration), I should be ready to move the images back onto the SSD. At that point I will be back to code on the SSD and data on the hard drive.

Now onto the rest of the outstanding issues.

2 Likes

NS8 requires a SSD :stuck_out_tongue:

1 Like

Yes, @stephdl that is in the documentation. It was a conscious decision because there was no alternative with my present home lab set up.

It doesn’t always make sense to put large amounts of mostly static data that requires limited access onto a relatively expensive SSD. If the migration had gone well, my NextCloud container would have stayed on the SSD but it didn’t go well.

In any case, once everything is working well and cleaned up, I’ll try to move it back to the SSD. At the moment it appears that it isn’t possible so I have to wait. I believe I read that it is envisioned to have the ability to move containers in one of the point updates in the future. Here’s hoping it is sooner rather than later. :grinning:

increase the timeout of the systemd service if you want to run it on spinning drives, a container is an OS, and even fast it needs a lot of IO

Thank you @stephdl .

I have to do some research as I have no idea how to “increase the timeout of the systemd service”. :grinning:

It is much appreciated to receive suggestions as to what to do. I’m becoming a fisherman.

this is an example with piler

if not present you take the default value, if enable you can increase the time before the service gives up

keep in mind that you cannot add 120s like that, probably another service is waiting for this service to be up to do some requires on it.

We ask SSD because when a service starts, it is a mini OS that is starting, A container is just a specialized operating system, that is running inside your operating system.

1 Like