NS8 Migration is failing with Nextcloud

[Andy_Wismer (André Wismer)]

NethServer Version: 8
Module: NextCloud

Hi

I am in the process of migrating a clients NS7 to NS8.

So far, everything has worked fairly well so far.

This is the current status, as seen from the NS8 Migration App:

As is shown, Mail has been sucessfully migrated. (About 145 GB data).

Nextcloud is the next one needed before the AD / File Server can be migrated.

A first sync has been done. Later syncs fail with this:

The contents of which are:

{"context":{"action":"import-module","data":{"credentials":["nextcloud1","9691202be877-03f5-47f2-98e8-f6bf60478fc4"],"port":20012,"volumes":["nextcloud-app-data"]},"extra":{"description":"ns8-action endpoint http://10.98.33.1:9311","isNotificationHidden":false,"title":"module/nextcloud1/import-module"},"id":"8493b2ea-cdb9-4dda-ac79-dd8fb8f971e2","parent":"","queue":"module/nextcloud1/tasks","timestamp":"2024-05-20T08:32:25.33650405Z","user":"admin"},"status":"validation-failed","progress":0,"subTasks":[],"validated":false,"result":{"error":"","exit_code":2,"file":"task/module/nextcloud1/8493b2ea-cdb9-4dda-ac79-dd8fb8f971e2","output":[{"error":"module_already_imported","field":"none","parameter":"none","value":""}]}}

---

I can provide more info / logs if needed.

How or Where can I see more information?

Or how can this be resolved?

Maybe @davidep or @Tbaile have some ideas?

TIA

My 2 cents
Andy

[Tbaile (Tommaso Bailetti)]

Hi Andy!

The error that pops up the most is module_already_imported in the response of the cluster api.

May I ask you if you have any instances of NextCloud in the cluster prior to the migration? And if so, can you list them using

api-cli run cluster/list-modules | jq '.[].installed | select(length > 0)'

(The output won’t be perfectly formatted json, don’t worry about that.)

[Andy_Wismer (André Wismer)]

Hi @Tbaile

No, as this is a fresh installation of Debian 12.5 on Proxmox from this weekend.

It doesn’t look like it from here either…:

[root@suma-ns8 ~]# api-cli run cluster/list-modules | jq '.[].installed | select(length > 0)'
Warning: using user "cluster" credentials from the environment
[
  {
    "id": "mail1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:07d3a1fde641c2600aaa036db27a22127a37d16e1ecb6830cc14ac383bb23f0d",
    "source": "ghcr.io/nethserver/mail",
    "version": "1.4.0",
    "logo": "apps/mail1/img/module_default_logo.ed42e539.png",
    "module": "mail",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "roundcubemail1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:d3d25788f94bc732ae27db4cda5a8377a911d53dfa5d82b0a87823dac91300f8",
    "source": "ghcr.io/nethserver/roundcubemail",
    "version": "2.0.2",
    "logo": "apps/roundcubemail1/img/module_default_logo.63a7380d.png",
    "module": "roundcubemail",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "imapsync1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:b82408d452f27b6e3f6f7af09c63c3b6ecd40263ea7c9674aaeccc0b3276cc42",
    "source": "ghcr.io/nethserver/imapsync",
    "version": "1.0.1",
    "logo": "apps/imapsync1/img/module_default_logo.ed42e539.png",
    "module": "imapsync",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "netdata1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:94af39a9ca7f846b8e9228a5c41130029bbd53a8c0f3097da8048600c5be89de",
    "source": "ghcr.io/nethserver/netdata",
    "version": "1.1.0",
    "logo": "",
    "module": "netdata",
    "flags": []
  }
]
[
  {
    "id": "nextcloud1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:549d18ecc9af7cb67666f2dd60de59ff421c3540f1976c78372a24be26e909f0",
    "source": "ghcr.io/nethserver/nextcloud",
    "version": "1.1.6",
    "logo": "apps/nextcloud1/img/module_default_logo.b209effe.png",
    "module": "nextcloud",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "mariadb1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:a36421e6ae22a318970d9a53bee47b35874603a1cbc4840f8c285622efd65cd2",
    "source": "ghcr.io/nethserver/mariadb",
    "version": "1.0.8",
    "logo": "apps/mariadb1/img/module_default_logo.bc0e2ce0.png",
    "module": "mariadb",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "webserver1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:3bf0229793fc8fe46c8aae6c72629a41817e684e0538307df550792e65130c6b",
    "source": "ghcr.io/nethserver/webserver",
    "version": "1.0.16",
    "logo": "apps/webserver1/img/module_default_logo.63283db0.png",
    "module": "webserver",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "grafana1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:c9cd732b23f58c9c1f4d7974f7e3e530ac41712789389f34c44f7cdfcea5a9a4",
    "source": "ghcr.io/nethserver/grafana",
    "version": "1.0.3",
    "logo": "apps/grafana1/img/module_default_logo.fcd8e61c.png",
    "module": "grafana",
    "flags": [
      "rootless"
    ]
  }
]
[
  {
    "id": "prometheus1",
    "ui_name": "",
    "node": "1",
    "node_ui_name": "",
    "digest": "sha256:7dcf082fbb8b723b44cb9a055cec92d217102b9b155d2497378ab6be5ae60aa5",
    "source": "ghcr.io/nethserver/prometheus",
    "version": "1.0.2",
    "logo": "apps/prometheus1/img/module_default_logo.cce7147c.png",
    "module": "prometheus",
    "flags": []
  }
]
[root@suma-ns8 ~]# 

When I couldn’t finish NextCloud, I tried finishing mail, that finished without issues, now Mail is on NS8.

But before AD and File-Server, I would like (need) Nextcloud migrated…

Maybe @mrmarkuz, @davidep or @Tbaile have some ideas?

My 2 cents
Andy

[mrmarkuz (Markus Neuberger)]

I found the error “module already imported” once more in the forum, is it the same NS7?

Test migration for a client running NS7

NethServer Version: 7.9.2009 Module: NS8 Migration Hi Doing a test migration for a client running NS7. NS8 is based on Debian12, all updates installed according to instructions in NS8 Admin Doku. NS8 shows no issuse when sarted or Cluster created. Local DNS is set correctly for both hosts, running both as VMs on a powerful HP server. NS7 and NS8 are both allocated 8 cores, about 200 / 300 GB sized disks (Ample!) and have 16 GB RAM each (No RAM Balooning!). VirtIO is used for NICs and Disk…

[Andy_Wismer (André Wismer)]

If I click on NS7 "Sync data* or “Finish Migration” (The initial sync ran without problems a day or two earlier), i get this on NS7 (both show the same error after a while):

Copy Command shows this:

 echo '{"app":"nethserver-nextcloud","action":"sync"}' | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/nethserver-ns8-migration/migration/update | jq

On NS8, the GUI shows this (faster than the error on NS7 appears):

With a similiar error message as in above screenshots…

---

As such, the error message seems partially correct, this has been migrated before, just never “finished”…

Just tested again…

My 2 cents
Andy

[Andy_Wismer (André Wismer)]

@mrmarkuz

I found the error “module already imported” once more in the forum, is it the same NS7?

This was a MUCH earlier attempt. Bioth VMs long since restored from Backup, NS8 recreated completly new.

The first time you refer to NS8 Migration would not finish the first time. Here, it did finish the first time.
Nov 23 and May 24 - NS8 is a different animal, and out of Beta!l…

NS7 has been working like a rock, solid as ever.

[mrmarkuz (Markus Neuberger)]

I asked because if it’s the same NS7 installation where Nextcloud migration isn’t working, then maybe there’s an issue with the NS7 nextcloud installation that causes the faulty migration.

Maybe it helps to check the logs from the first working sync if there was an issue?

Which NC apps are installed on NS7?

occ app:list

[Andy_Wismer (André Wismer)]

[root@suma-nethserver ~]# occ app:list
Enabled:

• activity: 2.19.0
• announcementcenter: 6.8.1
• apporder: 0.15.0
• approval: 1.1.1
• audioplayer: 3.4.1
• calendar: 4.6.7
• camerarawpreviews: 0.8.4
• checksum: 1.2.3
• circles: 27.0.1
• cloud_federation_api: 1.10.0
• collectives: 2.10.1
• comments: 1.17.0
• contacts: 5.5.3
• contactsinteraction: 1.8.0
• dashboard: 7.7.0
• dav: 1.27.0
• drawio: 2.1.4
• external: 5.2.1
• extract: 1.3.6
• federatedfilesharing: 1.17.0
• federation: 1.17.0
• files: 1.22.0
• files_automatedtagging: 1.17.0
• files_external: 1.19.0
• files_lock: 27.0.5
• files_pdfviewer: 2.8.0
• files_reminders: 1.0.0
• files_rightclick: 1.6.0
• files_sharing: 1.19.0
• files_trackdownloads: 1.11.0
• files_trashbin: 1.17.0
• files_versions: 1.20.0
• files_zip: 1.4.1
• firstrunwizard: 2.16.0
• flow_notifications: 1.7.0
• geoblocker: 0.5.13
• group_default_quota: 0.1.7
• groupfolders: 15.3.7
• groupquota: 0.1.13
• health: 2.2.2
• keeweb: 0.6.13
• login_notes: 1.3.1
• logreader: 2.12.0
• lookup_server_connector: 1.15.0
• music: 1.10.0
• nextcloud_announcements: 1.16.0
• notes: 4.9.4
• notifications: 2.15.0
• oauth2: 1.15.2
• password_policy: 1.17.0
• photos: 2.3.0
• privacy: 1.11.0
• provisioning_api: 1.17.0
• radio: 1.0.3
• recommendations: 1.6.0
• related_resources: 1.2.0
• richdocuments: 8.2.6
• serverinfo: 1.17.0
• settings: 1.9.0
• sharebymail: 1.17.0
• shifts: 1.9.7
• support: 1.10.0
• survey_client: 1.15.0
• systemtags: 1.17.0
• tasks: 0.15.0
• text: 3.8.0
• theming: 2.2.0
• timemanager: 0.3.13
• timetracker: 0.0.82
• twofactor_backupcodes: 1.16.0
• updatenotification: 1.17.0
• user_ldap: 1.17.0
• user_status: 1.7.0
• video_converter: 1.0.6
• viewer: 2.1.0
• weather_status: 1.7.0
• workflow_ocr: 1.27.5
• workflow_pdf_converter: 1.12.0
• workflow_script: 1.12.1
• workflowengine: 2.9.0
  Disabled:
• admin_audit: 1.17.0
• bruteforcesettings: 2.7.0
• encryption: 2.15.0
• suspicious_login: 5.0.0
• twofactor_totp: 9.0.0
  [root@suma-nethserver ~]#

I don’t really think these two tests can be compared. The first barfed due to Beta issues (Nov 2023). NS7 has never had any issues at this client.
This is a fresh install on NS8, and the initial migration of Nextcloud completed without issues. Here,even the mail migration is complete. That was not the case during the Beta…

My 2 cents
Andy

[Andy_Wismer (André Wismer)]

Latest attempt:

[root@suma-nethserver ~]#  echo '{"app":"nethserver-nextcloud","action":"finish","migrationConfig":{"virtualHost":"cloud.sumatronic.ch"}}' | /usr/bin/setsid /usr/bin/sudo /usr/libexec/nethserver/api/nethserver-ns8-migration/migration/update | jq
{
  "progress": "0.00",
  "time": "0.0",
  "exit": 0,
  "event": "migration-sync",
  "state": "running",
  "step": 0,
  "pid": 0,
  "action": ""
}
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
rsync: failed to connect to 10.98.33.1 (10.98.33.1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(126) [sender=3.1.2]
{
  "pid": 0,
  "status": "failed",
  "event": "migration-sync"
}
{
  "id": "1716332441",
  "type": "ApiFailed",
  "message": "finish nethserver-nextcloud failed"
}
[root@suma-nethserver ~]# 

[Andy_Wismer (André Wismer)]

Next steps:

• Aborted the migration of Nextcloud
• rebooted both VMs
• On NS7 went to NS8 Migration page
• Nextcloud was offered again for synching
• Reattempted a sync with Nextcloud.
• Intial sync went without issues
• Sync again, no issues.
• Finish Migration: On NS8 it stops at about 33%
• Same result as so far…

This screenshot shows the Nextcloud migration progressing to 33%, then it just hangs there…

[mrmarkuz (Markus Neuberger)]

OK, so it seems you are one step further as the sync is working now but there’s still an issue with finishing the migration.
Are there errors in the logs?

[Andy_Wismer (André Wismer)]

Hi @mrmarkuz

Actually, depending on vantage point, I am a little further or not.

I decided to migrate the Nextcloud by hand afterwards, as there are only about 12 user accounts with Calendar and Adressbook, but only a single user has files directly on NextCloud (For sharing with clients, a single folder / project).

So I migrated the rest finished (Mal was already done, just Files and AD missing since nextclouded was excluded. That took a whopping 14 hours, even though the NS8 is now running on a full NVME equipped Server with ample RAM.

Files are done. AD is now in the finishing moments, NS8 already shows Internal AD, NS7 also shows local AD, but that will change any moment now. Minutes, not hours!

Would have been nice, if Nextcloud could have been migrated, but as i recall from the Forums, the 33% was an issue I recall seeing VERY often (roughly a third) - then hangs…
I’m personally assuming this is a race condition or timing issue somewhere along the route. I’ld like to troubleshoot this, but my client needs to work too…

Others may start to whimper…

In a modification of what Asterix said to the Romans:

We’ve got the brains to plan this
we’ve got the brawn to pull this through
and we’ve got the b*lls to attempt this!