More logs
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070823623Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070847073Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070868924Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070889718Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070910817Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070932826Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T17:47:25+01:00 [1:loki1:loki-server] level=error ts=2025-03-24T16:47:25.070992582Z caller=retry.go:73 org_id=fake traceID=0b918a92a5e0b898 msg="error processing request" try=0 query="{node_id=~\".+\"} | json syslog_id=\"SYSLOG_IDENTIFIER\",message=\"MESSAGE\" | line_format \"[{{.node_id}}:{{.module_id}}:{{.syslog_id}}] {{.message}}\" |= \"backup\"" err="context canceled"
2025-03-24T18:01:24+01:00 [1::agent@cluster] task/cluster/bc1cc060-48e7-499d-9ace-a9ad0c953c89: list-backup-repositories/50list is starting
2025-03-24T18:01:25+01:00 [1::agent@cluster] task/cluster/bc1cc060-48e7-499d-9ace-a9ad0c953c89: action "list-backup-repositories" status is "completed" (0) at step validate-output.json
2025-03-24T18:01:25+01:00 [1::agent@cluster] task/cluster/493de8ee-f421-4f68-b4ef-9d465ef1370d: list-backups/50list is starting
2025-03-24T18:01:26+01:00 [1::agent@cluster] task/cluster/493de8ee-f421-4f68-b4ef-9d465ef1370d: action "list-backups" status is "completed" (0) at step validate-output.json
2025-03-24T18:01:41+01:00 [1::agent@cluster] task/cluster/fa13617f-6cb1-4488-8eb2-9a2c3e5c88a1: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:loki1:agent@loki1] task/module/loki1/dc4b9b62-d43f-45af-b4da-e8687b97a917: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:scratchpad1:agent@scratchpad1] task/module/scratchpad1/a2248cd0-4c57-4543-bb63-01b4345d3fcf: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:nextcloud8:agent@nextcloud8] task/module/nextcloud8/1e7b708a-b498-48a9-bb29-10057d450fa8: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:crowdsec1:agent@crowdsec1] task/module/crowdsec1/aa89bb28-c09a-4bb3-9f48-319a2bd13218: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:mail1:agent@mail1] task/module/mail1/866a05ba-2672-41a3-8f6e-7f3ad4dfb5aa: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:nextcloud10:agent@nextcloud10] task/module/nextcloud10/b6edb875-afbf-4d3d-a7d1-b49cab49156d: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:openldap1:agent@openldap1] task/module/openldap1/c66d008d-e408-4c14-b643-09acb31f708d: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:webserver1:agent@webserver1] task/module/webserver1/568c6d4f-3d0e-4286-b8bc-a9d20b301293: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:traefik1:agent@traefik1] task/module/traefik1/d844adc2-9d28-4456-a58e-4022589d9cf1: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:sogo2:agent@sogo2] task/module/sogo2/eb8c6048-af5f-472f-b727-b3a69a14cee5: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:bookstack1:agent@bookstack1] task/module/bookstack1/9863cb69-20bf-492f-9537-e97c1b037925: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:odoo1:agent@odoo1] task/module/odoo1/b951aa60-5c85-453a-a045-304aa0084932: run-backup/50run_backup is starting
2025-03-24T18:01:42+01:00 [1:mariadb1:agent@mariadb1] task/module/mariadb1/0b2a7a1a-d40d-449b-bdba-f87a0b0e013f: run-backup/50run_backup is starting
2025-03-24T18:01:46+01:00 [1:crowdsec1:agent@crowdsec1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:53+01:00 [1:scratchpad1:agent@scratchpad1] restic backup --json state/environment
2025-03-24T18:01:54+01:00 [1:webserver1:agent@webserver1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:55+01:00 [1:loki1:agent@loki1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:55+01:00 [1:traefik1:agent@traefik1] restic backup --json state/environment
2025-03-24T18:01:57+01:00 [1:openldap1:agent@openldap1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:58+01:00 [1:sogo2:agent@sogo2] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:58+01:00 [1:bookstack1:agent@bookstack1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:01:59+01:00 [1:mail1:agent@mail1] restic backup --json state/environment --files-from=/etc/state-include.conf --exclude-file=/etc/state-exclude.conf
2025-03-24T18:02:10+01:00 [1::agent@cluster] Handler of module/crowdsec1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:10+01:00 [1:crowdsec1:agent@crowdsec1] task/module/crowdsec1/aa89bb28-c09a-4bb3-9f48-319a2bd13218: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:10+01:00 [1::agent@cluster] task/cluster/beb4a860-9456-4b26-bfa7-6c570f7a1913: list-backups/50list is starting
2025-03-24T18:02:10+01:00 [1::agent@cluster] Handler of module/crowdsec1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:11+01:00 [1::agent@cluster] task/cluster/beb4a860-9456-4b26-bfa7-6c570f7a1913: action "list-backups" status is "completed" (0) at step validate-output.json
2025-03-24T18:02:11+01:00 [1:nextcloud10:agent@nextcloud10] restic backup --json state/environment state/ui_name.dump --files-from=/etc/state-include.conf
2025-03-24T18:02:12+01:00 [1:odoo1:agent@odoo1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:02:15+01:00 [1:nextcloud8:agent@nextcloud8] restic backup --json state/environment state/ui_name.dump --files-from=/etc/state-include.conf
2025-03-24T18:02:18+01:00 [1:mariadb1:agent@mariadb1] restic backup --json state/environment --files-from=/etc/state-include.conf
2025-03-24T18:02:19+01:00 [1::agent@cluster] Handler of module/scratchpad1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:20+01:00 [1::agent@cluster] Handler of module/scratchpad1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:21+01:00 [1:scratchpad1:agent@scratchpad1] task/module/scratchpad1/a2248cd0-4c57-4543-bb63-01b4345d3fcf: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:23+01:00 [1::agent@cluster] Handler of module/traefik1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:23+01:00 [1::agent@cluster] Handler of module/traefik1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:24+01:00 [1:traefik1:agent@traefik1] task/module/traefik1/d844adc2-9d28-4456-a58e-4022589d9cf1: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:28+01:00 [1::agent@cluster] Handler of module/sogo2/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:29+01:00 [1::agent@cluster] Handler of module/sogo2/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:29+01:00 [1::agent@cluster] Handler of module/openldap1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:29+01:00 [1::agent@cluster] Handler of module/openldap1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:30+01:00 [1:openldap1:agent@openldap1] task/module/openldap1/c66d008d-e408-4c14-b643-09acb31f708d: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:30+01:00 [1:sogo2:agent@sogo2] task/module/sogo2/eb8c6048-af5f-472f-b727-b3a69a14cee5: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:32+01:00 [1::agent@cluster] Handler of module/bookstack1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:33+01:00 [1::agent@cluster] Handler of module/bookstack1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:34+01:00 [1:bookstack1:agent@bookstack1] task/module/bookstack1/9863cb69-20bf-492f-9537-e97c1b037925: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:35+01:00 [1::agent@cluster] Handler of module/webserver1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:35+01:00 [1::agent@cluster] Handler of module/webserver1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:35+01:00 [1:webserver1:agent@webserver1] task/module/webserver1/568c6d4f-3d0e-4286-b8bc-a9d20b301293: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:38+01:00 [1::agent@cluster] Handler of module/loki1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:38+01:00 [1::agent@cluster] Handler of module/loki1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:39+01:00 [1:loki1:agent@loki1] task/module/loki1/dc4b9b62-d43f-45af-b4da-e8687b97a917: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:44+01:00 [1::agent@cluster] Handler of module/mariadb1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:44+01:00 [1::agent@cluster] Handler of module/mariadb1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:44+01:00 [1:mariadb1:agent@mariadb1] task/module/mariadb1/0b2a7a1a-d40d-449b-bdba-f87a0b0e013f: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:45+01:00 [1::agent@cluster] Handler of module/odoo1/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:45+01:00 [1::agent@cluster] Handler of module/odoo1/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:02:45+01:00 [1:odoo1:agent@odoo1] task/module/odoo1/b951aa60-5c85-453a-a045-304aa0084932: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:58+01:00 [1::agent@cluster] Handler of module/nextcloud10/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:02:59+01:00 [1:nextcloud10:agent@nextcloud10] task/module/nextcloud10/b6edb875-afbf-4d3d-a7d1-b49cab49156d: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:02:59+01:00 [1::agent@cluster] Handler of module/nextcloud10/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:03:09+01:00 [1::agent@cluster] Handler of module/nextcloud8/event/backup-status-changed is starting step 10node_monitor
2025-03-24T18:03:09+01:00 [1:nextcloud8:agent@nextcloud8] task/module/nextcloud8/1e7b708a-b498-48a9-bb29-10057d450fa8: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:03:09+01:00 [1::agent@cluster] Handler of module/nextcloud8/event/backup-status-changed exited with status "completed" (0) at step 10node_monitor
2025-03-24T18:03:21+01:00 [1:mail1:agent@mail1] task/module/mail1/866a05ba-2672-41a3-8f6e-7f3ad4dfb5aa: action "run-backup" status is "aborted" (1) at step 50run_backup
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/nextcloud8/run-backup run failed: {'output': 'Applying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags Reasons Paths\n-----------------------------------------------------------------------------------------------------\n582f1bb1 2025-03-23 07:47:49 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n\n234bb513 2025-03-23 23:00:39 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n\nd7da808c 2025-03-24 17:02:18 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n-----------------------------------------------------------------------------------------------------\n3 snapshots\n\n', 'error': 'restic snapshots\nRepository ce9fb2b8-d342-5aaa-b538-ada6b54553f7 is present at path nextcloud/233d8c4f-5af5-446d-bcb6-2d0811d0de49\nrestic backup --json state/environment state/ui_name.dump --files-from=/etc/state-include.conf\nrestic forget --prune --keep-last=5\nrestic stats --json latest\n2025/03/24 17:03:08 ERROR : 233d8c4f-5af5-446d-bcb6-2d0811d0de49.json: Failed to copy: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:03:08 ERROR : 233d8c4f-5af5-446d-bcb6-2d0811d0de49.json: Post request rcat error: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:03:08 Failed to rcat: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n', 'exit_code': 1}
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/nextcloud10/run-backup run failed: {'output': 'Applying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags Reasons Paths\n-----------------------------------------------------------------------------------------------------\ne8943da5 2025-03-23 07:48:23 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n\nc5acf474 2025-03-23 23:00:35 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n\n0b000a3f 2025-03-24 17:02:13 ns8 last snapshot /srv/state/config.json\n /srv/state/environment\n /srv/state/restore/dump.sql\n /srv/state/ui_name.dump\n /srv/volumes/nextcloud-app-data\n-----------------------------------------------------------------------------------------------------\n3 snapshots\n\n', 'error': 'restic snapshots\nRepository ce9fb2b8-d342-5aaa-b538-ada6b54553f7 is present at path nextcloud/6dc1d1d7-27c2-422d-8879-089a31f70540\nrestic backup --json state/environment state/ui_name.dump --files-from=/etc/state-include.conf\nrestic forget --prune --keep-last=5\nrestic stats --json latest\n2025/03/24 17:02:58 ERROR : 6dc1d1d7-27c2-422d-8879-089a31f70540.json: Failed to copy: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:58 ERROR : 6dc1d1d7-27c2-422d-8879-089a31f70540.json: Post request rcat error: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:58 Failed to rcat: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n', 'exit_code': 1}
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/mariadb1/run-backup run failed: {'output': "mkdir: created directory 'restore'\nApplying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags Reasons Paths\n---------------------------------------------------------------------------------------------------------\n4a87c818 2025-03-23 07:47:55 ns8 last snapshot /srv/state/environment\n /srv/state/restore/memos.sql\n /srv/state/restore/mysql.sql\n /srv/state/restore/phpmyadmin.sql\n /srv/state/restore/sys.sql\n /srv/state/restore/webtrees.sql\n /srv/state/secrets/passwords.secret\n /srv/volumes/config.user.inc.php\n /srv/volumes/mysql-conf.d\n\n26f2522b 2025-03-23 23:00:41 ns8 last snapshot /srv/state/environment\n /srv/state/restore/memos.sql\n /srv/state/restore/mysql.sql\n /srv/state/restore/phpmyadmin.sql\n /srv/state/restore/sys.sql\n /srv/state/restore/webtrees.sql\n /srv/state/secrets/passwords.secret\n /srv/volumes/config.user.inc.php\n /srv/volumes/mysql-conf.d\n\n7fd457a6 2025-03-24 17:02:21 ns8 last snapshot /srv/state/environment\n /srv/state/restore/memos.sql\n /srv/state/restore/mysql.sql\n /srv/state/restore/phpmyadmin.sql\n /srv/state/restore/sys.sql\n /srv/state/restore/webtrees.sql\n /srv/state/secrets/passwords.secret\n /srv/volumes/config.user.inc.php\n /srv/volumes/mysql-conf.d\n---------------------------------------------------------------------------------------------------------\n3 snapshots\n\nremoved 'restore/phpmyadmin.sql'\nremoved 'restore/memos.sql'\nremoved 'restore/sys.sql'\nremoved 'restore/webtrees.sql'\nremoved 'restore/mysql.sql'\nremoved directory 'restore/'\n", 'error': 'restic snapshots\nRepository ce9fb2b8-d342-5aaa-b538-ada6b54553f7 is present at path mariadb/1c8eaad9-7804-45b7-bb0d-b04fa545dc15\nrestic backup --json state/environment --files-from=/etc/state-include.conf\nrestic forget --prune --keep-last=5\nrestic stats --json latest\n2025/03/24 17:02:42 ERROR : 1c8eaad9-7804-45b7-bb0d-b04fa545dc15.json: Failed to copy: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:42 ERROR : 1c8eaad9-7804-45b7-bb0d-b04fa545dc15.json: Post request rcat error: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:42 Failed to rcat: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n', 'exit_code': 1}
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/sogo2/run-backup run failed: {'output': "Applying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags Reasons Paths\n--------------------------------------------------------------------------------------------\nde05ef40 2025-03-23 07:47:26 ns8 last snapshot /srv/state/backups\n /srv/state/environment\n /srv/state/sogo.sql\n /srv/state/templates\n\neb3c6398 2025-03-23 23:00:26 ns8 last snapshot /srv/state/backups\n /srv/state/environment\n /srv/state/sogo.sql\n /srv/state/templates\n\n5da7a0a3 2025-03-24 17:02:02 ns8 last snapshot /srv/state/backups\n /srv/state/environment\n /srv/state/sogo.sql\n /srv/state/templates\n--------------------------------------------------------------------------------------------\n3 snapshots\n\nremoved 'sogo.sql'\n", 'error': 'restic snapshots\nRepository ce9fb2b8-d342-5aaa-b538-ada6b54553f7 is present at path sogo/16cb1702-9994-434b-88bd-f82b3a18ad37\nrestic backup --json state/environment --files-from=/etc/state-include.conf\nrestic forget --prune --keep-last=5\nrestic stats --json latest\n2025/03/24 17:02:27 ERROR : 16cb1702-9994-434b-88bd-f82b3a18ad37.json: Failed to copy: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:27 ERROR : 16cb1702-9994-434b-88bd-f82b3a18ad37.json: Post request rcat error: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:27 Failed to rcat: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n', 'exit_code': 1}
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/traefik1/run-backup run failed: {'output': 'Applying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags Reasons Paths\n--------------------------------------------------------------------------------------------\n933bca77 2025-03-23 07:47:18 ns8 last snapshot /srv/state/environment\n54e45819 2025-03-23 23:00:21 ns8 last snapshot /srv/state/environment\n2213c8dc 2025-03-24 17:01:57 ns8 last snapshot /srv/state/environment\n--------------------------------------------------------------------------------------------\n3 snapshots\n\n', 'error': 'restic snapshots\nRepository ce9fb2b8-d342-5aaa-b538-ada6b54553f7 is present at path traefik/7ad86457-4b8b-465d-8ce1-6c9407fd0964\nrestic backup --json state/environment\nrestic forget --prune --keep-last=5\nrestic stats --json latest\n2025/03/24 17:02:21 ERROR : 7ad86457-4b8b-465d-8ce1-6c9407fd0964.json: Failed to copy: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:21 ERROR : 7ad86457-4b8b-465d-8ce1-6c9407fd0964.json: Post request rcat error: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n2025/03/24 17:02:21 Failed to rcat: failed to prepare upload: LocationConstraintConflict: The location constraint differs from the location you are trying to access. To avoid this error, please ensure the region parameter of your client matches the Hetzner location you are trying to access. For details, see https://docs.hetzner.com/cloud/general/locations/#what-locations-are-there\n\tstatus code: 400, request id: , host id: \n', 'exit_code': 1}
2025-03-24T18:03:22+01:00 [1::agent@cluster] Task module/crowdsec1/run-backup run failed: {'output': 'Applying Policy: keep 5 latest snapshots\nkeep 3 snapshots:\nID Time Host Tags
