NS8 Hermes Agent module

Stll0 · May 25, 2026, 3:07pm

yes, use hermes backup command inside container, it creates a .tar.gz, copy it in the new istance and hermes importit CLI Commands Reference | Hermes Agent

capote · May 25, 2026, 9:16pm

I did it an installed hermes-agent-2 from der Software Center-GUI
The following error occurred during agent registration:

Error: subpath: invalid mount option
Traceback (most recent call last):
  File "/home/hermes-agent2/.config/actions/configure-module/75seed-agent-home", line 157, in <module>
    seed_agent_home(agent_id, templates_path)
  File "/home/hermes-agent2/.config/actions/configure-module/75seed-agent-home", line 149, in seed_agent_home
    subprocess.run(command, check=True)
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['podman', 'run', '--rm', '--replace', '--name', 'hermes-agent-seed-1', '--network=none', '--user', 'hermes', '--entrypoint', '/bin/sh', '--env-file', '/home/hermes-agent2/.config/state/agents/1/agent.env', '--env', 'HERMES_HOME=/opt/data', '--mount', 'type=volume,src=hermes-agents-home,dst=/opt/data,subpath=1', '--volume', '/home/hermes-agent2/.config/templates:/templates:ro,z', 'ghcr.io/stell0/hermes-agent-hermes:0.4.1', '-eu', '-c', '\numask 077\nmkdir -p "$HERMES_HOME"\n\nensure_safe_target() {\n  target="$1"\n  if [ -L "$target" ]; then\n    rm -f "$target"\n  elif [ -e "$target" ] && [ ! -f "$target" ]; then\n    echo "unsafe seed target: $target" >&2\n    exit 1\n  fi\n}\n\nrender_template() {\n  template_path="$1"\n  target_path="$2"\n  role_mode="$3"\n  python3 - "$template_path" "$target_path" "$role_mode" <<\'PY\'\nimport os\nimport sys\nfrom pathlib import Path\n\ntemplate_path = Path(sys.argv[1])\ntarget_path = Path(sys.argv[2])\nrole_mode = sys.argv[3]\n\nif not template_path.is_file():\n    raise SystemExit(f"missing template: {template_path}")\n\nagent_role = os.environ["AGENT_ROLE"]\nreplacements = {\n    "__AGENT_ID__": os.environ["AGENT_ID"],\n    "__AGENT_NAME__": os.environ["AGENT_NAME"],\n    "__AGENT_ROLE__": agent_role.replace("_", " ") if role_mode == "title" else agent_role,\n}\n\ncontent = template_path.read_text(encoding="utf-8")\nfor placeholder, value in replacements.items():\n    content = content.replace(placeholder, value)\n\ntarget_path.write_text(content, encoding="utf-8")\nPY\n}\n\n# Seed files only when absent so operator edits inside the named volume survive\n# reconfiguration and service restarts.\nensure_safe_target "${HERMES_HOME}/SOUL.md"\nif [ ! -e "${HERMES_HOME}/SOUL.md" ]; then\n  render_template "/templates/SOUL/${AGENT_ROLE}.md.in" "${HERMES_HOME}/SOUL.md" title\nfi\n\nensure_safe_target "${HERMES_HOME}/.env"\nif [ ! -e "${HERMES_HOME}/.env" ]; then\n  render_template /templates/home.env.in "${HERMES_HOME}/.env" raw\nfi\n']' returned non-zero exit status 125.

AgentZero helped me sort out the problem. I hope AgentZero hasn’t made things worse.

Root Cause Analysis: subpath: invalid mount option for Hermes Agent (Nethserver 8)

The Problem

When creating a new agent via the NS8 web interface, the seed script 75seed-agent-home failed with the following error:
Error: subpath: invalid mount option
The container could not start because the Podman mount option subpath= was used.

Root Cause

Core issue: The node 192.168.3.21 (daho-ns8) runs Podman 4.3.1, which does not support the subpathmount option (known incompatibility).

Trigger: Both the systemd service unit hermes@.service and the action script 75seed-agent-home use Podman volumes with --mount type=volume,src=...,dst=...,subpath=%i by default to mount the agent home into the container image.

Module updates: After every Hermes module update, these files are reset to the faulty default version – the error therefore occurs repeatedly.

Additional pitfall (today): Our first correction created a backup file 75seed-agent-home.bak, which the NS8 framework accidentally executed instead of the patched original file. The .bak still contained the old subpathoption.

Subsequent error: After removing subpath and switching to bind mounts, a Permission denied occurred because the container user hermes could not write to the mounted host directory. The cause was UID mapping: the host UID of the directory did not match the effective UID expected by the container.

Solution – Step by Step

The following measures were executed on the host 192.168.3.21 via SSH (root, password XYZ).

Area Measure

Mount option in service unit Replaced --mount type=volume,src=hermes-agents-home,dst=/opt/data,subpath=%i with a bind mount -v /home/hermes-agentX/.local/share/containers/storage/volumes/hermes-agents-home/_data/%i:/opt/data

Mount option in seed script Replaced type=volume,src=...,subpath={agent_id} with type=bind,src=/home/hermes-agent2/.../_data/{agent_id},dst=... (hermes-agent2 only)

Remove backup file rm /home/hermes-agent2/.config/actions/configure-module/75seed-agent-home.bak – prevents NS8 from executing the old version

Correct write permissions chmod 777 on the agent directory _data/1 – ensures the container user can write (workaround for UID mapping issue)

Daemon reload runagent -m hermes-agentX systemctl --user daemon-reload – activates the changed service unit

Recreate the agent After the changes, create the agent again in the NS8 web interface; the seed phase now completes without errors.

Affected Files (hermes-agent2)

/home/hermes-agent2/.config/actions/configure-module/75seed-agent-home (line 141)

/home/hermes-agent2/.config/systemd/user/hermes@.service (ExecStart line)

/home/hermes-agent2/.local/share/containers/storage/volumes/hermes-agents-home/_data/1

Prevention and Repeat Protection

After each module update, the mount options must be patched again, as the update overwrites the files.

Long-term: Upgrading Podman to a version that supports subpath (e.g., 4.6+) would permanently resolve the issue.

The detailed guide is stored in hermes_agent_fix.promptinclude.md in the project directory.

Result

The agent was successfully created after these corrections. The dashboard is available again.

Stll0 · May 26, 2026, 6:20am

Oh well, that explain also the previous failure. Are you on Debian 12, right?
I’ve opened an issue

Stll0 · May 26, 2026, 10:34am

@capote I’m really sorry, but fixing it is a mess, i decided to not support debian 12 right now.
The architecture change and the subpath option has been introduced to fix the backup, because NS8 doesn’t allow to have dynamic volumes definition for backup. Removing it would be a problem rn.

BTW, you can keep using your istance with your workarund in systemd file. You can make it permanent creating a folder in your systemd directory with an override instead of directly changing the file (ask your bot to do it )

The best option would be of course using a newer podman

pike · May 26, 2026, 11:05am

So cross-distro striked. Again.

When?

Stll0 · May 26, 2026, 11:40am

So cross-distro striked. Again.

nope, just old debian 12, debian 13 still works I think, I haven’t tested.

I’m a Nethesis developer, but this isn’t a Nethesis supported application

That’s a personal project, not an official one, if you care about missing feature like older distro support you are very welcome to contribute.

capote · May 26, 2026, 11:49am

Yes. I must have overlooked a requirement such as Debian 13.

ou can keep using your istance with your workarund in systemd file. You can make it permanent creating a folder in your systemd directory with an override instead of directly changing the file (ask your bot to do it )

I will try it for fun.

The best option would be of course using a newer podman

I think the best thing to do is to upgrade to Debian 13. That should install the latest version of Podman, won’t it?

Stll0 · May 26, 2026, 11:55am

I think the best thing to do is to upgrade to Debian 13. That should install the latest version of Podman, won’t it?

Yes

Yes. I must have overlooked a requirement such as Debian 13.

You haven’t overlooked it, I didn’t knew it befor your issue, I added it later

capote · May 26, 2026, 12:34pm

Thats the solution:

Permanent Solution for Hermes Agent – systemd Drop-in Override

For hermes-agent2 the workaround has been turned into an update-safe, permanent configuration.

What was done?

Created directory:

/home/hermes-agent2/.config/systemd/user/hermes@.service.d/override.conf

2. Created override file:

/home/hermes-agent2/.config/systemd/user/hermes@.service.d/override.conf

Contents:

[Service]
ExecStart=
ExecStart=runagent /usr/bin/podman run \
    --name hermes-%i \
    --replace \
    --rm \
    --sdnotify=conmon \
    --cgroups=no-conmon \
    --pod hermes-pod-%i \
    --env-file %S/state/agents/%i/agent.env \
    --env-file %S/state/secrets/%i.env \
    --env API_SERVER_ENABLED=true \
    --env API_SERVER_HOST=127.0.0.1 \
    -v /home/hermes-agent2/.local/share/containers/storage/volumes/hermes-agents-home/_data/%i:/opt/data \
    --tz=${TIMEZONE} \
    ${HERMES_AGENT_HERMES_IMAGE} \
    hermes gateway run

The first empty ExecStart= clears the old value from the main unit; the second sets the corrected command with a bind mount instead of subpath.
Owner corrected: hermes-agent2:hermes-agent2
Daemon reload: runagent -m hermes-agent2 systemctl --user daemon-reload

Why is this update-safe?

NS8 module updates only overwrite the original unit file (hermes@.service) – not the files inside the override directory .d/.
The override configuration takes precedence over the unit file, so the faulty subpath mount will never become active again.

Verification

The command systemctl --user cat hermes@.service shows that both the original unit and the override are loaded. The override successfully replaces the ExecStart.

Note about hermes-agent1

The systemd directory for hermes-agent1 does not exist (presumably the module was not set up in the same way or has no systemd unit). Should a unit appear in the future, an identical override can be created following the same pattern.

Stll0 · May 27, 2026, 9:07am

OpenAI Codex (OAuth) is broken

If you are using OpenAI, you are experiencing this error

Non-retryable error (HTTP None): ‘NoneType’ object is not iterable

Its OpenAI fault, Hermes upstream already fixed it.

To workaround this, I made a module version that uses the Hermes main branch, that means that it isn’t a “stable” release, and something else is probably broken, but at least it isn’t “dead broken”

So if you are using OpenAI Codex, launch this command:

api-cli run update-module --data '{"module_url":"ghcr.io/stell0/hermes-agent:codex_fail","instances":["hermes-agent<ID>"], "force": true}'

And remember: you’ll need to update it again manually once next ns8 module release came out, it won’t update automatically

capote · May 27, 2026, 10:36am

root@daho-ns8:~# api-cli run update-module --data '{"module_url":"ghcr.io/stell0/hermes-agent:codex_fail","instances":["hermes-agent2"],"force":true}'

<7>podman pull ghcr.io/stell0/hermes-agent:codex_fail
Trying to pull ghcr.io/stell0/hermes-agent:codex_fail...
Getting image source signatures
Copying blob sha256:78657b78cbb94530f5261aa7d6df57379dbbfcc15eb6d6a3633248dddd578964
Copying config sha256:cc10b253eff7592dbf194d49c7a2aa52757e3edefbb7b37ce2b2c62b5d8fe5b5
Writing manifest to image destination
Storing signatures
cc10b253eff7592dbf194d49c7a2aa52757e3edefbb7b37ce2b2c62b5d8fe5b5
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)

<7>podman pull ghcr.io/stell0/hermes-agent:codex_fail
Trying to pull ghcr.io/stell0/hermes-agent:codex_fail...
Getting image source signatures
Copying blob sha256:78657b78cbb94530f5261aa7d6df57379dbbfcc15eb6d6a3633248dddd578964
Copying config sha256:cc10b253eff7592dbf194d49c7a2aa52757e3edefbb7b37ce2b2c62b5d8fe5b5
Writing manifest to image destination
Storing signatures
cc10b253eff7592dbf194d49c7a2aa52757e3edefbb7b37ce2b2c62b5d8fe5b5
<7>podman pull ghcr.io/stell0/hermes-agent-auth:codex_fail ghcr.io/stell0/hermes-agent-hermes:codex_fail ghcr.io/stell0/hermes-agent-socket:codex_fail
Trying to pull ghcr.io/stell0/hermes-agent-auth:codex_fail...
Getting image source signatures
Copying blob sha256:2f3cce5206698748d38ac29f25ba9d809290a69e3b1ffda2ca339ffaba972904
Copying blob sha256:730021980ca0688a57dfb7b98ed924eb12e3607817178d9278af400204e0e50d
Copying blob sha256:eff3576c9e7a6387c5f2d04c958711dec9791ee364ee1220d0eb92e92ab42b41
Copying blob sha256:c39513b5f21c259e4a903abcd304bf44b79bbb9fc9621ae2912f69d4f6a927c4
Copying blob sha256:7693d6ef93c9e7c58bb160c8fbe24814f4e49fcdee7b311b6522e018bc2e194d
Copying blob sha256:58d3257bdaafa16789addad906160fc5ee99b08b2a132c3a14db207d61fdd134
Copying config sha256:a792aea1b2d0ad01d1753376b13f5c8246683ec3245255670a1201926b0969ae
Writing manifest to image destination
Storing signatures
a792aea1b2d0ad01d1753376b13f5c8246683ec3245255670a1201926b0969ae
Trying to pull ghcr.io/stell0/hermes-agent-hermes:codex_fail...
Getting image source signatures
Copying blob sha256:1ae2393bb3a0564355e2c43b7a134cf2b236c9cb82e21dd07d975c9101e9450a
Copying blob sha256:7bb9595efdb32ccef5ed501c669ca95213d6df679d3368c2a43d2c06124152c3
Copying blob sha256:b265b1b803c75f6ebb8b2db35abb84c7f2f8a823924a7ada72d627bc3c0777b4
Copying blob sha256:639fcc2031a8da8f0ea313257edba4a6e331da029879e56938e91f69d537383f
Copying blob sha256:d53ba5e240cdd77c66373601c730fede76a7b7489085877efd3e7c2e6864ac2c
Copying blob sha256:9d7f34b7eab31726b8e9791bee9d4a575bdbf10003631e5e4df454692f4ec711
Copying blob sha256:d56d90f05c741f09aa539c110296613000b398c2a68d9cfc87997c988a1143ba
Copying blob sha256:984f49606fbb98a14b14cee2b52dbe949df2f7243179e4b42fbb02d1fb05c21a
Copying blob sha256:1f9c6407d725c2a55df54ec1d622744e53e96188a845dc541dd13a7649fcd6f7
Copying blob sha256:1a8379bbcdded745ac752a2d2b8f08dd10df68af1253be6de7b95ce1eca36029
Copying blob sha256:cf5dcf0f3e029c3357bc009e24a8e3105434d9c4007e31f36fbaf1bd927cf223
Copying blob sha256:5e981c9ee23759d5a69f89d8a0d766be739d5c7362aa04e8675fa63e1ed3ad7d
Copying blob sha256:cd0b1b194d615eb9f188893dafc59f301999b9be4e74cfe800605e4ed9df82b1
Copying blob sha256:f1e3473ae847c683908b0a9fc823a2d9dab68da5e8a5e255703a69171400d8e1
Copying blob sha256:4a03604c4e78a2287e548e6cac1bd57b286203ee4dd4538934c0df2ed198d6b7
Copying blob sha256:4d791f04f5fbe0a99c2c76ef2c7bf86368700de380de9e523cd8ea03f61fdbac
Copying blob sha256:ff59184da5aab4f855b3c1a73adb24404ee1c87b32254f6a7f57d75a3bc0a8af
Copying blob sha256:9b15667b72c382331a8c5e823ff3d9cae4298b297740c831c47212cbe5aebec6
Copying blob sha256:8db300bcbc93674f37bf710f0af810defd6904124857783a8ea005a29b5d8822
Copying blob sha256:6feae010156a56f810f27a3c2275e55fe18f49818fe4d3b57af0c9ccf163e71f
Copying blob sha256:836c70d1a4066236b3f52360559c9699bc9ab7216926bf7ac3130d2eb4410e7f
Copying blob sha256:2f1e75a04c6c4b70d7e734bcda5dc0b1003d66ae7437484b5c35a0516a09b3e2
Copying blob sha256:58f8ee72e327c0815d6263976c6c40c28b148174d8b602bc8fe09f7bc2a525b9
Copying blob sha256:f67d4abac231dd512b2224e19bc8e671c1e5283d17b45c1e27005e1493125a9d
Copying blob sha256:8a6264b293459817db1a234ff354449503cc293964be93eddbfea1469e56a1fd
Copying blob sha256:fb87f53c830facb8348a1517423638a02323b398be644b8c3724dfa6ac8be66f
Copying blob sha256:bb7aaeca3467b740cbfdf3e1047c906733ae9ea29a269d82c1de48532c7d1540
Copying blob sha256:cf1481c2a7b2de63a616044efd7de121294cd2bb0326ace0a3c4f07fb86bfb8d
Copying blob sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d
Copying blob sha256:397bab5e440f882cf9bcd34c6d0f4b2ad936b291b123c99dd274dab9944a75ad
Copying blob sha256:43bef0566cdac2381aba73f57e7ef9f64126b5cc464ee3d20f61012b4fb5d81f
Copying blob sha256:6f663b99325ced87898d9befefc6e4a07df063b7660437d7573714f87621f934
Copying blob sha256:15da08c1746058f04c40d577948ab255778f710e3dc37ca3b98a67b9cde3d58d
Copying blob sha256:ac613b73788c4867a4ad192ccbf2e032c0c108799fa4392595c6a6d2b10b3449
Copying blob sha256:c6475ff7af724f0b346d7b055419bce3d933f42524df5411976d7aa133bd0855
Copying blob sha256:a7132d0b40047684f538c0f30696e5646600484066d879b728a65eda47348798
Copying config sha256:f417f7eaa86103d6de1d465c2568613459e0a7e1c70f8cc9bb67c76600932474
Writing manifest to image destination
Storing signatures
f417f7eaa86103d6de1d465c2568613459e0a7e1c70f8cc9bb67c76600932474
Trying to pull ghcr.io/stell0/hermes-agent-socket:codex_fail...
Getting image source signatures
Copying blob sha256:4a9fdf739a4f3be2cb6220d1cbdba9dcfb888b78640ff644f8f205fb7df3373a
Copying blob sha256:55cfd63653c76188760d313211122850525467c0d3c469111f3ba8f64898d7f2
Copying config sha256:cc0256f3894dd882de09510bdfa8647c8539730e801207a5af7e0ebf496f70f7
Writing manifest to image destination
Storing signatures
cc0256f3894dd882de09510bdfa8647c8539730e801207a5af7e0ebf496f70f7
<7>extract-image ghcr.io/stell0/hermes-agent:codex_fail
flock: getting lock took 0.000008 seconds
'.imageroot.lst' -> '.imageroot.lst~'
Extracting container filesystem imageroot to /home/hermes-agent2/.config
Gesamtzahl gelesener Bytes: 9256960 (8,9MiB, 48MiB/s)
./AGENTS.md
./actions/
./actions/configure-module/
./actions/configure-module/10validate-input
./actions/configure-module/20persist-shared-env
./actions/configure-module/25configure-user-domain
./actions/configure-module/30remove-deleted-routes
./actions/configure-module/40remove-deleted-agents
./actions/configure-module/50write-agent-metadata
./actions/configure-module/60refresh-shared-settings
./actions/configure-module/70sync-agent-runtime
./actions/configure-module/75seed-agent-home
./actions/configure-module/80reload-systemd
./actions/configure-module/90reconcile-desired-routes
./actions/configure-module/95reconcile-agent-services
./actions/configure-module/validate-input.json
./actions/create-module/
./actions/create-module/05check-podman-version
./actions/create-module/10initialize-state
./actions/create-module/20discover-smarthost
./actions/destroy-module/
./actions/destroy-module/10remove-routes
./actions/destroy-module/20stop-services
./actions/destroy-module/30remove-agent-state
./actions/destroy-module/40remove-agents-root
./actions/get-agent-runtime/
./actions/get-agent-runtime/10read
./actions/get-agent-runtime/validate-output.json
./actions/get-configuration/
./actions/get-configuration/20read
./actions/get-configuration/validate-output.json
./actions/list-domain-users/
./actions/list-domain-users/10read
./actions/list-domain-users/validate-input.json
./actions/list-domain-users/validate-output.json
./actions/list-user-domains/
./actions/list-user-domains/10read
./actions/list-user-domains/validate-output.json
./actions/restore-module/
./actions/restore-module/06copyenv
./actions/restore-module/20configure
./bin/
./bin/discover-smarthost
./bin/ensure-agent-home-ownership
./bin/remove-agent-state
./bin/sync-agent-runtime
./etc/
./etc/state-include.conf
./events/
./events/smarthost-changed/
./events/smarthost-changed/10reload_services
./pypkg/
./pypkg/hermes_agent_state.py
./pypkg/hermes_user_domain.py
./systemd/
./systemd/user/
./systemd/user/hermes-auth.service
./systemd/user/hermes-pod@.service
./systemd/user/hermes-socket@.service
./systemd/user/hermes@.service
./templates/
./templates/SOUL/
./templates/SOUL/business_consultant.md.in
./templates/SOUL/customer_support.md.in
./templates/SOUL/default.md.in
./templates/SOUL/developer.md.in
./templates/SOUL/marketing.md.in
./templates/SOUL/researcher.md.in
./templates/SOUL/sales.md.in
./templates/SOUL/social_media_manager.md.in
./templates/home.env.in
./update-module.d/
./update-module.d/15migrate-agents-volume
./update-module.d/20migrate-secrets-dir
./update-module.d/25migrate-agent-env
./update-module.d/30ensure-agent-home-ownership
./update-module.d/80restart
chown: der Eigentümer von './systemd/user/hermes@.service.bak' wird geändert: Die Operation ist nicht erlaubt
chown: der Eigentümer von './systemd/user/hermes@.service.d' wird geändert: Die Operation ist nicht erlaubt
67b862265d6a1cc2fb74e8eb702dc9555388742637e1756fb63a5ae6352541d5
Traceback (most recent call last):
  File "/usr/local/agent/actions/update-module/05pullimages", line 81, in <module>
    ).check_returncode()
      ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 502, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '('extract-image', 'ghcr.io/stell0/hermes-agent:codex_fail')' returned non-zero exit status 1.

no chance to check it out

Stll0 · May 27, 2026, 11:53am

capote:

chown: der Eigentümer von './systemd/user/hermes@.service.bak' wird geändert: Die Operation ist nicht erlaubt
chown: der Eigentümer von './systemd/user/hermes@.service.d' wird geändert: Die Operation ist nicht erlaubt

Issue is the permissions of those files, change permission and selinux as the other files in the folder

capote · June 3, 2026, 7:25pm

the last update fails agian.

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
Trying to pull ghcr.io/stell0/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:000f68c1e5c86d2501cde901836af4c56a6f9c6e53c0856a6b1c9a3ddb7ff4ed
Copying config sha256:e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
Writing manifest to image destination
e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
time="2026-06-03T21:11:17+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-03T21:11:17+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:000f68c1e5c86d2501cde901836af4c56a6f9c6e53c0856a6b1c9a3ddb7ff4ed
Copying config sha256:e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
Writing manifest to image destination
e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
time="2026-06-03T21:11:19+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
<7>podman-pull-missing ghcr.io/stell0/hermes-agent-auth:0.5.0 ghcr.io/stell0/hermes-agent-hermes:0.5.0 ghcr.io/stell0/hermes-agent-socket:0.5.0
time="2026-06-03T21:11:19+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-03T21:11:19+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent-auth:0.5.0...
Getting image source signatures
Copying blob sha256:03a2b92e3c59854a821233c1f0578be46c031343ac3de30b3b8307e03bf3b11c
Copying blob sha256:7693d6ef93c9e7c58bb160c8fbe24814f4e49fcdee7b311b6522e018bc2e194d
Copying blob sha256:58d3257bdaafa16789addad906160fc5ee99b08b2a132c3a14db207d61fdd134
Copying blob sha256:eff3576c9e7a6387c5f2d04c958711dec9791ee364ee1220d0eb92e92ab42b41
Copying blob sha256:61acf16df8863c414ee8d73eacb8302c595079477985958bdafc51265d2190b9
Copying blob sha256:c39513b5f21c259e4a903abcd304bf44b79bbb9fc9621ae2912f69d4f6a927c4
Copying config sha256:1cd823de6b50c808e762bea27d8e990904e973e4c31ec6b2ab93859b278af1fa
Writing manifest to image destination
1cd823de6b50c808e762bea27d8e990904e973e4c31ec6b2ab93859b278af1fa
time="2026-06-03T21:11:26+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-03T21:11:26+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent-hermes:0.5.0...
Getting image source signatures
Copying blob sha256:d2bd5614174922ca1c3d05b0094003e7ba3ad0e93f0043d2445f6c81ca701111
Copying blob sha256:eac402f3d6e7c106f4665ceb755ac7c3bf6fde2940f09ab37a94a725eaddba97
Copying blob sha256:639fcc2031a8da8f0ea313257edba4a6e331da029879e56938e91f69d537383f
Copying blob sha256:ff4e86d7e8878019880fd902bc7b8f22aaa0320d488c69dd695bfc2119c963bb
Copying blob sha256:f79bf00cf5d9324f5aea1dff54286dd378728f831b51fc74285466af6871f8d8
Copying blob sha256:4f72dae80ae1a451655c36c281148022352a928e1903e2f6acb09a278e1cb106
Copying blob sha256:4ca7cb1011b0c011128a600d9365086d2cdc5b3f1614e5f7eb0e083818e4977a
Copying blob sha256:347b71c8122a4800e80911d7b7febdccfb3204f37a44a7a6ce7dcedf5e567b41
Copying blob sha256:c8ba6d97ebb1dd7fa37515afb0f846b9eaf3c77cd183f6f3db9a00466e56078b
Copying blob sha256:8332ec5f114be92c71965b19463ee8a2a375af28f0e4a2b64fc1be4c222b9460
Copying blob sha256:eeaa3c7bee18e8470fd3b0febb437731931b3c7e399fff7f902ac20a55c7801f
Copying blob sha256:5c9ba6501fc0efdfe832e26f3ad0c7f6d7dabc8e8caf85c06879ed30fd1c928c
Copying blob sha256:4659ea2b98f42cb8c6355b070b61be95ac88fe08c06b964e1c93f11af0890a8b
Copying blob sha256:82de33f1790d7151ca5f85b1dd62a2a81a6054c706f1b4c1b95128a60ac4039e
Copying blob sha256:81648dde7d7cf74a6e23f5252c6be563cb240fcec0a08c9d5bdd8587be27be87
Copying blob sha256:169e9eafb0f3553ec577baf232920717f65d39d00ff67ec5badd750dfee93beb
Copying blob sha256:4cb266eade751aa098433da2efc013ecc95f0f9cc7aa986dbc6ca366d2742b54
Copying blob sha256:31650fd3070ed1ad37ffdedf33c796f282f61505fbaa082324724ed86203d139
Copying blob sha256:8884c19c9800f7c086fe0a55a15f49b84123fa4394da795210c4d22acc21d9f8
Copying blob sha256:12ac73398190bf5a549fe32b6b2d1329aeec15fb8d6ee7878c506373751135d6
Copying blob sha256:592bc7889dc1658c867bf36374504dd14902d015ee99027d8abcedf6900fb7d0
Copying blob sha256:97e8c141886cc189f905b1c0e92f492b6a5f381ced63e1636b80ceee2188d96b
Copying blob sha256:1979ae7209ff184e570b13e17cb83e791a7cf9f5f390590722f119ecaab73e91
Copying blob sha256:0d9111c063985e0278a68fad756fe1f38c982ec1d5936bba3471627a39c3588b
Copying blob sha256:2ceb3523d4a53783cf419fba02945a5509095ace62f47b89b1dbe78243fa6bb4
Copying blob sha256:eda0390fe60ac2315fb94e1a4e25cdbaeea6d0c833c18c044b49393988ed534f
Copying blob sha256:14e20a0d01abfb578c5a717b1d7e9b06b04eaceacd437da555724eadb8744ed0
Copying blob sha256:3ea9158c6c5fe7e394dae44fa0f1ac327d6b16eaebc4aa04f04e50535efb70fe
Copying blob sha256:543676d1c7244781a0cd8c7567b8a30ea244fe041a28b780d272401e8807e652
Copying blob sha256:b79ac18e4a3c85e3afae30750dd2cf27921ad78775232414cfbafde539c4590e
Copying blob sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d
Copying blob sha256:a63c7ef64d8f4c001cd0f7aba0b8fea0553ee12dbdc2ca41df9ec9bc150e43b4
Copying blob sha256:0c2ec76d74426c9b940fba300d7b2f74f1df72bf7a89d9f49ef47afc2b8e3597
Copying blob sha256:2069b53cf78194be5daf0fa0ad6d409b9f80947624e6adc77d964f1d66fdb47d
Copying blob sha256:38c97070ee82bef174398b818cc15423d34f6534ebc401da66bffa3e160f7174
Copying blob sha256:c50d34a241546bf147241887ed70a8427837db9d9e46d179938c7b69f85727e5
Copying blob sha256:bae7819e3a82a61645b9335abe61680f48e5b6692f63fbb84770c371d5a8a6ea
Copying blob sha256:683fae117541b37af0fb7b685a0d7c6c8601523271c14d50fb3823f725694ec9
Error: writing blob: adding layer with blob "sha256:543676d1c7244781a0cd8c7567b8a30ea244fe041a28b780d272401e8807e652"/""/"sha256:8a5d05352f6dd27a862e266748b9f314cef874ca2de83d1cdf15ea549d8026c1": creating read-only layer with ID "7a24113266f3d941bd852820fc4d7823de60c13d67d2f9ba3e108c7d89e47f8d": open /home/hermes-agent2/.local/share/containers/storage/vfs/dir/7a24113266f3d941bd852820fc4d7823de60c13d67d2f9ba3e108c7d89e47f8d/root/.cache/uv/archive-v0/RTUqHqZPgJ434yMvTYfqS/anthropic/types/content_block_source_param.py: no space left on device
Traceback (most recent call last):
  File "/usr/local/agent/bin/podman-pull-missing", line 35, in <module>
    subprocess.run(['podman', 'pull', image_url]).check_returncode()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
                             self.stderr)
subprocess.CalledProcessError: Command '['podman', 'pull', 'ghcr.io/stell0/hermes-agent-hermes:0.5.0']' returned non-zero exit status 125.
Traceback (most recent call last):
  File "/usr/local/agent/actions/update-module/05pullimages", line 71, in <module>
    ).check_returncode()
      ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
                             self.stderr)
subprocess.CalledProcessError: Command '('podman-pull-missing', 'ghcr.io/stell0/hermes-agent-auth:0.5.0', 'ghcr.io/stell0/hermes-agent-hermes:0.5.0', 'ghcr.io/stell0/hermes-agent-socket:0.5.0')' returned non-zero exit status 1.

I’m using Debian 13 and had previously newly installed Hermes.

capote · June 3, 2026, 11:21pm

I increased the disk size.
now I got the error:

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
Trying to pull ghcr.io/stell0/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:000f68c1e5c86d2501cde901836af4c56a6f9c6e53c0856a6b1c9a3ddb7ff4ed
Copying config sha256:e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
Writing manifest to image destination
e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
time="2026-06-04T01:00:34+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T01:00:34+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:000f68c1e5c86d2501cde901836af4c56a6f9c6e53c0856a6b1c9a3ddb7ff4ed
Copying config sha256:e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
Writing manifest to image destination
e61fd21ccda85d36846ad484b93238a9bc3a8e1d4b6b93bd1d8cb1749c6fe5b8
time="2026-06-04T01:00:45+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
<7>podman-pull-missing ghcr.io/stell0/hermes-agent-auth:0.5.0 ghcr.io/stell0/hermes-agent-hermes:0.5.0 ghcr.io/stell0/hermes-agent-socket:0.5.0
time="2026-06-04T01:00:45+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T01:00:45+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent-auth:0.5.0...
Getting image source signatures
Copying blob sha256:7693d6ef93c9e7c58bb160c8fbe24814f4e49fcdee7b311b6522e018bc2e194d
Copying blob sha256:03a2b92e3c59854a821233c1f0578be46c031343ac3de30b3b8307e03bf3b11c
Copying blob sha256:eff3576c9e7a6387c5f2d04c958711dec9791ee364ee1220d0eb92e92ab42b41
Copying blob sha256:58d3257bdaafa16789addad906160fc5ee99b08b2a132c3a14db207d61fdd134
Copying blob sha256:61acf16df8863c414ee8d73eacb8302c595079477985958bdafc51265d2190b9
Copying blob sha256:c39513b5f21c259e4a903abcd304bf44b79bbb9fc9621ae2912f69d4f6a927c4
Copying config sha256:1cd823de6b50c808e762bea27d8e990904e973e4c31ec6b2ab93859b278af1fa
Writing manifest to image destination
1cd823de6b50c808e762bea27d8e990904e973e4c31ec6b2ab93859b278af1fa
time="2026-06-04T01:00:52+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T01:00:52+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent-hermes:0.5.0...
Getting image source signatures
Copying blob sha256:d2bd5614174922ca1c3d05b0094003e7ba3ad0e93f0043d2445f6c81ca701111
Copying blob sha256:eac402f3d6e7c106f4665ceb755ac7c3bf6fde2940f09ab37a94a725eaddba97
Copying blob sha256:639fcc2031a8da8f0ea313257edba4a6e331da029879e56938e91f69d537383f
Copying blob sha256:ff4e86d7e8878019880fd902bc7b8f22aaa0320d488c69dd695bfc2119c963bb
Copying blob sha256:4f72dae80ae1a451655c36c281148022352a928e1903e2f6acb09a278e1cb106
Copying blob sha256:f79bf00cf5d9324f5aea1dff54286dd378728f831b51fc74285466af6871f8d8
Copying blob sha256:4ca7cb1011b0c011128a600d9365086d2cdc5b3f1614e5f7eb0e083818e4977a
Copying blob sha256:347b71c8122a4800e80911d7b7febdccfb3204f37a44a7a6ce7dcedf5e567b41
Copying blob sha256:c8ba6d97ebb1dd7fa37515afb0f846b9eaf3c77cd183f6f3db9a00466e56078b
Copying blob sha256:8332ec5f114be92c71965b19463ee8a2a375af28f0e4a2b64fc1be4c222b9460
Copying blob sha256:eeaa3c7bee18e8470fd3b0febb437731931b3c7e399fff7f902ac20a55c7801f
Copying blob sha256:5c9ba6501fc0efdfe832e26f3ad0c7f6d7dabc8e8caf85c06879ed30fd1c928c
Copying blob sha256:4659ea2b98f42cb8c6355b070b61be95ac88fe08c06b964e1c93f11af0890a8b
Copying blob sha256:82de33f1790d7151ca5f85b1dd62a2a81a6054c706f1b4c1b95128a60ac4039e
Copying blob sha256:81648dde7d7cf74a6e23f5252c6be563cb240fcec0a08c9d5bdd8587be27be87
Copying blob sha256:169e9eafb0f3553ec577baf232920717f65d39d00ff67ec5badd750dfee93beb
Copying blob sha256:4cb266eade751aa098433da2efc013ecc95f0f9cc7aa986dbc6ca366d2742b54
Copying blob sha256:31650fd3070ed1ad37ffdedf33c796f282f61505fbaa082324724ed86203d139
Copying blob sha256:8884c19c9800f7c086fe0a55a15f49b84123fa4394da795210c4d22acc21d9f8
Copying blob sha256:12ac73398190bf5a549fe32b6b2d1329aeec15fb8d6ee7878c506373751135d6
Copying blob sha256:592bc7889dc1658c867bf36374504dd14902d015ee99027d8abcedf6900fb7d0
Copying blob sha256:97e8c141886cc189f905b1c0e92f492b6a5f381ced63e1636b80ceee2188d96b
Copying blob sha256:1979ae7209ff184e570b13e17cb83e791a7cf9f5f390590722f119ecaab73e91
Copying blob sha256:0d9111c063985e0278a68fad756fe1f38c982ec1d5936bba3471627a39c3588b
Copying blob sha256:2ceb3523d4a53783cf419fba02945a5509095ace62f47b89b1dbe78243fa6bb4
Copying blob sha256:eda0390fe60ac2315fb94e1a4e25cdbaeea6d0c833c18c044b49393988ed534f
Copying blob sha256:14e20a0d01abfb578c5a717b1d7e9b06b04eaceacd437da555724eadb8744ed0
Copying blob sha256:3ea9158c6c5fe7e394dae44fa0f1ac327d6b16eaebc4aa04f04e50535efb70fe
Copying blob sha256:543676d1c7244781a0cd8c7567b8a30ea244fe041a28b780d272401e8807e652
Copying blob sha256:b79ac18e4a3c85e3afae30750dd2cf27921ad78775232414cfbafde539c4590e
Copying blob sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d
Copying blob sha256:a63c7ef64d8f4c001cd0f7aba0b8fea0553ee12dbdc2ca41df9ec9bc150e43b4
Copying blob sha256:0c2ec76d74426c9b940fba300d7b2f74f1df72bf7a89d9f49ef47afc2b8e3597
Copying blob sha256:2069b53cf78194be5daf0fa0ad6d409b9f80947624e6adc77d964f1d66fdb47d
Copying blob sha256:38c97070ee82bef174398b818cc15423d34f6534ebc401da66bffa3e160f7174
Copying blob sha256:c50d34a241546bf147241887ed70a8427837db9d9e46d179938c7b69f85727e5
Copying blob sha256:bae7819e3a82a61645b9335abe61680f48e5b6692f63fbb84770c371d5a8a6ea
Copying blob sha256:683fae117541b37af0fb7b685a0d7c6c8601523271c14d50fb3823f725694ec9
Copying config sha256:a5f8ffde5c4d8c24bbedbc4e0370a2ce3428f1d7e13a141992c9417dde8d1085
Writing manifest to image destination
a5f8ffde5c4d8c24bbedbc4e0370a2ce3428f1d7e13a141992c9417dde8d1085
time="2026-06-04T01:13:04+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T01:13:04+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Trying to pull ghcr.io/stell0/hermes-agent-socket:0.5.0...
Getting image source signatures
Copying blob sha256:aef8a58af62aa00ccbfe761d01362fdb934b9b72c7ee6f904a406bdc6060a893
Copying blob sha256:55cfd63653c76188760d313211122850525467c0d3c469111f3ba8f64898d7f2
Copying config sha256:f543bab511bc1c5648dbb0575a6ca415bb18a6d777db1aa3cdd3d2b4fb20a996
Writing manifest to image destination
f543bab511bc1c5648dbb0575a6ca415bb18a6d777db1aa3cdd3d2b4fb20a996
<7>extract-image ghcr.io/stell0/hermes-agent:0.5.0
flock: getting lock took 0.000006 seconds
time="2026-06-04T01:13:06+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
'.imageroot.lst' -> '.imageroot.lst~'
time="2026-06-04T01:13:06+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Extracting container filesystem imageroot to /home/hermes-agent2/.config
time="2026-06-04T01:13:06+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Gesamtzahl gelesener Bytes: 9256960 (8,9MiB, 39MiB/s)
./AGENTS.md
./actions/
./actions/configure-module/
./actions/configure-module/10validate-input
./actions/configure-module/20persist-shared-env
./actions/configure-module/25configure-user-domain
./actions/configure-module/30remove-deleted-routes
./actions/configure-module/40remove-deleted-agents
./actions/configure-module/50write-agent-metadata
./actions/configure-module/60refresh-shared-settings
./actions/configure-module/70sync-agent-runtime
./actions/configure-module/75seed-agent-home
./actions/configure-module/80reload-systemd
./actions/configure-module/90reconcile-desired-routes
./actions/configure-module/95reconcile-agent-services
./actions/configure-module/validate-input.json
./actions/create-module/
./actions/create-module/05check-podman-version
./actions/create-module/10initialize-state
./actions/create-module/20discover-smarthost
./actions/destroy-module/
./actions/destroy-module/10remove-routes
./actions/destroy-module/20stop-services
./actions/destroy-module/30remove-agent-state
./actions/destroy-module/40remove-agents-root
./actions/get-agent-runtime/
./actions/get-agent-runtime/10read
./actions/get-agent-runtime/validate-output.json
./actions/get-configuration/
./actions/get-configuration/20read
./actions/get-configuration/validate-output.json
./actions/list-domain-users/
./actions/list-domain-users/10read
./actions/list-domain-users/validate-input.json
./actions/list-domain-users/validate-output.json
./actions/list-user-domains/
./actions/list-user-domains/10read
./actions/list-user-domains/validate-output.json
./actions/restore-module/
./actions/restore-module/06copyenv
./actions/restore-module/20configure
./bin/
./bin/discover-smarthost
./bin/ensure-agent-home-ownership
./bin/remove-agent-state
./bin/sync-agent-runtime
./etc/
./etc/state-include.conf
./events/
./events/smarthost-changed/
./events/smarthost-changed/10reload_services
./pypkg/
./pypkg/hermes_agent_state.py
./pypkg/hermes_user_domain.py
./systemd/
./systemd/user/
./systemd/user/hermes-auth.service
./systemd/user/hermes-pod@.service
./systemd/user/hermes-socket@.service
./systemd/user/hermes@.service
./templates/
./templates/SOUL/
./templates/SOUL/business_consultant.md.in
./templates/SOUL/customer_support.md.in
./templates/SOUL/default.md.in
./templates/SOUL/developer.md.in
./templates/SOUL/marketing.md.in
./templates/SOUL/researcher.md.in
./templates/SOUL/sales.md.in
./templates/SOUL/social_media_manager.md.in
./templates/home.env.in
./update-module.d/
./update-module.d/15migrate-agents-volume
./update-module.d/20migrate-secrets-dir
./update-module.d/25migrate-agent-env
./update-module.d/30ensure-agent-home-ownership
./update-module.d/80restart
chown: der Eigentümer von './systemd/user/hermes@.service.bak' wird geändert: Die Operation ist nicht erlaubt
chown: der Eigentümer von './systemd/user/hermes@.service.d' wird geändert: Die Operation ist nicht erlaubt
time="2026-06-04T01:13:07+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
2ff2d5cef96b5be7159263ef7e64f2f5605be4e94cdad23c1ba4e55718d51962
Traceback (most recent call last):
  File "/usr/local/agent/actions/update-module/05pullimages", line 81, in <module>
    ).check_returncode()
      ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
                             self.stderr)
subprocess.CalledProcessError: Command '('extract-image', 'ghcr.io/stell0/hermes-agent:0.5.0')' returned non-zero exit status 1.

Now, though, I’m at a loss

Stll0 · June 4, 2026, 5:36am

@capote New home for the module: GitHub - NethServer/ns8-hermes-agent: Deploy and manage Hermes-Agent(s) on NethServer 8 · GitHub

Change “stell0” with “nethserver” in your command. I haven’t said anything yet because migration is not complete, by today it should all be done.

capote · June 4, 2026, 7:59am

I didn’t really understand what you meant, as I initiated the update via the GUI.

I will try it again.

Same again:

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)

<7>podman-pull-missing ghcr.io/stell0/hermes-agent:0.5.0
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
<7>podman-pull-missing ghcr.io/stell0/hermes-agent-auth:0.5.0 ghcr.io/stell0/hermes-agent-hermes:0.5.0 ghcr.io/stell0/hermes-agent-socket:0.5.0
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
<7>extract-image ghcr.io/stell0/hermes-agent:0.5.0
flock: getting lock took 0.000008 seconds
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
'.imageroot.lst' -> '.imageroot.lst~'
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Extracting container filesystem imageroot to /home/hermes-agent2/.config
time="2026-06-04T09:59:28+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Gesamtzahl gelesener Bytes: 9256960 (8,9MiB, 46MiB/s)
./AGENTS.md
./actions/
./actions/configure-module/
./actions/configure-module/10validate-input
./actions/configure-module/20persist-shared-env
./actions/configure-module/25configure-user-domain
./actions/configure-module/30remove-deleted-routes
./actions/configure-module/40remove-deleted-agents
./actions/configure-module/50write-agent-metadata
./actions/configure-module/60refresh-shared-settings
./actions/configure-module/70sync-agent-runtime
./actions/configure-module/75seed-agent-home
./actions/configure-module/80reload-systemd
./actions/configure-module/90reconcile-desired-routes
./actions/configure-module/95reconcile-agent-services
./actions/configure-module/validate-input.json
./actions/create-module/
./actions/create-module/05check-podman-version
./actions/create-module/10initialize-state
./actions/create-module/20discover-smarthost
./actions/destroy-module/
./actions/destroy-module/10remove-routes
./actions/destroy-module/20stop-services
./actions/destroy-module/30remove-agent-state
./actions/destroy-module/40remove-agents-root
./actions/get-agent-runtime/
./actions/get-agent-runtime/10read
./actions/get-agent-runtime/validate-output.json
./actions/get-configuration/
./actions/get-configuration/20read
./actions/get-configuration/validate-output.json
./actions/list-domain-users/
./actions/list-domain-users/10read
./actions/list-domain-users/validate-input.json
./actions/list-domain-users/validate-output.json
./actions/list-user-domains/
./actions/list-user-domains/10read
./actions/list-user-domains/validate-output.json
./actions/restore-module/
./actions/restore-module/06copyenv
./actions/restore-module/20configure
./bin/
./bin/discover-smarthost
./bin/ensure-agent-home-ownership
./bin/remove-agent-state
./bin/sync-agent-runtime
./etc/
./etc/state-include.conf
./events/
./events/smarthost-changed/
./events/smarthost-changed/10reload_services
./pypkg/
./pypkg/hermes_agent_state.py
./pypkg/hermes_user_domain.py
./systemd/
./systemd/user/
./systemd/user/hermes-auth.service
./systemd/user/hermes-pod@.service
./systemd/user/hermes-socket@.service
./systemd/user/hermes@.service
./templates/
./templates/SOUL/
./templates/SOUL/business_consultant.md.in
./templates/SOUL/customer_support.md.in
./templates/SOUL/default.md.in
./templates/SOUL/developer.md.in
./templates/SOUL/marketing.md.in
./templates/SOUL/researcher.md.in
./templates/SOUL/sales.md.in
./templates/SOUL/social_media_manager.md.in
./templates/home.env.in
./update-module.d/
./update-module.d/15migrate-agents-volume
./update-module.d/20migrate-secrets-dir
./update-module.d/25migrate-agent-env
./update-module.d/30ensure-agent-home-ownership
./update-module.d/80restart
chown: der Eigentümer von './systemd/user/hermes@.service.bak' wird geändert: Die Operation ist nicht erlaubt
chown: der Eigentümer von './systemd/user/hermes@.service.d' wird geändert: Die Operation ist nicht erlaubt
time="2026-06-04T09:59:29+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
dd0595b8d2d36b32684937f815c028825da9fb433fc8aa515b812b657f4898ba
Traceback (most recent call last):
  File "/usr/local/agent/actions/update-module/05pullimages", line 81, in <module>
    ).check_returncode()
      ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
                             self.stderr)
subprocess.CalledProcessError: Command '('extract-image', 'ghcr.io/stell0/hermes-agent:0.5.0')' returned non-zero exit status 1.

Stll0 · June 4, 2026, 8:21am

the metadata in the nethforge repository should update in a few hours, in the meanwhile you can update it from command line.

api-cli run update-module --data ‘{“module_url”:“ghcr.io/nethserver/hermes-agent:0.5.0",“instances”:["hermes-agent2”], “force”: true}’

That happened because I moved the repository ownership from my personal github to the Nethserver one. It’s a small step toward further officialization I hope

capote · June 4, 2026, 8:59am

root@daho-ns8:~# api-cli run update-module --data '{"module_url":"ghcr.io/nethserver/hermes-agent:0.5.0","instances":["hermes-agent2"],"force":true}'
<7>podman pull ghcr.io/nethserver/hermes-agent:0.5.0
Trying to pull ghcr.io/nethserver/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:6821d03e091e9cc6a8079c4122a75f4915b15fde686b584d4b114d2df1818073
Copying config sha256:2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
Writing manifest to image destination
2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)
""
root@daho-ns8:~#

Stll0 · June 4, 2026, 9:27am

any other info in the logs?

capote · June 4, 2026, 9:44am

root@daho-ns8:/# runagent -m hermes-agent2 sh -x /usr/local/agent/bin/extract-image ghcr.io/nethserver/hermes-agent:0.5.0 2>&1 | tail -40
+ image=ghcr.io/nethserver/hermes-agent:0.5.0
+ set -e
+ exec 201
/usr/local/agent/bin/extract-image: 30: exec: 201: not found
root@daho-ns8:/#

root@daho-ns8:/# runagent -m hermes-agent2 bash -x /usr/local/agent/bin/extract-image ghcr.io/nethserver/hermes-agent:0.5.0 2>&1 | tail -60
./actions/get-configuration/
./actions/get-configuration/20read
./actions/get-configuration/validate-output.json
./actions/list-domain-users/
./actions/list-domain-users/10read
./actions/list-domain-users/validate-input.json
./actions/list-domain-users/validate-output.json
./actions/list-user-domains/
./actions/list-user-domains/10read
./actions/list-user-domains/validate-output.json
./actions/restore-module/
./actions/restore-module/06copyenv
./actions/restore-module/20configure
./bin/
./bin/discover-smarthost
./bin/ensure-agent-home-ownership
./bin/remove-agent-state
./bin/sync-agent-runtime
./etc/
./etc/state-include.conf
./events/
./events/smarthost-changed/
./events/smarthost-changed/10reload_services
./pypkg/
./pypkg/hermes_agent_state.py
./pypkg/hermes_user_domain.py
./systemd/
./systemd/user/
./systemd/user/hermes-auth.service
./systemd/user/hermes-pod@.service
./systemd/user/hermes-socket@.service
./systemd/user/hermes@.service
./templates/
./templates/SOUL/
./templates/SOUL/business_consultant.md.in
./templates/SOUL/customer_support.md.in
./templates/SOUL/default.md.in
./templates/SOUL/developer.md.in
./templates/SOUL/marketing.md.in
./templates/SOUL/researcher.md.in
./templates/SOUL/sales.md.in
./templates/SOUL/social_media_manager.md.in
./templates/home.env.in
./update-module.d/
./update-module.d/15migrate-agents-volume
./update-module.d/20migrate-secrets-dir
./update-module.d/25migrate-agent-env
./update-module.d/30ensure-agent-home-ownership
./update-module.d/80restart
+ [[ -f .imageroot.lst~ ]]
+ dirs_list=()
+ read -r entry
++ diff .imageroot.lst~ .imageroot.lst
++ sed -n '/^</ {s/^< // ; p}'
+ (( idx=0-1  ))
+ (( idx>=0  ))
+ chown -cR --reference=. .
+ podman rm c6ad906aa01f04bfd04852559fb994354f3e10a8dca6fd3d4e974e9c300fb132
time="2026-06-04T11:39:40+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
c6ad906aa01f04bfd04852559fb994354f3e10a8dca6fd3d4e974e9c300fb132
root@daho-ns8:/#

I believe around from here comes the error:

time="2026-06-04T11:39:40+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"

Possible explanation: The extract-image command was run manually successfully. It only failed during the automatic update (01:31). The error was:

subprocess.CalledProcessError: Command '('extract-image', ...)' returned non-zero exit status 1 at step 05pullimages

I tried again to get the current error:

root@daho-ns8:/# api-cli run update-module --data '{"module_url":"ghcr.io/nethserver/hermes-agent:0.5.0","instances":["hermes-agent2"],"force":true}'
<7>podman pull ghcr.io/nethserver/hermes-agent:0.5.0
Trying to pull ghcr.io/nethserver/hermes-agent:0.5.0...
Getting image source signatures
Copying blob sha256:6821d03e091e9cc6a8079c4122a75f4915b15fde686b584d4b114d2df1818073
Copying config sha256:2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
Writing manifest to image destination
2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
<3>Module instance "hermes-agent2" update failed with code 1
Assertion failed
  File "/var/lib/nethserver/cluster/actions/update-module/50update", line 158, in <module>
    agent.assert_exp(errors == 0)
""
root@daho-ns8:/# journalctl -t agent@hermes-agent2 --since "1 minute ago" --no-pager
Jun 04 11:43:25 daho-ns8 agent@hermes-agent2[3427]: task/module/hermes-agent2/dd59f0ba-af31-4d6b-a31a-abd8029da3a8: update-module/05pullimages is starting
Jun 04 11:43:25 daho-ns8 agent@hermes-agent2[3427]: podman pull ghcr.io/nethserver/hermes-agent:0.5.0
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:26+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Trying to pull ghcr.io/nethserver/hermes-agent:0.5.0...
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Getting image source signatures
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:6821d03e091e9cc6a8079c4122a75f4915b15fde686b584d4b114d2df1818073
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Copying config sha256:2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Writing manifest to image destination
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: 2a4de40ffb809807947cf12fd72faa1a515db443591b2e081de4dc7268b781c6
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:26+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: podman pull ghcr.io/nethserver/hermes-agent-auth:0.5.0 ghcr.io/nethserver/hermes-agent-hermes:0.5.0 ghcr.io/nethserver/hermes-agent-socket:0.5.0
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:26+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:26 daho-ns8 agent@hermes-agent2[3427]: Trying to pull ghcr.io/nethserver/hermes-agent-auth:0.5.0...
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Getting image source signatures
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:3cb3ac198a2f7b0bc9b90e3f943dd4a27cd2527488d08536ff41861d62ec9e34
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:7693d6ef93c9e7c58bb160c8fbe24814f4e49fcdee7b311b6522e018bc2e194d
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:c39513b5f21c259e4a903abcd304bf44b79bbb9fc9621ae2912f69d4f6a927c4
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:eff3576c9e7a6387c5f2d04c958711dec9791ee364ee1220d0eb92e92ab42b41
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:58d3257bdaafa16789addad906160fc5ee99b08b2a132c3a14db207d61fdd134
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:469ee591e193da4f6b7a2469147e5f9bbe5a6526939a79596a9fccfcaf56b1a5
Jun 04 11:43:27 daho-ns8 agent@hermes-agent2[3427]: Copying config sha256:69fafd4513e8fb32b8f8edbf0bef1ca7ffe673e73e22457bf3c272081bf01009
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Writing manifest to image destination
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: 69fafd4513e8fb32b8f8edbf0bef1ca7ffe673e73e22457bf3c272081bf01009
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Trying to pull ghcr.io/nethserver/hermes-agent-hermes:0.5.0...
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Getting image source signatures
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:d2bd5614174922ca1c3d05b0094003e7ba3ad0e93f0043d2445f6c81ca701111
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:eac402f3d6e7c106f4665ceb755ac7c3bf6fde2940f09ab37a94a725eaddba97
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:639fcc2031a8da8f0ea313257edba4a6e331da029879e56938e91f69d537383f
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:ff4e86d7e8878019880fd902bc7b8f22aaa0320d488c69dd695bfc2119c963bb
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:4f72dae80ae1a451655c36c281148022352a928e1903e2f6acb09a278e1cb106
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:f79bf00cf5d9324f5aea1dff54286dd378728f831b51fc74285466af6871f8d8
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:4ca7cb1011b0c011128a600d9365086d2cdc5b3f1614e5f7eb0e083818e4977a
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:347b71c8122a4800e80911d7b7febdccfb3204f37a44a7a6ce7dcedf5e567b41
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:c8ba6d97ebb1dd7fa37515afb0f846b9eaf3c77cd183f6f3db9a00466e56078b
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:8332ec5f114be92c71965b19463ee8a2a375af28f0e4a2b64fc1be4c222b9460
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:eeaa3c7bee18e8470fd3b0febb437731931b3c7e399fff7f902ac20a55c7801f
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:5c9ba6501fc0efdfe832e26f3ad0c7f6d7dabc8e8caf85c06879ed30fd1c928c
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:4659ea2b98f42cb8c6355b070b61be95ac88fe08c06b964e1c93f11af0890a8b
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:82de33f1790d7151ca5f85b1dd62a2a81a6054c706f1b4c1b95128a60ac4039e
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:81648dde7d7cf74a6e23f5252c6be563cb240fcec0a08c9d5bdd8587be27be87
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:169e9eafb0f3553ec577baf232920717f65d39d00ff67ec5badd750dfee93beb
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:4cb266eade751aa098433da2efc013ecc95f0f9cc7aa986dbc6ca366d2742b54
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:31650fd3070ed1ad37ffdedf33c796f282f61505fbaa082324724ed86203d139
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:8884c19c9800f7c086fe0a55a15f49b84123fa4394da795210c4d22acc21d9f8
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:12ac73398190bf5a549fe32b6b2d1329aeec15fb8d6ee7878c506373751135d6
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:592bc7889dc1658c867bf36374504dd14902d015ee99027d8abcedf6900fb7d0
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:97e8c141886cc189f905b1c0e92f492b6a5f381ced63e1636b80ceee2188d96b
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:1979ae7209ff184e570b13e17cb83e791a7cf9f5f390590722f119ecaab73e91
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:0d9111c063985e0278a68fad756fe1f38c982ec1d5936bba3471627a39c3588b
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:2ceb3523d4a53783cf419fba02945a5509095ace62f47b89b1dbe78243fa6bb4
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:eda0390fe60ac2315fb94e1a4e25cdbaeea6d0c833c18c044b49393988ed534f
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:14e20a0d01abfb578c5a717b1d7e9b06b04eaceacd437da555724eadb8744ed0
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:3ea9158c6c5fe7e394dae44fa0f1ac327d6b16eaebc4aa04f04e50535efb70fe
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:543676d1c7244781a0cd8c7567b8a30ea244fe041a28b780d272401e8807e652
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:ca94ae9b6af370cf5f857f54f5ba30a5be0bb59e78b815a55af5d28be10cf988
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:e5164748edf883db9954d834f384420ee312b860b41759f187d3db09d833ba52
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:b79ac18e4a3c85e3afae30750dd2cf27921ad78775232414cfbafde539c4590e
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:f425310c14c3de8d5df5824b8977550c59a6e96f202e22a158250997b6bd9e0e
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:c949b4798f9647db9daf80240cf6e952dd5c1abdeab3cc0a4bd8172612cb23be
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:4151c03cee491a6c8d961bc0ad045895367be984203c26ab72331128a10567b3
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:2fa145469af9af3b626259cf5c82de02b1212bdbd29e5436da576a8e05998ce1
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:b7e574fb4d01514cdbd0c768f46b8ebbc12758d0aff015c908b0a1df2cf8e608
Jun 04 11:43:28 daho-ns8 agent@hermes-agent2[3427]: Copying config sha256:f303dc957f6981b6700dbba5d2ca2cbae1557f849708d8133542a6918e6c4eac
Jun 04 11:43:29 daho-ns8 agent@hermes-agent2[3427]: Writing manifest to image destination
Jun 04 11:43:29 daho-ns8 agent@hermes-agent2[3427]: f303dc957f6981b6700dbba5d2ca2cbae1557f849708d8133542a6918e6c4eac
Jun 04 11:43:29 daho-ns8 agent@hermes-agent2[3427]: Trying to pull ghcr.io/nethserver/hermes-agent-socket:0.5.0...
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Getting image source signatures
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:196b350a78472377a42d100f95e4fbee01390b937055f80ed7a067b35f5b7430
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Copying blob sha256:55cfd63653c76188760d313211122850525467c0d3c469111f3ba8f64898d7f2
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Copying config sha256:491d04d02e89bb83dc035461802b67fadc32ccd246d67f4587f2e794236c7f74
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Writing manifest to image destination
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: 491d04d02e89bb83dc035461802b67fadc32ccd246d67f4587f2e794236c7f74
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: extract-image ghcr.io/nethserver/hermes-agent:0.5.0
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: flock: getting lock took 0.000007 seconds
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:30+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: '.imageroot.lst' -> '.imageroot.lst~'
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:30+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Extracting container filesystem imageroot to /home/hermes-agent2/.config
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:30+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: Gesamtzahl gelesener Bytes: 9256960 (8,9MiB, 35MiB/s)
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./AGENTS.md
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/10validate-input
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/20persist-shared-env
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/25configure-user-domain
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/30remove-deleted-routes
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/40remove-deleted-agents
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/50write-agent-metadata
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/60refresh-shared-settings
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/70sync-agent-runtime
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/75seed-agent-home
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/80reload-systemd
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/90reconcile-desired-routes
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/95reconcile-agent-services
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/configure-module/validate-input.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/create-module/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/create-module/05check-podman-version
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/create-module/10initialize-state
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/create-module/20discover-smarthost
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/destroy-module/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/destroy-module/10remove-routes
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/destroy-module/20stop-services
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/destroy-module/30remove-agent-state
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/destroy-module/40remove-agents-root
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-agent-runtime/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-agent-runtime/10read
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-agent-runtime/validate-output.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-configuration/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-configuration/20read
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/get-configuration/validate-output.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-domain-users/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-domain-users/10read
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-domain-users/validate-input.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-domain-users/validate-output.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-user-domains/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-user-domains/10read
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/list-user-domains/validate-output.json
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/restore-module/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/restore-module/06copyenv
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./actions/restore-module/20configure
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./bin/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./bin/discover-smarthost
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./bin/ensure-agent-home-ownership
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./bin/remove-agent-state
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./bin/sync-agent-runtime
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./etc/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./etc/state-include.conf
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./events/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./events/smarthost-changed/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./events/smarthost-changed/10reload_services
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./pypkg/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./pypkg/hermes_agent_state.py
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./pypkg/hermes_user_domain.py
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/user/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/user/hermes-auth.service
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/user/hermes-pod@.service
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/user/hermes-socket@.service
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./systemd/user/hermes@.service
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/business_consultant.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/customer_support.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/default.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/developer.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/marketing.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/researcher.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/sales.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/SOUL/social_media_manager.md.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./templates/home.env.in
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/15migrate-agents-volume
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/20migrate-secrets-dir
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/25migrate-agent-env
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/30ensure-agent-home-ownership
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: ./update-module.d/80restart
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: chown: der Eigentümer von './systemd/user/hermes@.service.bak' wird geändert: Die Operation ist nicht erlaubt
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: chown: der Eigentümer von './systemd/user/hermes@.service.d' wird geändert: Die Operation ist nicht erlaubt
Jun 04 11:43:30 daho-ns8 agent@hermes-agent2[3427]: time="2026-06-04T11:43:30+02:00" level=error msg="User-selected graph driver \"overlay\" overwritten by graph driver \"vfs\" from database - delete libpod local files (\"/home/hermes-agent2/.local/share/containers/storage\") to resolve.  May prevent use of images created by other tools"
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]: 35126e4e71194a2db7d3cfb2deb15f7acd9b09287b19f20bea24b6d275727166
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]: Traceback (most recent call last):
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:   File "/usr/local/agent/actions/update-module/05pullimages", line 81, in <module>
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:     ).check_returncode()
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:       ~~~~~~~~~~~~~~~~^^
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:   File "/usr/lib/python3.13/subprocess.py", line 508, in check_returncode
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:     raise CalledProcessError(self.returncode, self.args, self.stdout,
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]:                              self.stderr)
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]: subprocess.CalledProcessError: Command '('extract-image', 'ghcr.io/nethserver/hermes-agent:0.5.0')' returned non-zero exit status 1.
Jun 04 11:43:31 daho-ns8 agent@hermes-agent2[3427]: task/module/hermes-agent2/dd59f0ba-af31-4d6b-a31a-abd8029da3a8: action "update-module" status is "aborted" (1) at step 05pullimages
root@daho-ns8:/#

Area	Measure
Mount option in service unit	Replaced `--mount type=volume,src=hermes-agents-home,dst=/opt/data,subpath=%i` with a bind mount `-v /home/hermes-agentX/.local/share/containers/storage/volumes/hermes-agents-home/_data/%i:/opt/data`
Mount option in seed script	Replaced `type=volume,src=...,subpath={agent_id}` with `type=bind,src=/home/hermes-agent2/.../_data/{agent_id},dst=...` (hermes-agent2 only)
Remove backup file	`rm /home/hermes-agent2/.config/actions/configure-module/75seed-agent-home.bak` – prevents NS8 from executing the old version
Correct write permissions	`chmod 777` on the agent directory `_data/1` – ensures the container user can write (workaround for UID mapping issue)
Daemon reload	`runagent -m hermes-agentX systemctl --user daemon-reload` – activates the changed service unit
Recreate the agent	After the changes, create the agent again in the NS8 web interface; the seed phase now completes without errors.

NS8 Hermes Agent module

Root Cause Analysis: subpath: invalid mount option for Hermes Agent (Nethserver 8)

The Problem

Root Cause

Solution – Step by Step

Affected Files (hermes-agent2)

Prevention and Repeat Protection

Result

Permanent Solution for Hermes Agent – systemd Drop-in Override

What was done?

Why is this update-safe?

Verification

Note about hermes-agent1

Root Cause Analysis: `subpath: invalid mount option` for Hermes Agent (Nethserver 8)