Can you try to backup and then restore and check if it will work
Restored last nightâs backup; itâs still behaving the same way. Looks like the same error: permission denied on /dev/shm/counter_2.db:
2025-08-27T05:16:02-04:00 [1:goauthentik2:goauthentik-app] {"event": "Internal Server Error: /api/v3/outposts/instances/", "exception": [{"exc_notes": [], "exc_type": "PermissionError", "exc_value": "[Errno 13] Permission denied: '/dev/shm/counter_2.db'", "exceptions": [], "frames": [{"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 489, "locals": {"args": "'(functools.partial(<function response_for_exception at 0x7f2d2e1a6b60>, <ASGIReq'+85", "exc_info": "\"(<class 'PermissionError'>, PermissionError(13, 'Permission denied'), <traceback\"+27", "func": "<built-in method run of _contextvars.Context object at 0x7f2d241c6b00>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d247b1d30>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", "lineno": 42, "locals": {"exc": "PermissionError(13, 'Permission denied')", "get_response": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>"}, "name": "inner"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/asgi.py", "lineno": 243, "locals": {"_check_middleware_span": "<function _wrap_middleware.<locals>._check_middleware_span at 0x7f2d230d45e0>", "args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "f": "'<bound method MiddlewareMixin.__acall__ of <PrometheusAfterMiddleware get_respon'+36", "kwargs": "{}", "middleware_span": "\"<Span(op='middleware.django', description:'django_prometheus.middleware.Promethe\"+180", "self": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", "lineno": 141, "locals": {"request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 439, "locals": {"args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "child": "'functools.partial(<bound method PrometheusAfterMiddleware.process_request of <Pr'+121", "context": "<_contextvars.Context object at 0x7f2d232a5d40>", "current_thread_executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d24796270>", "exec_coro": "<Future finished exception=PermissionError(13, 'Permission denied')>", "executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d24796270>", "func": "<built-in method run of _contextvars.Context object at 0x7f2d232a5d40>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d247b1ef0>", "task_context": "[]"}, "name": "__call__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", "lineno": 40, "locals": {"self": "None"}, "name": "run"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 493, "locals": {"args": "'(functools.partial(<bound method PrometheusAfterMiddleware.process_request of <P'+124", "exc_info": "(None, None, None)", "func": "<built-in method run of _contextvars.Context object at 0x7f2d232a5d40>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d247b1ef0>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 218, "locals": {"method": "'GET'", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>", "transport": "'http'"}, "name": "process_request"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 213, "locals": {"labels": "{'method': 'GET'}", "metric": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "label_metric"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 179, "locals": {"labelkwargs": "{'method': 'GET'}", "labelvalues": "('GET',)", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "labels"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 127, "locals": {"_labelvalues": "('GET',)", "documentation": "'Count of requests by method.'", "labelnames": "('method',)", "name": "'django_http_requests_total_by_method'", "namespace": "''", "registry": "<prometheus_client.registry.CollectorRegistry object at 0x7f2d346cfa10>", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "subsystem": "''", "unit": "''"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 282, "locals": {"self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "_metric_init"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 68, "locals": {"help_text": "'Count of requests by method.'", "kwargs": "{}", "labelnames": "('method',)", "labelvalues": "('GET',)", "lock": "<unlocked _thread.lock object at 0x7f2d23017510>", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'", "values": "'[<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f'+104"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 82, "locals": {"file_prefix": "'counter'", "filename": "'/dev/shm/counter_2.db'", "files": "{'gauge_all': <prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2443c350>}", "help_text": "'Count of requests by method.'", "labelnames": "('method',)", "labelvalues": "('GET',)", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "pid": "{'value': 2}", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'"}, "name": "__reset"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/mmap_dict.py", "lineno": 64, "locals": {"filename": "'/dev/shm/counter_2.db'", "read_mode": "False", "self": "<prometheus_client.mmap_dict.MmapedDict object at 0x7f2d24446f30>"}, "name": "__init__"}], "is_cause": false, "is_group": false, "syntax_error": null}], "level": "error", "logger": "django.request", "timestamp": "2025-08-27T09:16:02.569622"}
2025-08-27T05:16:02-04:00 [1:goauthentik2:goauthentik-app] {"error":"500 Internal Server Error","event":"Failed to fetch outpost configuration","level":"error","timestamp":"2025-08-27T09:16:02Z"}
2025-08-27T05:16:48-04:00 [1:goauthentik2:goauthentik-app] {"event": "Internal Server Error: /api/v3/core/brands/current/", "exception": [{"exc_notes": [], "exc_type": "PermissionError", "exc_value": "[Errno 13] Permission denied: '/dev/shm/counter_2.db'", "exceptions": [], "frames": [{"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 489, "locals": {"args": "'(functools.partial(<function response_for_exception at 0x7f2d2e1a6b60>, <ASGIReq'+86", "exc_info": "\"(<class 'PermissionError'>, PermissionError(13, 'Permission denied'), <traceback\"+27", "func": "<built-in method run of _contextvars.Context object at 0x7f2d241c7340>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d24796430>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", "lineno": 42, "locals": {"exc": "PermissionError(13, 'Permission denied')", "get_response": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1", "request": "<ASGIRequest: GET '/api/v3/core/brands/current/'>"}, "name": "inner"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/asgi.py", "lineno": 243, "locals": {"_check_middleware_span": "<function _wrap_middleware.<locals>._check_middleware_span at 0x7f2d230d45e0>", "args": "(<ASGIRequest: GET '/api/v3/core/brands/current/'>,)", "f": "'<bound method MiddlewareMixin.__acall__ of <PrometheusAfterMiddleware get_respon'+36", "kwargs": "{}", "middleware_span": "\"<Span(op='middleware.django', description:'django_prometheus.middleware.Promethe\"+180", "self": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", "lineno": 141, "locals": {"request": "<ASGIRequest: GET '/api/v3/core/brands/current/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 439, "locals": {"args": "(<ASGIRequest: GET '/api/v3/core/brands/current/'>,)", "child": "'functools.partial(<bound method PrometheusAfterMiddleware.process_request of <Pr'+122", "context": "<_contextvars.Context object at 0x7f2d23460880>", "current_thread_executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d2400ad50>", "exec_coro": "<Future finished exception=PermissionError(13, 'Permission denied')>", "executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d2400ad50>", "func": "<built-in method run of _contextvars.Context object at 0x7f2d23460880>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d24796b30>", "task_context": "[]"}, "name": "__call__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", "lineno": 40, "locals": {"self": "None"}, "name": "run"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 493, "locals": {"args": "'(functools.partial(<bound method PrometheusAfterMiddleware.process_request of <P'+125", "exc_info": "(None, None, None)", "func": "<built-in method run of _contextvars.Context object at 0x7f2d23460880>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d24796b30>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 218, "locals": {"method": "'GET'", "request": "<ASGIRequest: GET '/api/v3/core/brands/current/'>", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>", "transport": "'https'"}, "name": "process_request"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 213, "locals": {"labels": "{'method': 'GET'}", "metric": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "request": "<ASGIRequest: GET '/api/v3/core/brands/current/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "label_metric"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 179, "locals": {"labelkwargs": "{'method': 'GET'}", "labelvalues": "('GET',)", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "labels"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 127, "locals": {"_labelvalues": "('GET',)", "documentation": "'Count of requests by method.'", "labelnames": "('method',)", "name": "'django_http_requests_total_by_method'", "namespace": "''", "registry": "<prometheus_client.registry.CollectorRegistry object at 0x7f2d346cfa10>", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "subsystem": "''", "unit": "''"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 282, "locals": {"self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "_metric_init"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 68, "locals": {"help_text": "'Count of requests by method.'", "kwargs": "{}", "labelnames": "('method',)", "labelvalues": "('GET',)", "lock": "<unlocked _thread.lock object at 0x7f2d23017510>", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'", "values": "'[<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f'+104"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 82, "locals": {"file_prefix": "'counter'", "filename": "'/dev/shm/counter_2.db'", "files": "{'gauge_all': <prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2443c350>}", "help_text": "'Count of requests by method.'", "labelnames": "('method',)", "labelvalues": "('GET',)", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "pid": "{'value': 2}", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'"}, "name": "__reset"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/mmap_dict.py", "lineno": 64, "locals": {"filename": "'/dev/shm/counter_2.db'", "read_mode": "False", "self": "<prometheus_client.mmap_dict.MmapedDict object at 0x7f2d24795e10>"}, "name": "__init__"}], "is_cause": false, "is_group": false, "syntax_error": null}], "level": "error", "logger": "django.request", "timestamp": "2025-08-27T09:16:48.036558"}
Did you mean to restore a backup of the old version?
No i meant the current
let me see how i can resolve this
@mrmarkuz is there a way to run a service as root user
example
worker:
command: worker
depends_on:
postgresql:
condition: service_healthy
redis:
condition: service_healthy
env_file:
- .env
environment:
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?secret key required}
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2025.10.0-rc1}
restart: unless-stopped
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./media:/media
- ./certs:/certs
- ./custom-templates:/templates
i think this can be a major issue that why we are getting the issue of permission errors
to fix this
runagent -m goauthentik1 podman exec -it goauthentik-app /bin/bash
make sure you are in root dir /
cd /
Then run
python manage.py repair_permissions
let me know if it works
root@goauthentik:/# python manage.py repair_permissions
{"event": "Loaded config", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1756374223.1556637, "file": "/authentik/lib/default.yml"}
{"event": "Loaded environment variables", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1756374223.156387, "count": 14}
{"event": "Starting authentik bootstrap", "level": "info", "logger": "authentik.lib.config", "timestamp": 1756374224.1125207}
{"event": "PostgreSQL connection successful", "level": "info", "logger": "authentik.lib.config", "timestamp": 1756374224.131076}
{"event": "Redis Connection successful", "level": "info", "logger": "authentik.lib.config", "timestamp": 1756374224.1349638}
{"event": "Finished authentik bootstrap", "level": "info", "logger": "authentik.lib.config", "timestamp": 1756374224.135134}
{"event": "Booting authentik", "level": "info", "logger": "authentik.lib.config", "timestamp": 1756374227.0548036, "version": "2025.8.1"}
{"event": "Loaded app settings", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1756374227.0638678, "path": "authentik.enterprise.settings"}
{"event": "Loaded app settings", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1756374227.068556, "path": "authentik.enterprise.search.settings"}
{"event": "Loaded app settings", "level": "debug", "logger": "authentik.lib.config", "timestamp": 1756374227.0809877, "path": "authentik.stages.authenticator_totp.settings"}
{"domain_url": null, "event": "Loaded MMDB database", "file": "/geoip/GeoLite2-ASN.mmdb", "last_write": 1755171260.0, "level": "info", "logger": "authentik.events.context_processors.mmdb", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:48.251078"}
{"domain_url": null, "event": "Loaded MMDB database", "file": "/geoip/GeoLite2-City.mmdb", "last_write": 1755171259.0, "level": "info", "logger": "authentik.events.context_processors.mmdb", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:48.256306"}
{"app_name": "authentik.tenants", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.tenants.checks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.329070"}
{"app_name": "authentik.tenants", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.tenants.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.330874"}
{"app_name": "authentik.tasks", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.tasks.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.395982"}
{"app_name": "authentik.tasks", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.tasks.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.403592"}
{"app_name": "authentik.admin", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.admin.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.442473"}
{"app_name": "authentik.core", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.core.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.445677"}
{"app_name": "authentik.core", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.core.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.449039"}
{"app_name": "authentik.crypto", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.crypto.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.467814"}
{"app_name": "authentik.enterprise", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.470648"}
{"app_name": "authentik.enterprise", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.472969"}
{"app_name": "authentik.events", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.events.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.476466"}
{"app_name": "authentik.events", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.events.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.476900"}
{"app_name": "authentik.flows", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.flows.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.577743"}
{"app_name": "authentik.outposts", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.outposts.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.672354"}
{"app_name": "authentik.outposts", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.outposts.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.676649"}
{"app_name": "authentik.policies.reputation", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.policies.reputation.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.680777"}
{"app_name": "authentik.policies", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.policies.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.712922"}
{"app_name": "authentik.providers.oauth2", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.oauth2.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.725447"}
{"app_name": "authentik.providers.oauth2", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.oauth2.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.727278"}
{"app_name": "authentik.providers.proxy", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.proxy.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.729031"}
{"app_name": "authentik.providers.proxy", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.proxy.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.730672"}
{"app_name": "authentik.providers.rac", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.rac.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.738884"}
{"app_name": "authentik.providers.scim", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.scim.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.747734"}
{"app_name": "authentik.providers.scim", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.providers.scim.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.750964"}
{"app_name": "authentik.rbac", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.rbac.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.753899"}
{"app_name": "authentik.sources.kerberos", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.kerberos.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.759564"}
{"app_name": "authentik.sources.kerberos", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.kerberos.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.761614"}
{"app_name": "authentik.sources.ldap", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.ldap.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.785866"}
{"app_name": "authentik.sources.ldap", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.ldap.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.797052"}
{"app_name": "authentik.sources.oauth", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.oauth.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.845613"}
{"app_name": "authentik.sources.plex", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.plex.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.850327"}
{"app_name": "authentik.sources.saml", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.saml.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.852075"}
{"app_name": "authentik.sources.scim", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.sources.scim.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.853883"}
{"app_name": "authentik.stages.authenticator_static", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.authenticator_static.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.856612"}
{"app_name": "authentik.stages.authenticator_webauthn", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.authenticator_webauthn.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.928309"}
{"app_name": "authentik.stages.email", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.email.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.929990"}
{"app_name": "authentik.stages.identification", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.identification.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.941904"}
{"app_name": "authentik.stages.invitation", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.invitation.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.942811"}
{"app_name": "authentik.stages.prompt", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.prompt.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.943458"}
{"app_name": "authentik.stages.user_write", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.stages.user_write.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.944673"}
{"app_name": "authentik.tasks.schedules", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.tasks.schedules.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.946386"}
{"app_name": "authentik.blueprints", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.blueprints.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.975585"}
{"app_name": "authentik.enterprise.policies.unique_password", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.policies.unique_password.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.979711"}
{"app_name": "authentik.enterprise.policies.unique_password", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.policies.unique_password.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.981450"}
{"app_name": "authentik.enterprise.providers.google_workspace", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.google_workspace.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.984487"}
{"app_name": "authentik.enterprise.providers.google_workspace", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.google_workspace.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.987226"}
{"app_name": "authentik.enterprise.providers.microsoft_entra", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.microsoft_entra.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.990006"}
{"app_name": "authentik.enterprise.providers.microsoft_entra", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.microsoft_entra.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.991399"}
{"app_name": "authentik.enterprise.providers.ssf", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.ssf.tasks", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.994774"}
{"app_name": "authentik.enterprise.providers.ssf", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.enterprise.providers.ssf.signals", "pid": 20054, "schema_name": "public", "timestamp": "2025-08-28T09:43:51.998240"}
Checking app django_tenants (django_tenants)
Checking app authentik.tenants (authentik_tenants)
Checking app django.contrib.messages (messages)
Checking app django.contrib.staticfiles (staticfiles)
Checking app django.contrib.humanize (humanize)
Checking app rest_framework (rest_framework)
Checking app django_filters (django_filters)
Checking app drf_spectacular (drf_spectacular)
Checking app django_prometheus (django_prometheus)
Checking app django_countries (django_countries)
Checking app pgactivity (pgactivity)
Checking app pglock (pglock)
Checking app channels (channels)
Checking app django_dramatiq_postgres (django_dramatiq_postgres)
Checking app authentik.tasks (authentik_tasks)
Checking app django.contrib.auth (auth)
Checking app django.contrib.contenttypes (contenttypes)
Checking app django.contrib.sessions (sessions)
Checking app pgtrigger (pgtrigger)
Checking app authentik.admin (authentik_admin)
Checking app authentik.api (authentik_api)
Checking app authentik.core (authentik_core)
Checking app authentik.crypto (authentik_crypto)
Checking app authentik.enterprise (authentik_enterprise)
Checking app authentik.events (authentik_events)
Checking app authentik.flows (authentik_flows)
Checking app authentik.outposts (authentik_outposts)
Checking app authentik.policies.dummy (authentik_policies_dummy)
Checking app authentik.policies.event_matcher (authentik_policies_event_matcher)
Checking app authentik.policies.expiry (authentik_policies_expiry)
Checking app authentik.policies.expression (authentik_policies_expression)
Checking app authentik.policies.geoip (authentik_policies_geoip)
Checking app authentik.policies.password (authentik_policies_password)
Checking app authentik.policies.reputation (authentik_policies_reputation)
Checking app authentik.policies (authentik_policies)
Checking app authentik.providers.ldap (authentik_providers_ldap)
Checking app authentik.providers.oauth2 (authentik_providers_oauth2)
Checking app authentik.providers.proxy (authentik_providers_proxy)
Checking app authentik.providers.rac (authentik_providers_rac)
Checking app authentik.providers.radius (authentik_providers_radius)
Checking app authentik.providers.saml (authentik_providers_saml)
Checking app authentik.providers.scim (authentik_providers_scim)
Checking app authentik.rbac (authentik_rbac)
Checking app authentik.recovery (authentik_recovery)
Checking app authentik.sources.kerberos (authentik_sources_kerberos)
Checking app authentik.sources.ldap (authentik_sources_ldap)
Checking app authentik.sources.oauth (authentik_sources_oauth)
Checking app authentik.sources.plex (authentik_sources_plex)
Checking app authentik.sources.saml (authentik_sources_saml)
Checking app authentik.sources.scim (authentik_sources_scim)
Checking app authentik.stages.authenticator (authentik_stages_authenticator)
Checking app authentik.stages.authenticator_duo (authentik_stages_authenticator_duo)
Checking app authentik.stages.authenticator_email (authentik_stages_authenticator_email)
Checking app authentik.stages.authenticator_sms (authentik_stages_authenticator_sms)
Checking app authentik.stages.authenticator_static (authentik_stages_authenticator_static)
Checking app authentik.stages.authenticator_totp (authentik_stages_authenticator_totp)
Checking app authentik.stages.authenticator_validate (authentik_stages_authenticator_validate)
Checking app authentik.stages.authenticator_webauthn (authentik_stages_authenticator_webauthn)
Checking app authentik.stages.captcha (authentik_stages_captcha)
Checking app authentik.stages.consent (authentik_stages_consent)
Checking app authentik.stages.deny (authentik_stages_deny)
Checking app authentik.stages.dummy (authentik_stages_dummy)
Checking app authentik.stages.email (authentik_stages_email)
Checking app authentik.stages.identification (authentik_stages_identification)
Checking app authentik.stages.invitation (authentik_stages_invitation)
Checking app authentik.stages.password (authentik_stages_password)
Checking app authentik.stages.prompt (authentik_stages_prompt)
Checking app authentik.stages.redirect (authentik_stages_redirect)
Checking app authentik.stages.user_delete (authentik_stages_user_delete)
Checking app authentik.stages.user_login (authentik_stages_user_login)
Checking app authentik.stages.user_logout (authentik_stages_user_logout)
Checking app authentik.stages.user_write (authentik_stages_user_write)
Checking app authentik.tasks.schedules (authentik_tasks_schedules)
Checking app authentik.brands (authentik_brands)
Checking app authentik.blueprints (authentik_blueprints)
Checking app guardian (guardian)
Checking app authentik.enterprise.audit (authentik_enterprise_audit)
Checking app authentik.enterprise.policies.unique_password (authentik_policies_unique_password)
Checking app authentik.enterprise.providers.google_workspace (authentik_providers_google_workspace)
Checking app authentik.enterprise.providers.microsoft_entra (authentik_providers_microsoft_entra)
Checking app authentik.enterprise.providers.ssf (authentik_providers_ssf)
Checking app authentik.enterprise.search (authentik_search)
Checking app authentik.enterprise.stages.authenticator_endpoint_gdtc (authentik_stages_authenticator_endpoint_gdtc)
Checking app authentik.enterprise.stages.mtls (authentik_stages_mtls)
Checking app authentik.enterprise.stages.source (authentik_stages_source)
root@goauthentik:/#
Still errors out on attempted login, with what appears to be the same error:
2025-08-27T23:57:00-04:00 [1:goauthentik2:goauthentik-app] {"event": "Internal Server Error: /api/v3/outposts/instances/", "exception": [{"exc_notes": [], "exc_type": "PermissionError", "exc_value": "[Errno 13] Permission denied: '/dev/shm/counter_2.db'", "exceptions": [], "frames": [{"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 489, "locals": {"args": "'(functools.partial(<function response_for_exception at 0x7f2d2e1a6b60>, <ASGIReq'+85", "exc_info": "\"(<class 'PermissionError'>, PermissionError(13, 'Permission denied'), <traceback\"+27", "func": "<built-in method run of _contextvars.Context object at 0x7f2d243558c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22ce5fd0>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", "lineno": 42, "locals": {"exc": "PermissionError(13, 'Permission denied')", "get_response": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>"}, "name": "inner"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/asgi.py", "lineno": 243, "locals": {"_check_middleware_span": "<function _wrap_middleware.<locals>._check_middleware_span at 0x7f2d230d45e0>", "args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "f": "'<bound method MiddlewareMixin.__acall__ of <PrometheusAfterMiddleware get_respon'+36", "kwargs": "{}", "middleware_span": "\"<Span(op='middleware.django', description:'django_prometheus.middleware.Promethe\"+180", "self": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", "lineno": 141, "locals": {"request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 439, "locals": {"args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "child": "'functools.partial(<bound method PrometheusAfterMiddleware.process_request of <Pr'+121", "context": "<_contextvars.Context object at 0x7f2d2421e6c0>", "current_thread_executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d247816a0>", "exec_coro": "<Future finished exception=PermissionError(13, 'Permission denied')>", "executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d247816a0>", "func": "<built-in method run of _contextvars.Context object at 0x7f2d2421e6c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22a0e430>", "task_context": "[]"}, "name": "__call__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", "lineno": 40, "locals": {"self": "None"}, "name": "run"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 493, "locals": {"args": "'(functools.partial(<bound method PrometheusAfterMiddleware.process_request of <P'+124", "exc_info": "(None, None, None)", "func": "<built-in method run of _contextvars.Context object at 0x7f2d2421e6c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22a0e430>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 218, "locals": {"method": "'GET'", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>", "transport": "'http'"}, "name": "process_request"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 213, "locals": {"labels": "{'method': 'GET'}", "metric": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "label_metric"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 179, "locals": {"labelkwargs": "{'method': 'GET'}", "labelvalues": "('GET',)", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "labels"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 127, "locals": {"_labelvalues": "('GET',)", "documentation": "'Count of requests by method.'", "labelnames": "('method',)", "name": "'django_http_requests_total_by_method'", "namespace": "''", "registry": "<prometheus_client.registry.CollectorRegistry object at 0x7f2d346cfa10>", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "subsystem": "''", "unit": "''"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 282, "locals": {"self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "_metric_init"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 68, "locals": {"help_text": "'Count of requests by method.'", "kwargs": "{}", "labelnames": "('method',)", "labelvalues": "('GET',)", "lock": "<unlocked _thread.lock object at 0x7f2d23017510>", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'", "values": "'[<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f'+288"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 82, "locals": {"file_prefix": "'counter'", "filename": "'/dev/shm/counter_2.db'", "files": "{'gauge_all': <prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2443c350>}", "help_text": "'Count of requests by method.'", "labelnames": "('method',)", "labelvalues": "('GET',)", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "pid": "{'value': 2}", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'"}, "name": "__reset"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/mmap_dict.py", "lineno": 64, "locals": {"filename": "'/dev/shm/counter_2.db'", "read_mode": "False", "self": "<prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2355ec50>"}, "name": "__init__"}], "is_cause": false, "is_group": false, "syntax_error": null}], "level": "error", "logger": "django.request", "timestamp": "2025-08-28T03:57:00.786851"}
2025-08-27T23:57:00-04:00 [1:goauthentik2:goauthentik-app] {"error":"500 Internal Server Error","event":"Failed to fetch outpost configuration","level":"error","timestamp":"2025-08-28T03:57:00Z"}
Contents and permissions in /dev/shm:
root@goauthentik:/dev/shm# ls -lh
total 1.6M
-rw-------. 1 70 70 1.0M Aug 28 09:44 PostgreSQL.1599363400
-rw-------. 1 70 70 27K Aug 27 09:15 PostgreSQL.3530919398
-rw-------. 1 root root 88 Aug 27 09:15 authentik-core-ipc.key
-rw-------. 1 root root 88 Aug 27 09:15 authentik-core-metrics.key
srwxrwxrwx. 1 root root 0 Aug 27 09:15 authentik-core.sock
-rw-r--r--. 1 root root 3 Aug 27 09:15 authentik-gunicorn.601273919.pid
-rw-r--r--. 1 root root 7 Aug 27 09:15 authentik-mode
-rw-r--r--. 1 authentik authentik 1 Aug 27 09:15 authentik-worker.pid
drwxr-xr-x. 2 root root 40 Aug 28 01:51 authentik_gunicorn_tmp
-rw-r--r--. 1 root root 64K Aug 27 09:26 counter_1.db
-rw-r--r--. 1 root root 64K Aug 28 01:51 counter_13680.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 counter_17.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 counter_2.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 counter_32.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 counter_41.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 counter_42.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 counter_43.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 counter_52.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 counter_76.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 counter_77.db
drwxr-xr-x. 2 authentik authentik 40 Aug 27 09:15 dramatiq-prometheus
-rw-r--r--. 1 root root 64K Aug 27 09:15 gauge_all_1.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 gauge_all_2.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 gauge_livesum_32.db
-rw-r--r--. 1 root root 128K Aug 28 09:44 histogram_1.db
-rw-r--r--. 1 root root 64K Aug 28 01:51 histogram_13680.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 histogram_17.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 histogram_2.db
-rw-r--r--. 1 authentik authentik 256K Aug 28 08:38 histogram_32.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 histogram_41.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 histogram_42.db
-rw-r--r--. 1 authentik authentik 64K Aug 27 09:15 histogram_43.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 histogram_52.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 histogram_76.db
-rw-r--r--. 1 root root 64K Aug 27 09:15 histogram_77.db
Can you make this to have root permission
The service should run as root so that the error can go away
Done, still getting the same error AFAICT:
2025-08-27T23:57:00-04:00 [1:goauthentik2:goauthentik-app] {"event": "Internal Server Error: /api/v3/outposts/instances/", "exception": [{"exc_notes": [], "exc_type": "PermissionError", "exc_value": "[Errno 13] Permission denied: '/dev/shm/counter_2.db'", "exceptions": [], "frames": [{"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 489, "locals": {"args": "'(functools.partial(<function response_for_exception at 0x7f2d2e1a6b60>, <ASGIReq'+85", "exc_info": "\"(<class 'PermissionError'>, PermissionError(13, 'Permission denied'), <traceback\"+27", "func": "<built-in method run of _contextvars.Context object at 0x7f2d243558c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22ce5fd0>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", "lineno": 42, "locals": {"exc": "PermissionError(13, 'Permission denied')", "get_response": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>"}, "name": "inner"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/sentry_sdk/integrations/django/asgi.py", "lineno": 243, "locals": {"_check_middleware_span": "<function _wrap_middleware.<locals>._check_middleware_span at 0x7f2d230d45e0>", "args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "f": "'<bound method MiddlewareMixin.__acall__ of <PrometheusAfterMiddleware get_respon'+36", "kwargs": "{}", "middleware_span": "\"<Span(op='middleware.django', description:'django_prometheus.middleware.Promethe\"+180", "self": "'<django_prometheus.middleware.PrometheusAfterMiddleware object at 0x7f2d24743380'+1"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django/utils/deprecation.py", "lineno": 141, "locals": {"request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "__acall__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 439, "locals": {"args": "(<ASGIRequest: GET '/api/v3/outposts/instances/'>,)", "child": "'functools.partial(<bound method PrometheusAfterMiddleware.process_request of <Pr'+121", "context": "<_contextvars.Context object at 0x7f2d2421e6c0>", "current_thread_executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d247816a0>", "exec_coro": "<Future finished exception=PermissionError(13, 'Permission denied')>", "executor": "<asgiref.current_thread_executor.CurrentThreadExecutor object at 0x7f2d247816a0>", "func": "<built-in method run of _contextvars.Context object at 0x7f2d2421e6c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22a0e430>", "task_context": "[]"}, "name": "__call__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", "lineno": 40, "locals": {"self": "None"}, "name": "run"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/asgiref/sync.py", "lineno": 493, "locals": {"args": "'(functools.partial(<bound method PrometheusAfterMiddleware.process_request of <P'+124", "exc_info": "(None, None, None)", "func": "<built-in method run of _contextvars.Context object at 0x7f2d2421e6c0>", "kwargs": "{}", "loop": "<uvloop.Loop running=True closed=False debug=False>", "self": "<asgiref.sync.SyncToAsync object at 0x7f2d22a0e430>", "task_context": "[]"}, "name": "thread_handler"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 218, "locals": {"method": "'GET'", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>", "transport": "'http'"}, "name": "process_request"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/middleware.py", "lineno": 213, "locals": {"labels": "{'method': 'GET'}", "metric": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "request": "<ASGIRequest: GET '/api/v3/outposts/instances/'>", "response": "None", "self": "<PrometheusAfterMiddleware get_response=BaseHandler._get_response_async>"}, "name": "label_metric"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 179, "locals": {"labelkwargs": "{'method': 'GET'}", "labelvalues": "('GET',)", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "labels"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 127, "locals": {"_labelvalues": "('GET',)", "documentation": "'Count of requests by method.'", "labelnames": "('method',)", "name": "'django_http_requests_total_by_method'", "namespace": "''", "registry": "<prometheus_client.registry.CollectorRegistry object at 0x7f2d346cfa10>", "self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)", "subsystem": "''", "unit": "''"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/metrics.py", "lineno": 282, "locals": {"self": "prometheus_client.metrics.Counter(django_http_requests_total_by_method)"}, "name": "_metric_init"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 68, "locals": {"help_text": "'Count of requests by method.'", "kwargs": "{}", "labelnames": "('method',)", "labelvalues": "('GET',)", "lock": "<unlocked _thread.lock object at 0x7f2d23017510>", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'", "values": "'[<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f'+288"}, "name": "__init__"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/values.py", "lineno": 82, "locals": {"file_prefix": "'counter'", "filename": "'/dev/shm/counter_2.db'", "files": "{'gauge_all': <prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2443c350>}", "help_text": "'Count of requests by method.'", "labelnames": "('method',)", "labelvalues": "('GET',)", "metric_name": "'django_http_requests_total_by_method'", "multiprocess_mode": "''", "name": "'django_http_requests_total_by_method_total'", "pid": "{'value': 2}", "self": "'<prometheus_client.values.MultiProcessValue.<locals>.MmapedValue object at 0x7f2'+10", "typ": "'counter'"}, "name": "__reset"}, {"filename": "/ak-root/.venv/lib/python3.13/site-packages/prometheus_client/mmap_dict.py", "lineno": 64, "locals": {"filename": "'/dev/shm/counter_2.db'", "read_mode": "False", "self": "<prometheus_client.mmap_dict.MmapedDict object at 0x7f2d2355ec50>"}, "name": "__init__"}], "is_cause": false, "is_group": false, "syntax_error": null}], "level": "error", "logger": "django.request", "timestamp": "2025-08-28T03:57:00.786851"}
2025-08-27T23:57:00-04:00 [1:goauthentik2:goauthentik-app] {"error":"500 Internal Server Error","event":"Failed to fetch outpost configuration","level":"error","timestamp":"2025-08-28T03:57:00Z"}
What I found but not tested and just as a test:
To show current setting of âfs.protected_regularâ
sysctl fs.protected_regular
Probably it is set to 1, to set it to zero (and vise versa)
sudo sysctl fs.protected_regular=0
With set to zero try again.
2 Likes
[root@ns8 ~]# sysctl fs.protected_regular
fs.protected_regular = 1
[root@ns8 ~]# sysctl fs.protected_regular=0
fs.protected_regular = 0
Attempted login again, and it works.
1 Like
And if you try at container level instead of server/node level?
[root@ns8 ~]# runagent -m goauthentik2 podman exec -it goauthentik-app /bin/bash
root@goauthentik:/# sysctl fs.protected_regular
bash: sysctl: command not found
Debian or Rocky? My best guess, DebianâŚ
Rocky 9.
1 Like
The âfs.protected_regularâ tag was already introduced in kernel 4.xx, so it seems something is not releasing /dev/shm/xxx with the latest Authentik version. alI best guesses.
1 Like
If you get permission denied, just go through this tutorial
Maybe you need to add capabilities?.
Yeah i need too run the service as root
As you can see the
the worker service need to run as root
worker:
command: worker
depends_on:
postgresql:
condition: service_healthy
redis:
condition: service_healthy
env_file:
- .env
environment:
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?secret key required}
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2025.10.0-rc1}
restart: unless-stopped
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./media:/media
- ./certs:/certs
- ./custom-templates:/templates
I see that in the official compose file, but what I donât understand is whyâwhy does a SSO/IAM service need privileged access to the system?
1 Like
I have no idea why
And also in the previous version it was able to run without root
Another example and comments
worker:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2023.10.2}
restart: unless-stopped
command: worker
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
# `user: root` and the docker socket volume are optional.
# See more for the docker socket integration here:
# https://goauthentik.io/docs/outposts/integrations/docker
# Removing `user: root` also prevents the worker from fixing the permissions
# on the mounted folders, so when removing this make sure the folders have the correct UID/GID
# (1000:1000 by default)
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./media:/media
- ./certs:/certs
- ./custom-templates:/templates
env_file:
- .env
depends_on:
- postgresql
- redis
Worker Service
Image: Utilizes the same image as the server for background tasks.
User: Optionally runs as root to manage Docker integrations.
Volumes: Similar to the server, for media, certificates, and templates.