Traefik access log: keep the User-Agent header (CrowdSec and SIEM)

Feature
1

Hi all,

on NS8 the Traefik static configuration is created with accessLog: {}. With this default Traefik drops every request header, so the User-Agent field of the access log line is always "-" (the example in the ns8-traefik README shows it too).

This limits log-based security tools:

  • the official ns8-crowdsec module installs crowdsecurity/base-http-scenarios, but scenarios such as http-bad-user-agent can never match, because the User-Agent is always "-";
  • external SIEMs reading the access log (we send it to Wazuh) cannot spot known scanners or Shellshock attempts in the User-Agent, and cannot skip the 404s of legitimate search engine crawlers.

Proposal: keep only the User-Agent header and keep dropping all the others. Referer can carry tokens or OAuth codes, and Authorization/Cookie must never be logged.

accessLog:
  fields:
    headers:
      defaultMode: drop
      names:
        User-Agent: keep
  • new installations: in the create-module action;
  • existing installations: an idempotent step in update-module.d (like add_metrics) that changes only the default accessLog: {} and leaves disabled or customized access logs alone.

I have a small patch ready (3 files, about 20 lines). If you agree, I’d open an issue in NethServer/dev and a PR on ns8-traefik. Would you prefer this as the new default, or as an opt-in action like set-trusted-proxies?

Thanks!

2 Likes
2

interesting

3

Hi Fabrizio,

I’m working on ns8-wazuh, so I’m interested too.

Today my decoder parses the Traefik access log, but it skips the User-Agent field because it is always "-". With the header kept, I could add rules for known scanners and Shellshock attempts. I could also lower the noise of my web scan rule when the request comes from a legit crawler.

I checked CrowdSec on one of my nodes. crowdsecurity/http-bad-user-agent is installed and enabled, but it doesn’t even show up in cscli metrics show scenarios. It never got a single event, while the other HTTP scenarios trigger fine.

I would vote for making it the default. User-Agent is not sensitive like Referer, Cookie or Authorization. With an opt-in action, most installs would keep the CrowdSec scenario dead without knowing it.

When the PR is open, I can test it with ns8-wazuh and CrowdSec.

cc @davidep

1 Like