Apache config debugging?

So I’m getting some strange results from my Apache virtual host config files for LemonLDAP::NG–noticed when I was trying to help @Shane_Treweek figure out why he couldn’t get a cert from Let’s Encrypt, but they also result in 403s to the system’s main hostname. Obviously something’s wrong, but it isn’t making a lot of sense where–all of those config files specify a virtual host name, and none of them matches neth-lemon.mydomain (which is the name of my test server), but they’re still interfering.

So I’m wondering if it’s possible (and if so, how) to tell Apache to be very verbose in its logging–what I’d really like to see, per-request, is “which configuration file told you how to handle this request, and where?” But I’m not finding a way to do this–I find reference to the LogLevel directive in httpd.conf, but even at its most verbose setting of trace8, this is what I get:

[Sat Jun 19 11:02:02.017604 2021] [http:trace4] [pid 17589] http_request.c(323): [client 192.168.1.116:56711] Headers received from client:
[Sat Jun 19 11:02:02.017679 2021] [http:trace4] [pid 17589] http_request.c(327): [client 192.168.1.116:56711]   Host: auth.familybrown.org
[Sat Jun 19 11:02:02.017693 2021] [http:trace4] [pid 17589] http_request.c(327): [client 192.168.1.116:56711]   User-Agent: curl/7.64.1
[Sat Jun 19 11:02:02.017702 2021] [http:trace4] [pid 17589] http_request.c(327): [client 192.168.1.116:56711]   Accept: */*
[Sat Jun 19 11:02:02.017825 2021] [rewrite:trace2] [pid 17589] mod_rewrite.c(470): [client 192.168.1.116:56711] 192.168.1.116 - - [auth.familybrown.org/sid#55844dad4948][rid#55844eb1d870/initial] init rewrite engine with requested uri /.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI
[Sat Jun 19 11:02:02.017860 2021] [rewrite:trace3] [pid 17589] mod_rewrite.c(470): [client 192.168.1.116:56711] 192.168.1.116 - - [auth.familybrown.org/sid#55844dad4948][rid#55844eb1d870/initial] applying pattern '(.*)' to uri '/.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI'
[Sat Jun 19 11:02:02.017959 2021] [rewrite:trace4] [pid 17589] mod_rewrite.c(470): [client 192.168.1.116:56711] 192.168.1.116 - - [auth.familybrown.org/sid#55844dad4948][rid#55844eb1d870/initial] RewriteCond: input='/.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI' pattern='!^/.well-known/acme-challenge/' => not-matched
[Sat Jun 19 11:02:02.017986 2021] [rewrite:trace1] [pid 17589] mod_rewrite.c(470): [client 192.168.1.116:56711] 192.168.1.116 - - [auth.familybrown.org/sid#55844dad4948][rid#55844eb1d870/initial] pass through /.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI
[Sat Jun 19 11:02:02.018138 2021] [authz_core:debug] [pid 17589] mod_authz_core.c(809): [client 192.168.1.116:56711] AH01626: authorization result of Require method GET POST OPTIONS: granted
[Sat Jun 19 11:02:02.018168 2021] [authz_core:debug] [pid 17589] mod_authz_core.c(809): [client 192.168.1.116:56711] AH01626: authorization result of <RequireAny>: granted
[Sat Jun 19 11:02:02.018184 2021] [core:trace3] [pid 17589] request.c(312): [client 192.168.1.116:56711] request authorized without authentication by access_checker_ex hook: /.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI
[Sat Jun 19 11:02:02.018253 2021] [core:trace3] [pid 17589] request.c(368): [client 192.168.1.116:56711] fixups hook gave 404: /.well-known/acme-challenge/O55YjlsLd5EaIVla3iKRiXwDUbrpgCfI
[Sat Jun 19 11:02:02.018365 2021] [http:trace3] [pid 17589] http_filters.c(1129): [client 192.168.1.116:56711] Response sent with status 302, headers:
[Sat Jun 19 11:02:02.018389 2021] [http:trace5] [pid 17589] http_filters.c(1136): [client 192.168.1.116:56711]   Date: Sat, 19 Jun 2021 15:02:02 GMT
[Sat Jun 19 11:02:02.018402 2021] [http:trace5] [pid 17589] http_filters.c(1139): [client 192.168.1.116:56711]   Server: Apache
[Sat Jun 19 11:02:02.018414 2021] [http:trace4] [pid 17589] http_filters.c(958): [client 192.168.1.116:56711]   Location: http://auth.familybrown.org/lmerror/404
[Sat Jun 19 11:02:02.018423 2021] [http:trace4] [pid 17589] http_filters.c(958): [client 192.168.1.116:56711]   Content-Length: 223
[Sat Jun 19 11:02:02.018432 2021] [http:trace4] [pid 17589] http_filters.c(958): [client 192.168.1.116:56711]   Content-Type: text/html; charset=iso-8859-1

Hi Dan, IIRC is used LogLevel debug rewrite:trace8 to debug a while ago.

Here the very clear Apache documentation (to be clear : being sarcastic here :slight_smile: )
https://httpd.apache.org/docs/2.4/mod/core.html#loglevel

1 Like

Thanks. It looks like the syntax you gave allows you to specify one log level generally, and another for certain modules. But even at trace8 globally, I’m getting less complete logging than I’d really like to see here.

Hearing nothing else, I’m going to have to conclude that the kind of logging I’m looking for just isn’t available, which is kind of unfortunate. But fortunately, I was able to get this sorted, and it still looks like trace* as a log level could be useful for other things.