Skill
apache
apache · current version v2
Troubleshoot and support an Apache HTTP Server (httpd) — validate config, read modules and vhosts, and resolve problems on Docker, bare metal, systemd (Debian apache2 or RHEL httpd), or Kubernetes. Covers apachectl/httpd -t/-S/-M/-V, graceful reload, the MPM, and 403/500/502/503/SSL failures. Use when someone reports Apache will not start or reload, returns 403/500/502/503, serves the wrong vhost, leaks processes, or fails a TLS handshake.
13 downloads · published 2026-09-02
What this grants
- skill apache
Skill Card
- License or terms: check the skill's own repository for license details (apache on GitHub).
Security Audits
- NanoInfra Scanner PASS no issues found
- VirusTotal PASS no engines flagged this file (full report)
Version history
| Version | Published | Status |
|---|---|---|
| v2 | 2026-09-02 | published |
| v1 | 2026-09-02 | published |
Files
SKILL.md(9445 bytes)
SKILL.md
raw | preview
--- name: apache description: Troubleshoot and support an Apache HTTP Server (httpd) — validate config, read modules and vhosts, and resolve problems on Docker, bare metal, systemd (Debian apache2 or RHEL httpd), or Kubernetes. Covers apachectl/httpd -t/-S/-M/-V, graceful reload, the MPM, and 403/500/502/503/SSL failures. Use when someone reports Apache will not start or reload, returns 403/500/502/503, serves the wrong vhost, leaks processes, or fails a TLS handshake. --- # Apache HTTP Server (httpd) — Troubleshooting & Support A support runbook, not a tutorial. Work top to bottom: find which packaging you are on (this is Apache's biggest trap), **configtest before you reload**, and treat every reload/restart as a deliberate act. Commands and output below were run against Apache 2.4.68. Apache is on the **2.4** line and has been for years — there is no Apache 7/8/9, and 2.2 is long EOL. The axes that actually matter for support are the **packaging** (below) and the **MPM** (prefork / worker / event), which decides the whole process/threading model. ## 0. The packaging trap — identify it first Run the **detect-platform** skill first — the OS/package family it reports (Debian vs RHEL) is exactly what decides the packaging below, and its SELinux/AppArmor finding drives the permission diagnostics later. The same server ships three ways, with different binary names, control commands, and config layouts. Getting this wrong is why a documented command "does not exist": | Packaging | Binary / control | Config root | Enable a module/site | |---|---|---|---| | Debian/Ubuntu | `apache2`, `apache2ctl` | `/etc/apache2/` (`sites-enabled/`, `mods-enabled/`) | `a2enmod` / `a2ensite` + reload | | RHEL/Fedora | `httpd`, `apachectl` | `/etc/httpd/` (`conf.d/`, `conf.modules.d/`) | drop a `.conf`, reload | | Upstream Docker image | `httpd`, `apachectl` at `/usr/local/apache2/bin/` | `/usr/local/apache2/conf/httpd.conf` | edit `httpd.conf` | Detect which: ```bash command -v apache2ctl apachectl httpd apache2 2>&1 # which names exist here httpd -V 2>/dev/null || apache2ctl -V # -V prints HTTPD_ROOT + SERVER_CONFIG_FILE ``` `-V` is authoritative — it printed `HTTPD_ROOT="/usr/local/apache2"` and `SERVER_CONFIG_FILE="conf/httpd.conf"` on the upstream image, and it also prints the MPM. Below, `apachectl` means "your control binary" — substitute `apache2ctl` on Debian/Ubuntu. ## 1. Version, MPM, modules ```bash httpd -v # Server version: Apache/2.4.68 (Unix) httpd -V | grep -E 'Server MPM|HTTPD_ROOT|SERVER_CONFIG_FILE' # Server MPM: event <- event | worker (threaded) | prefork (process-per-request) httpd -M # loaded modules; "(shared)" = DSO via LoadModule, "(static)" = built in ``` The **MPM** is a frequent root cause. `prefork` (still forced by old `mod_php`) is memory-heavy and caps concurrency at `MaxRequestWorkers` processes; `event`/`worker` are threaded. A box "running out of memory under light load" is often prefork with a high `MaxRequestWorkers`. ## 2. Where things are, per deployment **Docker (upstream image)** ```bash docker exec -it <container> httpd -t docker exec -it <container> httpd -S # vhost map (see §3) docker logs --tail 200 -f <container> # logs go to /proc/self/fd/1,2 -> docker logs ``` **systemd (bare metal or VM)** ```bash systemctl status apache2 2>/dev/null || systemctl status httpd journalctl -u apache2 -n 200 --no-pager 2>/dev/null || journalctl -u httpd -n 200 --no-pager apachectl configtest # alias for httpd -t tail -f /var/log/apache2/error.log 2>/dev/null || tail -f /var/log/httpd/error_log ``` **Bare metal — host limits and SELinux** ```bash # SELinux is the number-one RHEL Apache surprise: it blocks non-standard ports, proxying out, # and serving from a non-default docroot. getenforce # a denied action leaves an AVC: ausearch -m avc -ts recent 2>/dev/null | tail # the usual fixes are booleans/ports/labels, not disabling SELinux: # setsebool -P httpd_can_network_connect 1 # allow mod_proxy to reach a backend # semanage port -a -t http_port_t -p tcp 8080 # allow Listen 8080 # chcon -R -t httpd_sys_content_t /srv/www # label a custom docroot # Open files / worker ceiling: cat /proc/$(pgrep -o httpd 2>/dev/null || pgrep -o apache2)/limits | grep 'open files' ``` **Kubernetes** ```bash kubectl get pods -l app=httpd kubectl exec -it <pod> -- httpd -t kubectl logs <pod> --tail 200 -f ``` ## 3. The commands and arguments you will reach for ```bash httpd -t # configtest: syntax check. ALWAYS before a reload. (apachectl configtest) httpd -S # dump the VirtualHost map + ServerRoot + DocumentRoot + ErrorLog + ports. httpd -M # list loaded modules. httpd -V # version, MPM, compile-time paths and -D defines. httpd -D DUMP_VHOSTS # same vhost map as -S; -D DUMP_MODULES == -M; -D DUMP_RUN_CFG for run config. apachectl graceful # reload: finish in-flight requests, then re-read config. No dropped conns. apachectl graceful-stop # stop, but let in-flight requests finish. apachectl restart # hard restart: drops in-flight requests. apachectl -k start|stop|restart|graceful # the -k form, identical verbs. ``` Verified: `httpd -t` → `Syntax OK`; `httpd -k graceful` exits 0. The `AH00558: Could not reliably determine the server's fully qualified domain name` notice is harmless — set `ServerName` globally to silence it; it is not the cause of a failure. ## 4. Diagnostics — read-only first ```bash httpd -t # loads? names the file:line on failure httpd -S # which vhost wins for which name:port — the #1 "wrong site" tool httpd -M | grep -E 'ssl|proxy|rewrite|php|mpm' # is the module the config needs actually loaded? ps -o pid,ppid,rss,cmd -C httpd # (or -C apache2) parent + children; RSS to spot prefork bloat ss -ltnp | grep -E 'httpd|apache' # listening where you expect? tail -n 100 /var/log/apache2/error.log # or /var/log/httpd/error_log — the AH##### code is greppable ``` Every Apache error carries an `AHxxxxx` code; grep the code, not the prose — it maps to an exact cause in the docs. ## 5. Common problems → resolution **Won't start / reload** - `httpd -t` names the file and line. A common one: `AH00526` (syntax) or a `LoadModule` for a module not installed — on Debian that is a missing `a2enmod`, on RHEL a missing package. - `(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80` — another process holds the port; `ss -ltnp | grep :80`. - Started but a directive "does nothing": it is in a file not `Include`d, or inside a `<VirtualHost>` that does not match the request. `httpd -S` shows the vhost that actually serves a given name:port. **403 Forbidden** - Filesystem: the Apache user (`www-data` on Debian, `apache` on RHEL) cannot read the docroot. - Config: missing `Require all granted` in the `<Directory>` (2.4 denies by default), or `Options` without an index file. - SELinux (RHEL): the docroot is unlabeled — `chcon -R -t httpd_sys_content_t <dir>`; confirm with `ausearch -m avc`. **500 Internal Server Error** - The app behind Apache errored, or a broken `.htaccess`/rewrite. The **error log** has the real cause (a PHP fatal, a `mod_rewrite` loop `AH00124`). A 500 with nothing in Apache's log = the handler/CGI/FPM logged it instead. **502 / 503 (as a reverse proxy)** - 502 `AH00898`/`AH01102` from `mod_proxy`: the backend refused or spoke bad HTTP. `curl` the backend directly from the Apache host. On RHEL, `setsebool -P httpd_can_network_connect 1`. - 503 Service Unavailable: `mod_proxy_balancer` has all workers in error state, or `MaxRequestWorkers` is exhausted (every worker busy) — check `ps` count vs the directive and the server-status page if `mod_status` is enabled. **Processes/memory climbing** - prefork MPM with a high `MaxRequestWorkers` is process-per-request: each child holds its memory. Either lower `MaxRequestWorkers` to fit RAM, or move off prefork (possible once nothing requires it, e.g. PHP via FPM instead of `mod_php`). **TLS handshake fails** - `httpd -M | grep ssl` — is `mod_ssl` even loaded? Then check `SSLCertificateFile` / `SSLCertificateKeyFile` (and the chain) match: `openssl x509 -noout -modulus -in cert | openssl md5` vs the key. `openssl s_client -connect host:443 -servername <name>` shows what is served per SNI. ## 6. Before you run anything that changes serving `graceful`/`restart` and config edits change what users get, and on a nanoinfra deployment they resolve to a `mutate.remote` capability — approval interactively, a standing grant unattended. - **`httpd -t` / `apachectl configtest` before every reload.** A `restart` on a broken config exits the server (site down); validating first turns that into a refused reload. - `apachectl graceful` over `restart`: graceful drains in-flight requests; restart drops them. - On RHEL, reach for SELinux booleans/labels before `setenforce 0` — disabling SELinux to "fix" a 403 hides the problem and removes the protection. - On Debian, `a2enmod`/`a2ensite`/`a2dissite` edit symlinks under `mods-enabled`/`sites-enabled` — the reload still has to follow, and it is still a deliberate change.