See every API your server calls (and every call it receives) without changing a line of code
How passive HTTP capture on a Linux host works with AF_PACKET and a BPF filter, what it can and cannot see with TLS, and how to turn it into per-endpoint latency, error rates and a service map.
Instrumenting an application for APM means picking a vendor SDK, adding it to every service, redeploying, and hoping the framework version is supported. There is an older trick that gets you most of the value with none of that: listen to the network interface.
What the host can see
A raw socket opened with socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL)) on Linux receives a copy of every frame that crosses any interface, the same tap tcpdump uses. On its own that is expensive on a busy box, so the first thing to do is attach a classic BPF program with setsockopt(SO_ATTACH_FILTER). The nine-instruction program that tcpdump -dd tcp prints accepts IPv4 and IPv6 frames whose next header is TCP and rejects everything else inside the kernel, before a single byte is copied to user space. The agent uses exactly that program, so UDP, ARP and ICMP never reach Python.
What remains is a stream of TCP segments. You do not need full stream reassembly to get useful data, because the interesting bytes sit at the start of a segment: an HTTP/1.x request line begins a client segment, a status line begins a server segment, and a TLS ClientHello is the first record on a fresh connection. Keying by the 4-tuple (source address, source port, destination address, destination port) and pairing a request with the next response on the reversed tuple gives you a latency sample per request. From that you can reconstruct:
- Plain HTTP/1.x: method, path (query string stripped),
Host, status code, bytes each way, and the time from the request segment to the first byte of the response. That is enough for per-endpoint counts, 4xx/5xx rates and p50/p95/max latency. Absolute-form request targets, which proxies use, are normalised to a path. - TLS: not the payload. The
ClientHello(handshake type 1 inside a record of content type 22) carries the Server Name Indication extension (type 0) in clear text on TLS 1.2 and 1.3, so you still learn which host the server is talking to, on which port, how many connections, how many bytes each way, and the round-trip time from ClientHello to ServerHello. That is your outgoing dependency map: Stripe, your database host, an S3 endpoint, a third-party API nobody knew about. Encrypted ClientHello (ECH) hides SNI, but adoption on API endpoints is still rare. - Local services: traffic between your reverse proxy and your app on
127.0.0.1:8000is usually plain HTTP, so you get full request-level detail for your own application even when the public side is TLS.
What it cannot see
HTTP/2 multiplexes streams inside one connection and is almost always inside TLS; without keys it shows up as an opaque flow with byte counts. gRPC likewise. Cleartext HTTP/2 (h2c) is binary-framed, so the request-line parser skips it too. For those you want either the reverse proxy's access log (nginx, Apache and Caddy formats, which the same agent tails from the standard paths) or an in-process SDK. Two more limits worth knowing: a request that spans several segments is only parsed if the request line and Host header are in the first 2 KB of the first segment, and keep-alive connections that pipeline requests are attributed by order, which is correct for HTTP/1.1 because responses must arrive in request order.
From packets to something useful
Raw requests are noise. The useful transformation is:
- Normalise paths so cardinality stays bounded:
/users/1234/orders/550e8400-e29b-41d4-a716-446655440000becomes/users/{id}/orders/{uuid}. Purely numeric segments become{id}, UUIDs{uuid}, hex strings of 16 or more characters{hash}, long base64-looking segments{token}. Everything else is kept verbatim so/v1/usersand/v2/usersstay distinct. - Aggregate per reporting interval: count, 4xx and 5xx counts, and p50/p95/max latency per
(direction, host, method, path). Percentiles are computed from the raw samples in that interval, not from averages of averages. - Keep a small sample of the slowest and failing requests with their timing, so an incident shows evidence, not just a number.
- Roll up hourly for history, and build the service map from the outgoing side: this server talks to these hosts, this often, this slowly. The reverse-DNS name of each destination is resolved once and cached, so the map shows
api.stripe.com, not an IP.
Doing it with the Vigil agent
The agent is a single Python file that uses only the standard library, so it runs on any Linux with Python 3.8 or newer. Installed as root it enables capture automatically; without root it still reports metrics, processes and connections from /proc/net/tcp, but not per-request detail:
curl -fsSL https://puzaricloud.in/install-server-agent.sh | sudo sh -s -- --token vgs_YOUR_TOKEN
The server's APIs tab then shows incoming endpoints with p95 and 5xx rates, the Network tab shows outgoing hosts with connections and bytes, and the Services tab draws the map. Rules on http_error_pct, http_p95_ms and new_destination open incidents, so a new outbound host appearing at 3 am pages you, which is one of the cheaper ways to notice a compromised box.
For request traces across services, drop in the Python or Node SDK; it reports to the same local agent on 127.0.0.1:9111 and links spans to the captured endpoints.
Privacy and safety notes
The parser reads the request line and the Host header, and the status line of the response. It never parses other headers or any body, so Authorization, Cookie, query strings and payloads are not in memory beyond the kernel buffer and never leave the machine. Only aggregates and a handful of slow-request samples (method, path, status, timing) are reported. If your compliance rules forbid even that, turn capture off per server and rely on access logs and SDK spans instead.