How AfterPatch Was Tested Before Its First Release

A repair tool earns trust in one way: it has to leave alone everything it isn’t supposed to touch. So most of the AfterPatch tests don’t check the repairs. They check what happens around them.

Every number here comes from the recorded results in the repository’s test/results folder, written by a script, not by hand.

The headline test

The first test sends 400 random requests straight to a test server, then sends the same 400 through AfterPatch. The random parts are the methods, paths with odd characters, query strings, headers, binary bodies up to 70 KB, gzip and chunked responses, and status codes from 200 to 503. For every pair, the server must see the same request and the client must get the same status, headers and bytes. Rules are loaded, but none of them match.

A second test sends query strings that a well-behaved client never would: ?a=1;b=2, a lone %, broken escapes. They have to reach the server exactly as written, whether a rule matches the request or not.

Then the failure cases

• Broken bodies. Truncated JSON, plain text, trailing garbage and empty bodies must all come back byte for byte, with a “skipped” line in the audit log.
• Bodies too big to change. A 100 KB body under a 4 KiB limit has to pass through untouched.
• Compression. Gzip bodies get patched and compressed again with a correct length. Brotli and broken gzip get passed on as they are.
• JSON that wouldn’t survive a rewrite. Duplicate keys, invalid UTF-8 and unpaired escapes are left alone.
• A dead upstream gets a 502, and a slow one a 504 within the time limit. That includes an upstream that sends half a body and stops.
• Paths with ... A rule for /public/** must not fire for /public/../admin, and a path rewrite must never produce a .. segment, whatever the query string holds.
• Rule order. Three rules that all change the same field must give the same answer 200 times in a row.
• Dry run. With rules that would rewrite the path, headers and body, the server and client must see exactly what they’d see without the proxy.
• The audit log must not contain a single secret value, even when the upstream’s error message quotes one.
• Load. 32 clients sending 4,800 requests at once, half patched and half not, with every answer checked. The whole suite also runs with Go’s race detector on.

The example has to fail first

The legacy-customer example is a test too. The old client must fail against the new service directly, and still fail through AfterPatch with unrelated rules. It must work with the two example rules. Then it must break again when either rule is taken out, which proves both rules are needed.

A second pair of eyes

Before release, an independent review went through the code looking for ways to break it, and it found real problems. The worst were query strings with semicolons getting dropped, and path rewrites that could be steered with .. in a query value. Everything it found was fixed, and each finding now has its own test. The recorded run has 157 tests and subtests, all passing, on Linux and macOS in CI.

What it costs

The example over loopback, on a 2-CPU Linux machine, with 20,000 requests for each path:

So the proxy adds about 57 µs at the median when it only passes traffic on, and about 104 µs when it rewrites the path and four JSON fields. The 99th percentile numbers are noisier, because the client, the proxy and the service share two CPUs. These figures come from a small test machine and a tiny response, so treat them as an order of magnitude, not a benchmark.