How AfterAPI Was Tested Against an Old Client Test Suite

AfterAPI makes one main promise: an old client keeps working against a new API, and nothing it doesn’t cover gets touched. The tests were written to try to break that promise.

An old client with its own test suite

The example in examples/customer-api has three parts. There’s a v1 service, a v2 service with a different shape, and the old client’s test suite. The suite has twelve checks, and they’re strict. Field names, extra fields, status codes and error messages all count. It looks up customers, lists them with and without a limit, searches, creates a customer and reads it back, deletes it, and checks the errors for a missing customer and a missing email.

Three tests run that suite:

• Against v1 it passes all twelve checks, which shows the suite itself is right.
• Against v2 directly every one of the twelve fails. The old client really is broken.
• Against v2 through adapt all twelve pass.

A fourth test removes each of the five endpoint mappings in turn and runs the suite again. Something must fail every time, which proves every mapping is needed.

Traffic it doesn’t cover

Requests no endpoint matches must come out byte for byte. A test sends 400 random requests and responses through adapt and compares them with a direct call: binary bodies, odd paths, gzip, chunked and empty bodies, unknown lengths and every common status. The random set includes near misses too, such as a v1 path with the wrong method, or without its query parameter. Another test checks that unmatched responses stream, rather than waiting for the whole body.

When it can’t translate

Each way a translation can fail has its own test: a body that isn’t JSON, JSON with duplicate keys or invalid UTF-8, a body over the size limit, compression other than gzip, a template whose source is missing, a path value of .. or one with a slash in it. In each case the request or response must pass through exactly as it was sent, status and headers included, with a skipped line in the audit log.

The rest follows AfterPatch. A dead upstream gives a 502 and a slow one a 504. A dry run with mappings that would change the method, path, query, headers, status and body must leave every byte alone. The audit log must never contain a parameter, header value or body. And 32 clients sending 3,200 lookups at once must each get the right customer back.

What the review found

Before release, an independent review went through the code looking for ways to break it. It found no serious problems and ten smaller ones. The two that mattered most:

• A request that matched a mapping, was skipped, and then hit an upstream error could put the raw path in the audit log, customer ID included. The log now uses the endpoint’s pattern for every event of that request.
• A dry run could turn a slow response body into a 504, which broke the promise that a dry run changes nothing. AfterPatch had the same problem.

The others were edge cases: a client stalling mid-body could hold a connection open, a leftover query parameter could override a translated one, two header renames could overlap, and a number such as 1e400 was accepted in a template. Each finding now has its own regression test. The recorded run has 234 tests, all passing with Go’s race detector on, and CI runs them on Linux and macOS.

What it costs

Over loopback on a 2-CPU Linux machine, with 20,000 requests for each path:

So adapt adds about 67 µs at the median for a lookup and 82 µs for a list of three. As with AfterPatch, these are small responses on a small machine, so read them as an order of magnitude. The raw numbers are in test/results in the code repository.