Case Study: Catching a Renamed Field Before Release With AfterTest and AfterDiff

This is a worked case on two small stand-in versions of an orders service. Every command and every line of output below comes from a real run of aftermarket 0.3.0.

The question

A team is about to replace version 1 of an orders service with version 2. The release notes say nothing about the API. Clients they don’t control call it every day. Before switching, they want to know: does v2 answer the same questions the same way?

Reading the code won’t settle that. Asking both versions will.

Recording

First they put record in front of v1. It changes nothing on the way through. The only config adds one field to blank, since order answers carry customer email addresses:

version: 1
redact:
  json: [customer.email]

$ aftermarket record check --config orders-record.yaml
orders-record.yaml:
  records: all traffic
  redacts headers: Authorization, Cookie, Proxy-Authorization, Set-Cookie, X-Api-Key
  redacts query keys: access_token, api_key, apikey, client_secret, password, sig, signature, token
  redacts JSON fields: customer.email
  max_body 1048576 bytes, upstream_timeout 30s, max_cases no limit
ok

Then they send the usual calls through it, each with a real bearer token: two order lookups, a missing order, the open-orders list with an api_key in the query string, and one new order.

$ aftermarket record --config orders-record.yaml --upstream http://127.0.0.1:9201 --out orders.yaml --listen 127.0.0.1:8201
aftermarket record 0.3.0: http://127.0.0.1:8201 -> http://127.0.0.1:9201, writing orders.yaml
aftermarket record: stopped, 5 case(s) written to orders.yaml

The first case in the file:

  - name: "1 GET /orders/1001"
    request:
      method: "GET"
      path: "/orders/1001"
      headers:
        "Accept": "*/*"
        "Authorization": "REDACTED"
        "User-Agent": "curl/8.5.0"
    upstream:
      status: 200
      headers:
        "Content-Type": "application/json"
      body: "{\"id\":1001,\"status\":\"open\",\"customer\":{\"name\":\"Ada Lovelace\",\"email\":\"REDACTED\"},\"total\":39.8,\"created_at\":\"2026-10-20T08:58:05.765900609Z\"}"

The list call was saved as /orders?status=open&api_key=REDACTED. A search of the whole file for the token, the key and both email addresses finds nothing.

Replaying

Now the same requests go to both versions. --set-header puts a real token back in place of REDACTED:

$ aftermarket replay --fixture orders.yaml --a http://127.0.0.1:9201 --b http://127.0.0.1:9202 \
    --set-header "Authorization: Bearer live-token-123"
5 case(s): 2 same, 2 different, 1 skipped, 0 error(s)
...
### 1 GET /orders/1001

`GET /orders/1001`

- created_at: "2026-10-20T08:58:06.10516018Z" vs "2026-10-20T08:58:06.105708202Z"
- currency: only in B ("EUR")
- total: only in A (39.8)
- total_amount: only in B (39.80)

created_at is the time of the call, so it differs every time. Add --ignore created_at and what’s left is the real change:

### 1 GET /orders/1001

`GET /orders/1001`

- currency: only in B ("EUR")
- total: only in A (39.8)
- total_amount: only in B (39.80)

## Skipped

- 5 POST /orders: POST changes data; use --include-writes to send it

Some things are missing from that list, and they matter as much. v2 writes its keys in a different order, 39.80 instead of 39.8, and the customer object the other way round. None of that is a difference, because JSON is compared by meaning. The 404 for the missing order and the open-orders list came out the same. The POST wasn’t sent at all, since replay sends only reads unless told otherwise.

So the finding is precise: v2 renamed total to total_amount and added currency. Any client reading total would break on release day.

Repairing and proving it

The right fix is in v2 itself. Until it ships, two AfterPatch operations put the old shape back:

version: 1
rules:
  - id: order-total-renamed
    description: v2 renamed total to total_amount and added currency.
    when:
      method: GET
      path: /orders/*
      status: 200
      content_type: application/json
    response:
      json:
        - rename: {from: total_amount, to: total}
        - delete: {path: currency}

Then the same recording is replayed against v1 and against v2 behind that rule:

$ aftermarket replay --fixture orders.yaml --a http://127.0.0.1:9201 --b http://127.0.0.1:8202 \
    --ignore created_at --set-header "Authorization: Bearer live-token-123"
5 case(s): 4 same, 0 different, 1 skipped, 0 error(s)

Exit code 0. The repair is checked against the calls clients really make, not against a test someone wrote after the fact.

Keeping the recording

The recording is an ordinary fixture file, so it can go into CI as it is. Run against the repair rule, it checks that the rule leaves answers in the old shape alone:

$ aftermarket patch test --config orders-rules.yaml --fixture orders.yaml
PASS  1 GET /orders/1001
PASS  2 GET /orders/1002
PASS  3 GET /orders/9999
PASS  4 GET /orders
PASS  5 POST /orders
5 case(s), 0 failed

The full formats are in the AfterTest and AfterDiff reference. The use cases page has shorter examples.