Shadowing the JSON Server Rewrite With Reads and Writes

JSON Server gives you a full REST API from one JSON file, and a lot of front-end projects and test suites lean on it. Version 1.0, still in beta, is a rewrite, and its own README asks you to expect breaking changes. That makes it a good subject for shadow mode: a team on 0.17.4 wants to know what its client would see on 1.0 before the switch, including for the requests that change data.

The primary here is json-server 0.17.4 and the candidate is 1.0.0-beta.15, each with its own copy of the same data file. The client is a short script written against 0.17: it reads posts with filters, sorting and paging, reads a nested route, then creates a post, edits one, replaces the profile, deletes a comment and reads back the post it created by the id it was given. Every command and output below comes from a real run of the downloaded aftermarket 0.10.1 on Linux, recorded with its script under test/results/case-studies/shadow/json-server in the code repository.

Noise first

0.17.4 against a second 0.17.4, writes included, gave 19 case(s): 19 same. No timestamps, no generated values. The data file starts with ids 1 to 6, and both copies hand out 7 for the new post, so there was nothing to ignore. JSON Server 0.17 also gzips its answers when the client asks, and this client does; shadow compares the decoded body, so that didn’t show up either.

Reads only, the default

aftermarket shadow --primary http://127.0.0.1:9201 --candidate http://127.0.0.1:9202 \
  --listen 127.0.0.1:8290 --report report.md --json report.json

Without --include-writes, only GET and HEAD go to the candidate. The report counts the rest:

- 19 request(s) served from A; 13 copied to B
- Not copied: 0 outside when:, 0 left out by sampling (1), 6 writes without --include-writes, 0 dropped because the queue (64) was full, 0 that couldn't be compared safely

All 13 reads differed. That’s safe to run against a candidate that shares nothing with production, and it already tells most of the story.

Writes too, on the candidate’s own data

aftermarket shadow --primary http://127.0.0.1:9201 --candidate http://127.0.0.1:9202 \
  --listen 127.0.0.1:8290 --report report.md --json report.json \
  --include-writes --workers 1

--workers 1 sends the copies one at a time, so the candidate gets the writes in the order the primary did. The candidate has its own candidate.json; pointed at the primary’s file, every write would happen twice. All 19 requests were copied and all 19 differed, while the client got 0.17.4’s answers and passed every step.

What changed

Ids are strings in 1.0. Every read shows it:

### 2 GET /posts/1

`GET /posts/1`

- header Content-Type: "application/json; charset=utf-8" vs "application/json"
- id: 1 vs "1"

The Content-Type lost its charset too, on every answer. Harmless for most clients, but a strict one comparing the whole header would notice.

Three query parameters the old client used don’t mean the same thing any more:

`GET /posts?_sort=views&_order=desc`
- body: 6 items vs 0 items

`GET /posts?_page=1&_limit=2`
- body: a list vs an object

`GET /posts?title_like=report`
- body: 2 items vs 0 items

1.0 sorts with _sort=-views and pages with _page and _per_page, returning an object with first, next, pages, items and data instead of a bare list. _order and title_like aren’t 1.0 parameters, and those reads came back empty instead of failing. An empty list is the hardest kind of change to catch by hand, because nothing errors. The nested route /posts/1/comments answered 404, and ?_embed=comments came back with empty comment lists. The comments in the data file point at posts with numbers, and 1.0 matches them against string ids; with "postId": "1" the embed works.

The created id

This is the part that only shows up with writes. The POST worked on both sides, and the generated ids differ, which is expected:

### 11 POST /posts

`POST /posts`

- header Content-Type: "application/json; charset=utf-8" vs "application/json"
- id: 7 vs "E8UYwdWEZSM"

Then the client did what clients do and used the id it got back, 7, which came from the primary:

### 16 GET /posts/7

`GET /posts/7`

- status: 200 vs 404
- header Content-Type: "application/json; charset=utf-8" vs "application/json"
- authorId: only in A (2)
- error: only in B ("Not Found")
- id: only in A (7)
- title: only in A ("Release notes for 2.0")
- views: only in A (0)

The PATCH and DELETE on post 7 got the same 404. The candidate does have the new post, under its own id, so the final read of all posts showed 6 items vs 7 items: the primary deleted its copy, the candidate never got a request for the one it made. Neither version did anything wrong here. The same happens whenever the server picks the ids, and it’s worth knowing before reading the report: after the first create, every follow-up on that record differs for that reason alone. Ignoring id hides the first difference but not the 404s.

One more write difference: deleting a comment answers {} in 0.17.4 and the deleted comment in 1.0. A client that checks for an empty object would read that as a change.

What a team would take from it

A list of what the client has to change: ids as strings, the new sort and paging parameters, no _like filters or nested routes, a paging object instead of a list. The writes run adds the part a recorded replay can’t give you without care, that a server choosing its own ids makes later requests diverge. For that, judge writes by their status codes and by the shape of the answers, and keep the reads as the main signal.

The Datasette case study shows the noise run with real noise in it, and the shadow mode recipes cover ignore rules for ids. The full options are in the AfterTest and AfterDiff reference.