afterhook

How AfterHook Was Tested to Turn Every Change Into One Event

AfterHook makes four promises. Every change gives exactly one event. Nothing is lost while the receiver is down or across restarts. A source that didn’t change by meaning sends nothing. And a half-written file is never reported. The tests go after each one, with a fake clock so a test can jump past retry waits without sleeping through them.

One change, one event

A random test makes 200 edits to a JSON file. Some change a real field. Others only move a timestamp that the config ignores, or write the same document with other spacing, key order, or 2.0 for 2. The test keeps its own count of the edits that matter, independent of AfterHook, and every round must give exactly that many events: one or none. At the end the receiver must hold exactly that number, with sequences 1, 2, 3 and no id twice. Every signature must check out. Three more quiet checks must send nothing.

Nothing lost

The receiver answers 503 to the first seven requests. Meanwhile the file changes six times, and AfterHook is restarted twice on the same state file, the way a crash or a deploy would do it. Every event must still arrive, in order. Repeats are allowed, since delivery is at least once, but a repeat must carry the same id as the first copy.

A second test stops AfterHook after it found a change but before it saved, by putting the old state file back. On restart it finds the change again, and the id must match.

Half-written files

Here the test plays the writer. While AfterHook waits between its two reads of a file, the test appends a line. That change must wait for the next check. A JSON document cut off in the middle must not be reported at all. In a folder, a file still growing is left out until it settles.

What the review found

Before release, an independent review tried to break AfterHook. The serious find was a leak. When a comparison couldn’t go field by field, it fell back to lines of text, and lines aren’t redacted. An HTTP endpoint that went from JSON to an HTML error page sent its old JSON as a raw line, token included. sqlite3 prints nothing for a query with no rows, so the first row of a table went out as text, password column and all. A plain app.ini file sent password = ... as it was.

All three are closed. Empty command output now counts as an empty list. JSON that turns into something else is reported by size and hash only. Lines of text get key = value pairs redacted the same way fields are.

The review found six medium issues too:

  • A file deleted and written again between checks gave two events for no change.
  • A YAML or .env file cut short still reads, so it was reported.
  • One busy file held up a whole folder.
  • Event ids repeated after the state file was moved aside.
  • A command whose child process kept its output open could outlast the timeout.
  • hook check printed webhook URLs with their path, where services like Slack keep the secret.

Now a removal or a big shrink must show up on two checks in a row, the busy file is left out on its own, and each baseline adds a random value to the ids. A timed-out command loses its whole process group, and check prints only the scheme and host.

Each finding has its own test.

After the release

The last check runs the downloaded binary against the example receiver, with a real SQLite table and a receiver that goes down in the middle. That turned up one more thing. An event that waited in the state file through the outage went out indented, because the state file is written that way. The signature still checked out, but the same event id arrived as different bytes. 0.5.1 sends it exactly as it was first built, and a test now compares the two byte for byte.

The recorded run has 333 tests, all passing with Go’s race detector on. CI runs them on Linux and macOS, and the release smoke test runs hook run --once on every system it ships for. The raw results are in test/results in the code repository.