How AfterConfig Was Tested to Never Write a Vendor Config File
AfterConfig makes four promises, and the tests go after each one. It never writes the vendor’s file. It never changes what the overlay doesn’t name. A failed apply leaves the old output alone. And it never prints a secret.
The vendor’s file
The test makes the vendor’s file read-only and runs every command against it. Then it names that file as the output in six ways: directly, through a symbolic link, through a hard link, through a linked folder, as a relative path, and with a .. detour. Every one must be refused, and the file must not change by a byte.
Failed writes are tested too. Writing into a folder that doesn’t exist, or onto a folder, must fail and leave no temporary file behind. The output takes the vendor file’s permissions, so a config readable only by its owner stays that way.
Nothing else changes
A random test builds 400 JSON and YAML documents full of awkward values. They hold numbers like 1.50 and 12345678901234567890, strings like "true", "007", a: b and x #y, line breaks, quotes, and keys with spaces and accents. It sets one random field in each. The output has to read back. The changed field must hold the new value, and every other field must hold its old value in its old place.
A second random test writes 1,000 YAML documents and reads each back with the same reader. Then come the exact cases:
- An overlay with nothing to change must give back the vendor’s file byte for byte, whatever its layout, including one-line JSON and Windows line endings.
- A
.envfile must keep every comment, blank line and untouched line. - The example’s overlay, applied twice, must give the same bytes.
What the review found
Before release, an independent review went looking for ways to break AfterConfig. It found one serious bug. A rename into a child of itself, a to a.b, silently deleted a and everything under it. It’s now refused with a clear message.
The rest were smaller, and most were about a file meaning something different to the app than intended:
- A YAML string ending in
:was written without quotes, so the result couldn’t be read. - A YAML key written as
80:came back as"80":, which is text to readers that type keys, not a number. - Renaming a
.envvalue like'lit $USER'rewrote it in double quotes, where a shell expands$USER. - A secret renamed to an innocent path such as
db.pwshowed up in the diff, and a failing fixture printed whole files, passwords included.
The fixes go further than the cases found. Every result is now read back before it’s written. Keys keep the style they were written in. .env values are quoted so nothing gets expanded, and a secret stays hidden wherever rename or copy moves it. Each finding has its own test, and the recorded run has 296 tests, all passing with Go’s race detector on. CI runs them on Linux and macOS, and the release smoke test runs the example upgrade on every system it ships for.
The upgrade test
The example has a vendor config at 1.0 and 1.1, and an overlay with seven changes. On 1.0 all seven must apply. On 1.1, which raised the default timeout and switched logging to JSON itself, apply must stop at exactly those two lines. It must write nothing and leave the old output byte for byte as it was. The raw results are in test/results in the code repository.