Phase 12 — The deploy boundary
31 AUG AT 11:29 AM

Phase 12 — The deploy boundary

0 LOVES 5 VIEWS
Something rebuilt the production binary with uncommitted code in it, and the next restart would have run it. Configuration was already separated; executables were not.

Documented procedure, not drift

At 09:55 on 2026-08-21, something rebuilt the production bridge binary with uncommitted, unreviewed code in it. The live supervisor pointed at exactly that path, so the next restart of either production bridge would have executed it.

It was noticed only because the agent had been checking the file’s modification time after each of its own builds — all of which correctly targeted a different filename — and it was repaired by rebuilding from the committed tree through a temporary worktree.

⚠ Nothing detected it, nothing warned, and the writer was never identified. No tracked run configuration writes that path, so it was an untracked local config or a stray command. Which means it will happen again.

The shape of it, measured from the live supervisor’s own environment: all four executables — the bridge, the approval gate, the MCP binary and the supervisor itself — resolved into the source tree. Meanwhile the environment file correctly pointed at the production directory.

⭐ Configuration was already separated and executables were not, and that asymmetry is the entire finding. It is the kind of gap that survives review indefinitely because the half you look at is right.

⚠ And it was documented procedure, not drift. The bring-up runbook instructed exactly that: build in the source tree, then point production at the results. Nobody deviated. So the runbook was rewritten as part of the fix rather than after it — a fix that leaves the instructions intact is a fix that expires.

The failure mode inverted

Production now runs binaries from a deploy directory, and the source tree is no longer on the path to anything running.

Proven rather than asserted: after the cutover, building in the source repository left the deployed binary byte-identical. Nothing about the deployed artefact moved, which is the only evidence that the boundary is real rather than declared.

⭐ And the failure mode inverted, which is the durable win. Forgetting to run the deploy step now means production keeps the version it already has. Forgetting used to be how unreviewed code shipped. The same human lapse now produces staleness instead of an incident — and staleness is visible, boring, and cheap.

The shared machine, incidentally, is a decision rather than an oversight — development and production live on one box on purpose, and this phase had to survive that rather than wish it away. Separation of executables is what makes the shared machine safe; separating the machines was never on the table.

⚠ One residual worth keeping visible: the supervisor deploys cheaply, because stopping it leaves the bridges running and the new one adopts them. Bridge binaries still need a restart to take effect — so a deployed bridge binary is not a running one until someone says so.

Pushing Tin — managing a bridge fleet from inside the product
Pushing Tin — managing a bridge fleet from inside the product
Aug 29, 2026 Pushing Tin
← Back to Pushing Tin