I read David Cramer’s “A Little Determinism” this morning and found myself nodding along for most of it. David (one of the founders of Sentry) writes about Junior, the agent Sentry built in-house that turns Slack threads into GitHub issues and these days goes all the way to opening the pull request. His problem is one that anyone who has run an agent for more than a week will recognise: how do you get it to behave the way a sensible person would, every single time, without tying it up in so many rules that it stops being useful? And his answer to the obvious fix (write it in the system prompt, or in AGENTS.md, or in a skill) is that “it almost never is”.

I couldn’t agree more. And I’ve got the scars to prove it 🙂

What Sentry Actually Did

The interesting bit is where Junior’s rules live. Every request from its sandbox goes out through an egress proxy, and the proxy is where the behaviour gets enforced. The sandbox never holds any credentials: the proxy checks each request against a policy and only then injects the token. If the agent tries to open a pull request with a plain gh pr create, the proxy spots the underlying GraphQL mutation, rejects it with a 403 and a message telling the agent to use Junior’s own pull request tool instead (the one that always adds the right footer and the Co-Authored-By line). Junior can’t approve a pull request either, even though the token it uses technically could. The API scope says yes, the proxy says no.

The second half is what David calls “watches”. When Junior opens a pull request it subscribes to the failed CI checks, the review comments, the approvals and the merge, and those events get fed back into its conversation as messages. Nobody has to tell it to keep checking on the build, because, in his words, “‘loops’ are a horrible way to create automations”. So the agent gets told when something happens, rather than being asked to remember to look.

Two patterns, then: stop the wrong action at a boundary the agent can’t talk its way past, and push the reaction to it instead of hoping it polls. Everything else stays free, so the agent keeps all its cleverness for the actual work.

A Smaller Version of the Same Lesson

I don’t have an egress proxy or anything near Sentry’s scale, but I hit exactly this problem building Calendrz, where a lot of the engineering is done with Claude Code alongside me. Merging a pull request sounds trivial: check CI is green, check nobody left an unresolved review comment, squash-merge, get back onto main, tidy up the branch. Six steps, easy enough to write down in plain English. But a list of steps in a prompt is a suggestion, and an agent that hits an error halfway through will do what agents do: try to be helpful and find a clever way round it. Which is precisely what you don’t want when the thing in question is merging into main.

So I don’t ask nicely. The merge is now a single bash script that does all the gate checks itself (PR open and mergeable, every check passed or skipped, zero unresolved review threads) and exits with a plain error if any of them fails. The skill that wraps it is allowed to run that one script and nothing else, it can only be started when I ask for it, and its instructions boil down to “run the script, report what it says, don’t work around it”. What makes it stick is the permission rather than the wording: the harness simply won’t let the agent run anything else in that skill. Same idea as David’s proxy, just a lot smaller.

Funnily enough, the script itself has needed fixing a couple of times since (GitHub auto-deleting the remote branch was making a perfectly good merge look like a failure, for one). But each fix went into the script, once, and every run after that got it for free. Compare that with a rule in a prompt, which is only as good as the agent’s attention on the day.

The “watches” half rang a bell too. When the same skill follows a release through to production, the deploy takes about half an hour, and its instructions say plainly not to poll: start the watcher in the background and wait for the notification that it’s done. Bear in mind that’s the agent version of what us engineers have been telling each other for decades: don’t sit in a loop asking “are we there yet?” when you can be told.

Where I Haven’t Got There Yet

Now, I’ll be honest: not everything I run is this tidy. There’s a nightly job where an agent reads the production logs and opens a pull request or an issue for anything it finds (I wrote about it here), and one part of it, retrying a write when two jobs collide, still relies on a retry loop described in the prompt. The comment in that workflow says as much, a little sheepishly. It works. But it’s precisely the sort of thing David’s post argues should move out of the prompt and into something deterministic, and it’s on my list.

So What Should Engineering Leaders Do?

Here’s the thing: I keep seeing teams trying to fix agent behaviour by writing better instructions. Longer prompts, more rules, more capital letters. That’s the equivalent of managing a junior engineer purely through a wiki page and being surprised when they skip a step on a busy Friday. We’d never run production that way with people, so why do it with agents?

What I’d do on Monday morning:

• List the handful of things your agents must always or never do (merge only on green, never touch production config, always attribute the commit). That list is usually short.
• For each one, ask where it can be enforced outside the model: a script, a permission, a proxy, a CI check, a token with narrower rights. If it can, move it there and delete the paragraph from the prompt.
• Replace “keep checking until…” instructions with events: webhooks, notifications, a background job that wakes the agent up.
• Keep everything else loose. The whole point is that a little determinism in the right places lets you give the agent more freedom everywhere else, not less.

I suspect this is where agent engineering is heading in general: less prompt craft, more plain old systems design around the model, with boundaries, contracts and events. Which, as it turns out, is the stuff us techies were already good at.

Bottom line: if a rule really matters, don’t write it in a prompt, write it in code 😉