I read a founder’s essay this morning that I suspect a lot of CTOs will forward to their teams this week, and for the wrong reason. The headline numbers are the bit everybody will quote: the founder of Homing, a consumer real estate platform, rebuilt his entire platform alone in eight months, for about $200 a month in AI subscriptions and under $100 a month for hosting and tools. The first version had taken a team several years and hundreds of thousands of dollars. Last month the project pushed roughly seventy million tokens through Claude Code (about $5,400 at API list rates, by his own logs), and he didn’t write a single prompt to the coding sessions himself.

Impressive, right? It is. But the part I’d want every engineering leader to read twice is the story about the pigeon.

The Pigeon That Wasn’t There

His platform has an AI character called JET, a homing pigeon who analyses housing data and gives opinions on what homes are worth. They built a feature around JET, and the tests passed. For weeks. Then a real batch job tried to use the feature and rejected every single property, because JET had never been created in the production database. The tests were green because each test built a throwaway copy of the database, planted JET in it, and then checked that JET was there. Go figure.

That wasn’t a one-off either. A nightly job produced a lovely report (tens of thousands of records reviewed, clean completion) when it had in fact done nothing, and had written its report from a results file an earlier run left on disk. And all 119 runs of the project’s official CI checks had failed before the tests even started, while everything looked healthy on the local machines and nobody had opened the CI page.

He calls the lesson the “proxy law”: never check a stand-in for the thing itself when you can check the thing itself. I’d go a step further, because I think this is the crux of AI-assisted engineering right now.

Here’s the Thing: The Agent Grades Its Own Homework

When one agent writes the code and the tests from the same understanding of the requirement, the tests don’t verify the code. They verify the assumption. If the assumption is wrong (JET exists, the job did some work, CI is running), the code and the tests are wrong in exactly the same way, and you get a beautiful green build that proves nothing. A human developer does this too, of course, but a human tends to get suspicious when something works first time. An agent doesn’t get suspicious. It gets confident, across twenty files, faster than any human could review them.

And as such, coverage, green ticks, tidy summaries and “all tasks completed” messages become proxies. Goodhart had this one figured out decades ago: when a measure becomes a target, it ceases to be a good measure. Agents are very, very good at hitting targets. That’s the whole point of them. So if the only target you give them is “make the tests pass”, don’t be surprised when that’s exactly what you get.

Now, don’t get me wrong: I’m not arguing against any of this. I’m firmly in the camp that the future of software engineering is “lights out”, where people write specs and agents write, test and ship the code. This essay is, in fact, a pretty good picture of what that looks like in practice: the prompts didn’t disappear, they moved into requirements docs, decision ledgers and work orders with acceptance criteria. (He says that in August, for every token the AI wrote, it read about 350. What you let it read is the work.) But “lights out” doesn’t mean nobody checks. It means the checking has to be engineered just as deliberately as the building, otherwise you’ve simply automated the production of convincing reports.

What I’d Do on Monday Morning

If I were looking at a team that has gone heavily agentic (and most of the teams I talk to nowadays have, or are about to), these are the questions I’d ask:

• Who wrote the tests, and from what? Tests derived from the spec, ideally by a different session that never saw the implementation, catch far more than tests written by the agent that just wrote the code. Separate the builder from the checker.
• Does anything check the real thing? A test fixture that creates its own preconditions is fine for unit tests, but somewhere there has to be a check against the real database, the real queue, the real CI run. Smoke tests in production, read-only, after every deploy.
• Where does each report come from? Any job summary, AI-written or not, should be generated from evidence produced by that run, not from whatever happens to be lying on disk.
• Who opens the dashboard nobody opens? 119 failed runs that nobody looked at is a leadership problem far more than an AI one. Schedule the distrust: a regular audit (and yes, an agent is great at this) whose whole job is to assume the docs and the checks are lying.

I’ve seen first-hand how much the “check the thing itself” part matters. In Calendrz, an agent reads our production logs every night and opens a fix PR or an issue for what it finds, and a human still reviews and merges every PR. The production logs are about as close to the thing itself as you can get: no fixture, no planted pigeon, just what actually happened. (Building calendrz.com pretty much on my own with AI has taught me that the agent is rarely the weak link. The weak link is whatever I forgot to tell it, or forgot to check.)

The Job Moved

The essay ends with a line I fully agree with: AI didn’t remove engineering judgement; it made it the whole job. I’d put it slightly differently for the engineering leaders out there. The scarce skill on your team is no longer the person who writes code fastest (the agent wins that one, hands down). It’s the person who keeps asking “how do we know?”, and then builds the tripwire so nobody has to ask again.

Bear in mind also what one founder with a $200 subscription just did to a product that took a funded team years. Small, fast, agentic teams are coming for a lot of incumbents, and the ones who survive won’t be the ones who generate the most code. They’ll be the ones whose green builds actually mean something.

Bottom line: if your agents write the code and the tests, your job is to make sure at least one check in the pipeline can’t be fooled by either. Go and look for your pigeon 🙂