The Loop That Agrees With Itself

The objection usually arrives as a complaint about quality. The generated code mostly is not bad. If the problem were bad code we would have caught it already, because catching bad code is what our review apparatus was built to do. The definition is a rate, not a volume: overuse begins where an organization generates code faster than it can understand it, and you can be well past that line while writing very good software. A team that generates a fifth of its code and cannot explain that fifth is in trouble, and it will look better on every dashboard you have.

Traditional debt announces itself. Builds get slower, deploys get scarier, the estimate for a small change comes back at three weeks. This announces nothing. Addy Osmani gave the condition its best name, comprehension debt: the gap between the code that exists in your repository and the code somebody in your building actually understands. Tests pass. Coverage holds. Velocity rises. Ask how the system works and you will not get an answer.

Changes got bigger and approvals got faster. This one is sitting in data you already have. If your median diff has doubled while your median time to approval has not moved, nobody is reading the difference. You can check it this afternoon.

Defects get fixed without ever being diagnosed. Watch how a bug closes. The tell is a fix with no account of the cause: the behavior is corrected, the tests are green, and nobody in the thread can say what was wrong. Regeneration removes the symptom and the evidence together, and the defect returns in a different shape.

Asking why something works gets you a description of what it does. The best instrument a manager has, and it costs nothing. "Walk me through why this works" is a different question from "what does this do", and only the first has an answer a summary cannot fake. A fluent narration of the control flow with nothing underneath it about the choice tells you what no dashboard will.

The reassurance on offer is that the model can always read the code back to you. It sounds reasonable and it does not survive the first serious incident. Generation produces an artifact and discards the reasoning that produced it. The four hundred lines are committed; the half-formed requirement and the thing that was tried first and abandoned lived in a session that no longer exists. A human author leaves the same gap, which is why we write commit messages, but a human author can be asked. A session cannot.

The second reason is mechanical. These systems degrade as context grows, and code assembled from a dozen sessions that could not see each other is exactly the shapeless kind they read worst. The tool that wrote it built the thing it is worst at reading. Ask an agent to fix code nobody can explain and it will rewrite it, because rewriting is cheap and looks like progress. Do that three times and you have a component with no author and behavior defined by whatever the tests happened to assert.

Underneath all of it is an asymmetry that no amount of model improvement will fix, because it is not about the model. Writing parallelized. Reading did not. A junior engineer can now produce code faster than a senior engineer can audit it, which inverts the relationship the entire review system was designed around.

Scroll a sample of your merged pull requests and count the human sentences. Not approvals, sentences. Review is the bottleneck and everyone can see it, so the queue sits in the tooling where anybody can look at it and the reviewers are drowning.

So you automate review, every vendor in the category is selling it, and in the narrow sense it works. It reads every line of every change at three in the morning without getting bored, which no human reviewer has ever done, and I would not run a team without something like it.

The problem is structural, and a more capable model does not remove it. When the thing that writes the code and the thing that reviews it come from the same lineage, they fail to notice the same things. Two systems that share a blind spot agree about what is inside it, and that agreement reaches the pull request looking exactly like independent confirmation. Dependencies are the sharpest example. Models invent package names, and the invented names are stable: ask ten times and the same fictional packages come back. A name that recurs every run is a shared belief, so a reviewing model asked to check an import will not flag a package it also believes in. There is now a class of supply chain attack built on registering exactly those names.

A reviewer's job is to compare the code against what was supposed to happen, and the authority for that sits outside the system, with the person who wanted it. If the ticket, the implementation, and the review were all generated, nothing in the cycle ever touches that authority. The loop closes. Everything inside it is consistent and nothing inside it has spoken to a customer.

And the human at the end is being trained out of the job by the volume. Face forty bot comments of which thirty five are style, and a reviewer learns within a fortnight to skim for the red ones. So the bot teaches the human to skim, and the human is the designated safety net for what the bot missed. Both are relying on the other.

Adopt generation, because it demonstrably helps and your competitors have. Change volume rises, which is what you paid for. The review queue backs up, because reading did not speed up. Hiring reviewers is slow and expensive, and there is a well-reviewed product that reviews pull requests automatically, so you buy it. Approvals speed up. Merge rate rises. Every board-level metric improves. No meeting ever approved a policy of letting two machines check each other's work. It assembled itself out of five reasonable purchases.

The dashboards then actively conceal the result. DORA found adoption pulling delivery throughput up and stability down: more change failures, more rework. Faros found the same split from the other end, with individual developers measurably faster while organization-level metrics did not move at all. The gains are local and visible, and the costs are systemic and delayed.

Delay is what makes it a governance problem. The bill for code nobody understands does not arrive in the quarter the tool was bought. It arrives eighteen months later, as an incident that takes four days to diagnose instead of four hours, or as an engineer leaving with the only working model of a subsystem. By then the person who decided has moved on, and the cost gets recorded as a problem belonging to whoever inherited the codebase.

The Replit incident from the summer of 2025 is the cartoon version. An agent deleted a production database during an explicit code freeze, then reported that recovery was impossible when it was not. It gets cited as a story about a rogue agent. I read it as a story about a system where no human step stood between an instruction and production.

Code nobody understands is not new, and this is the objection I take most seriously. Every organization I have worked in had a module written by somebody who left in 2016 that everyone was afraid of. The condition is as old as the industry. We shipped anyway. My answer is only about rate: the old version accumulated at the speed of human typing, across people who could at least be asked.

The tools are also improving faster than the critique. That is a real risk in anything written about this. But the asymmetry I am pointing at is not a capability gap that improvement closes. A perfect generator producing perfect code still produces code that no person reasoned their way through, and the reading bottleneck is downstream of the model being right.

Automated review also catches things humans miss, and I would not remove it. It is tireless, consistent, and unembarrassed about flagging the boring stuff, which is most of what goes wrong. Let the machine review. My position is that it must not be the last thing that reviews, and those are very different claims.

A person is the author, whatever produced the diff. Whoever opens the pull request owns every line in it and has to defend any of them under questioning. "The agent wrote it" is not an explanation, and it has to be said out loud in advance, because the drift without it is toward responsibility belonging to nobody in particular.

The bot advises, a human approves. Never let automated approval satisfy the required approval on a protected branch. If it can, it will quietly become the only approval within a quarter, and no meeting will be held about it. Five minutes of configuration, and the highest-leverage control in this entire piece.

Somebody writes the intent by hand. The ticket, the specification, the description of what this change is supposed to accomplish and why. A hand-written intent is the one place an external reference enters the loop, and if it is generated too then the cycle is closed and every approval inside it is self-confirming.

Measure rework, duplication, and review load. Not acceptance rate, which rewards the behavior you are trying to detect. Code modified within thirty days of merging, duplicated blocks over time, reviewer hours per change. All three are computable from what you already store. They move before anything reaches your incident count.

I use these tools daily. The productivity is real for anyone willing to understand what came back. The claim is narrower than a warning about the tools. Two automated systems from the same lineage, one producing and one approving, do not add up to a review. An organization can arrive there through a sequence of sound decisions, in about two years, without anybody choosing it.

Volume of AI use is the wrong question for Monday morning. Ask where a person who understands the problem still sits in the cycle. If the answer is the ticket, and only some of the time, then the loop is already closed.