A tempting argument is going around: nobody reads the assembly a C compiler produces, so once AI models are good enough, we should stop reading the code they write too. It sounds reasonable and it is wrong, in a way that matters for anyone shipping software with AI in the loop.

The analogy breaks at the source of truth

When you compile C, the code you wrote by hand is the source of truth. The assembly is a deterministic translation of it, and if the two ever disagree, that is a fixable bug in a tool that has been hardened over decades. When an LLM writes code from your prompt, the relationship inverts: the generated code is the source of truth, and your prompt is not. A prompt cannot be dense enough to specify every state and edge case, because if it were, you would have written a program, just in a slower language. English lacks the semantics for that. So the model produces the real artifact, an artifact you have never read, and when it breaks, the first place you must look is the exact code you decided not to look at.

Non-deterministic, and not the same kind of bug

There is a second break in the analogy. A compiler is deterministic; fix an edge case once and it stays fixed forever, tested across millions of programs over decades. A language model is a probabilistic next-token predictor. It gets a fresh slate of context every single time, remembers nothing about your system between turns beyond what you flash into its window, and your real world is far larger than any context window. Over a large enough codebase it will eventually miss the edge case that lived five hundred commits ago, the same way a great human developer eventually does. Which is exactly the point.

Why review never goes away

Google, Amazon, and Microsoft do not require code review because their engineers are weak. They require it because the best people alive still make mistakes, from fatigue or from not knowing what they do not know. An LLM sits squarely in that category, minus the persistent memory. So the answer to “how good must models get before we stop reviewing” is never, for the same reason we never stopped reviewing our best humans. This is not pessimism about AI; it is just where responsibility lives. When an automated step can act on real systems, someone has to be able to see what it did and answer for it, which is why we treat accountability and inspectability as non-negotiable and keep a person on every consequential call.

The practical version

Use AI to write as much code as you like; it is a genuine accelerant. Just do not confuse producing code with shipping trustworthy code. Read it, test it, and keep the human review gate that every serious engineering org already runs. In our solutions the model does the volume and a person owns what goes to production. The day you YOLO unread AI output straight to users is the day you find out, in production, exactly which edge case it forgot.

Prompted by a video reacting to a “should you stop reading LLM code” debate on X (2026). Credit to the participants; the argument here is ours.