Don’t read every line: code review in the age of AI agents
When AI writes the code, nobody can read it line by line any more. Experienced people are right that AI code has more bugs. They are wrong that the only defence is reading it. Where review is moving, and what stays human.
I recently talked to an experienced team leader who has been working in .NET for over twenty years. His view was clear: “I understand code far better than any AI. I know why to use native logging instead of a popular library. That’s why I check every line to make it perfect. AI is just guessing.”
He is right about a lot. But on a recent project, where an AI agent wrote over seven thousand lines of application code and five thousand lines of tests in a few days, I asked the question differently. Not “who understands code better”, but “what actually catches the bugs”.
Where the expert is right
AI code really does have more problems. CodeRabbit compared 470 open-source pull requests. Those with AI involvement had on average 10.83 issues found, human ones 6.45 – about 1.7 times more. AI had more problems with logic, security and performance. People, on the other hand, made more typos and wrote less testable code. Mind the context: CodeRabbit sells AI review, so it has a stake in the result.
CodeRabbit: State of AI vs Human Code Generation ↗
Performance is a weak spot. A model learns from the average code on the internet and reaches for what is popular, not what is best for the case at hand. In the EffiBench benchmark, code from the best model tested was on average 1.69 times slower than the reference human solutions. This is exactly what an experienced person sees: the difference between “it works” and “ideal for this case”.
Where the expert is wrong
“AI is just guessing” doesn’t hold for review. ByteDance runs AI review for more than 12,000 developers a week and, after a two-stage filter, reached 75 % precision of comments. Their data also shows that human comments lead to code changes more often. So AI review is a good first filter, not a replacement for an expert.
BitsAI-CR: Automated Code Review via LLM in Practice ↗
Human review also finds less than we think. A Microsoft study found that finding defects is the main motivation for review, but review turns out to be less about defects than expected. Its real benefits are knowledge transfer, team awareness and alternative solutions.
Bacchelli & Bird: Expectations, Outcomes, and Challenges of Modern Code Review (ICSE 2013) ↗
And an expert doesn’t scale. Writing code is no longer the bottleneck. Review is, and review by eye doesn’t scale. When an agent produces in a day what a team used to produce in a week, nobody reads it carefully. Either review speeds up to “approved”, or it becomes the brake.
What caught the bugs in my project
While building Payment matching with AI, I wrote down every mistake the agent made and what caught it. The result surprised me: none of the significant bugs was found by reading the code.
The agent’s code always compiled and looked right at first glance. The bugs only showed up where reading doesn’t reach: on specific data, on another operating system or when tests ran in parallel. A person reading the code easily misses them, because it looks right. A mechanism doesn’t. A different kind of mistake – logic not thought through, which tests won’t catch – is described in What the AI agent got wrong. There, it was a person using the app who helped.
Review by risk
I’m not saying code isn’t worth checking. I’m saying mechanisms check the code, and a person checks the intent, the criteria and the evidence. In practice:
- Line by line only where a lot is at stake: money, security and permissions, concurrency, the data model, public APIs, prompts and AI guardrails.
- Automated gates everywhere else: analysers with warnings as errors, tests, evals, benchmarks and load tests. Plus an AI reviewer with a checklist, ideally a different model from the one that wrote the code.
- A person approves the plan, not the diff: the brief, architecture decisions, interfaces, the data model and the criteria for “done”. Reviewing a ten-line plan adds more than reading a thousand lines of diff.
- Performance isn’t judged by eye, it’s measured. A load test, a profiler, the number of SQL queries per request. What an expert would “see” can be turned into a number that every build checks.
- Turn expert knowledge into rules. Now and then give a module to a real expert, compare their comments with the AI review, and write what the AI missed into the agent’s rules, an analyser or a checklist. The expert’s experience then guards every line, not just the ones they have time to read.
What this means for companies
For teams bringing AI into development, the most expensive mistake is to leave review unchanged. It either becomes a rubber stamp, or a bottleneck that cancels out the benefit of AI.
Instead, it pays to invest in what replaces review: tests against real behaviour, evals, measurements and clear rules for agents. And to move experienced people to where they are irreplaceable: the brief, the architecture, the risky parts and the question of what “correct” actually means. How this changes a developer’s role is the subject of Where developer work is going.
An experienced team leader knows better than anyone what to look for and where to focus. He just shouldn’t be looking for it by eye.