Review Overcorrection

Antipattern: A recurring trap that causes harm — learn to recognize and escape it.

An AI reviewer asked to find problems reports problems in work that’s already correct, and the agent acting on the review “fixes” them, turning a quality step into churn, over-engineering, and new bugs.

Also known as: Reviewer Overcorrection, Fixer Overreach

You add a fresh reviewer to an agent loop. It sees a correct 12-line fix with a passing test and returns six findings anyway: a redundant null check, a test for an impossible input, and a helper extraction among them. The coding agent obeys. The diff grows to 90 lines, and a new defensive branch breaks a downstream caller.

Both agents followed their instructions. The reviewer found problems; the fixer fixed them. Review overcorrection turns that faithful behavior into churn on work that was already sound. It’s the mirror image of Sycophancy: one agent agrees because agreement pleases, while the other disagrees because disagreement is its assigned job.

Symptoms

• The reviewer almost never returns “no findings,” even on small, well-tested changes.

• Findings repeat a few shapes: an unreproduced logic error, an unstated requirement, an unreachable edge case, or a request for more tests.

• Diffs grow after review. The fix pass touches files the original change didn’t, and the result is harder to read than the version that went into review.

• Tests that passed before the fix round fail after it, or behavior shifts in code the finding never mentioned.

• Each review round produces a fresh set of findings, so a loop that waits for a clean review ends only at its iteration cap.

• Abstraction layers, defensive checks, and configuration options appear that no requirement asked for.

• People resolve the reviewer’s comments with “won’t fix” by reflex, and the occasional real defect gets dismissed with the rest.

Why It Happens

The prompt presupposes a defect. “Review this diff and list the problems” tells the model that problems exist. Returning nothing feels like failing. Anthropic’s Claude Code guidance makes the same point while recommending adversarial review: a reviewer asked to find gaps will usually report some, even in sound work, and chasing them all causes over-engineering.

Reviewers are biased toward rejection. Haolin Jin and Huaming Chen asked GPT-4o, Claude 4.5 Sonnet, Gemini 2.0 Flash, Llama 3.1 8B, and Mistral Small 3.1 24B whether implementations met their requirements. The models judged paired correct and buggy functions from three benchmarks. Even with a yes-or-no prompt, they rejected between 25 and about 75 percent of correct implementations, depending on the model and benchmark.

Asking for an explanation and fix makes it worse. On HumanEval, that prompt moved GPT-4o’s rejection of correct implementations from 26.2 percent to 73.2 percent. Buggy code passed fell from 2.44 percent to zero. The model caught more defects by inventing many more.

Invented defects have recognizable shapes. Jin and Chen sorted 4,190 false-rejection rationales. Four causes explained 87.2 percent: claimed logic errors (48.2 percent), unstated requirements (14.1 percent), claimed boundary errors (13.2 percent), and misread specifications (11.7 percent). None came with a failing input.

The fixer overreaches. Weishi Zeng and Mark Ally at Imbue split Python tasks from SWE-bench Pro among implementer, reviewer, and fixer agents. Of 15 tasks already solved completely, reviewers flagged eight; six findings asked for more tests. Detailed reviews made fixes balloon. One fixer changed 113 lines across eight files instead of four lines, dropping the score from 16 of 17 tests passing to 8 of 17. Zeng and Ally call this overreach.

Loops amplify both halves. A new full review after every fix creates fresh findings each round. In Layers and Generators terms, the reviewer is a generator. Repairing its output one item at a time never exhausts it.

The Harm

Correct code gets worse. Review meant to catch regressions instead introduces them: a defensive branch changes an edge-case result, a refactor breaks a caller, or a test gets rewritten to match new behavior.

The code also grows without a requirement. Imagined defects add abstractions, defensive checks, and tests for impossible cases that KISS and YAGNI exist to prevent. Each addition looks reasonable alone.

Review loses its signal. People learn to dismiss a noisy reviewer, and real bugs leave with the false alarms. Static analyzers have long failed this way. AI reviewers earn trust by the same rule.

Automation multiplies the cost. A Generator-Evaluator loop that waits for reviewer satisfaction spends tokens and hours on correct work. The final diff is harder for a person to review than the first one.

Reviewers can catch bugs that tests miss. The trap is acting on an undemonstrated finding, then letting its fix change more than the finding named.

The Way Out

State the bar. A finding must affect correctness or break a stated requirement. Label style notes, alternative designs, and extra tests as optional. Without a bar, the reviewer sets a low one.

Make “no findings” valid. Ask for a verdict against the bar and allow an empty findings list. A Structured Outputs schema that requires one finding guarantees one.

LLM-as-Judge recommends reasoning before a score. That advice applies to grading on a scale. A conformance gate faces a different risk: demanding an explanation of the problem and a fix presumes failure. Keep the reasoning, drop the presumption, and request a fix only after evidence shows a defect. Jin and Chen tested single functions, not repository-scale review, so the direction matters more than the exact rates.

Require evidence, and let execution decide. A correctness finding needs a failing input, counterexample, or test. Run it. Jin and Chen’s Fix-guided Verification Filter tests the original and proposed fix, keeping rejections only when the original fails and the fix passes. Across models, it cut false rejection from 54.8 to 16.3 percent on HumanEval, 69.0 to 28.9 percent on MBPP, and 51.0 to 24.0 percent on QuixBugs, while passing slightly more buggy code. Test Oracle and Verification Loop describe the machinery. Tests arbitrate, not confidence.

Scope the fixer. Let it change only what an accepted finding names, and permit it to skip unsafe fixes. This removed catastrophic regressions from Imbue’s Sonnet fixer; permission to skip mattered more than exact wording. Route skips through Human in the Loop.

Pass the finding, not the essay. A short accepted finding limits the fix. A long review reads as permission to touch everything it mentions.

Bound the loop. Cap review rounds. Later rounds should check only the fix diff against accepted findings, not reopen the whole change. Loop Engineering supplies the stop conditions.

Measure the overturn rate. Add known-correct changes to your Eval suite and track how often people reject findings. Google’s static-analysis checks had to stay below 10 percent effective false positives, meaning results developers ignored. An analyzer whose “not useful” clicks passed 10 percent was disabled until improved.

Tip: Add one sentence to your reviewer prompt: “If the change is correct and meets the stated requirements, say so and report no findings.” Then test it on a change you know is correct. If the reviewer still finds something, you’ve measured its bias on your own code before it reached a real pull request.

How It Plays Out

A data team’s unattended loop revises until its reviewer approves or ten rounds pass. A two-line timezone fix with a test is correct on the first attempt, yet hits the cap. Round three validates timestamps the upstream schema guarantees. Round five extracts a date utility. Round seven rewrites the original test and loses the timezone assertion.

The team requires a failing test for correctness findings and runs it before the coder sees the finding. Passing tests turn findings into suggestions. Most tickets now clear review within two rounds.

An engineering manager samples 50 automated review comments dismissed as “won’t fix” and asks two senior engineers to label them. Most dismissals were right: the reviewer wanted null checks for non-null fields and tests for unreachable branches. Two comments found real bugs, including an off-by-one pagination error that shipped. The team requires reproducing cases for correctness and requirement findings, separates style notes, and tracks weekly dismissals. Comment volume falls by more than half. Developers read what remains.

Detected by: Eval — A labeled set of known-correct changes measures how often a reviewer rejects sound work before it reviews real code.

Informed by: Layers and Generators — A reviewer that finds something every round is a generator of findings, so fixing them one at a time never converges.

Informed by: LLM-as-Judge — A code reviewer is an LLM judge applied to a diff, and false rejection of correct work is a judge bias the four-bias catalog doesn't list.

Informed by: Sycophancy — Sycophancy's warning against objections on demand anticipates this failure; overcorrection is that failure measured in code review.

Mitigated by: Human in the Loop — Findings the fixer skips as unsafe go to a person instead of being forced through by the agent.

Mitigated by: Loop Engineering — Capping review rounds and re-reviewing only the fix diff keeps a finding generator from driving the loop indefinitely.

Mitigated by: Structured Outputs — A review schema with a verdict field and an allowed empty findings list makes "no findings" a normal response.

Mitigated by: Test Oracle — An executable oracle settles whether a finding describes a real defect, so the reviewer's confidence stops being the arbiter.

Mitigated by: Trusted Monitoring — A monitor that checks a reviewer's claims against ground truth catches findings with no failing case behind them.

Mitigated by: Verification Loop — Running a finding's failing case against the current code separates demonstrated defects from imagined ones before anyone edits.

Violates: Code Review — Code review adds value only when its findings are real; a reviewer that invents defects spends the author's attention and teaches them to ignore it.

Violates: Generator-Evaluator — Generator-Evaluator depends on the evaluator passing sound work; an evaluator that always finds something keeps the loop running on correct code.

Violates: KISS — Fixes for imagined defects add abstraction and defensive code the problem never required.

Violates: YAGNI — Tests for impossible inputs and guards for states that can't occur are speculative work a reviewer requested and nobody needed.

Sources

• Haolin Jin and Huaming Chen, in Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement (arXiv preprint, 2026), measured how often five models reject correct code, showed that prompts requiring an explanation and a fix raise that rate, classified the causes, and proposed the execution-based Fix-guided Verification Filter.

• Weishi Zeng and Mark Ally at Imbue, in How AI Code Review Can Make Correct Code Worse (2026), demonstrated the fixer side in an implementer, reviewer, and fixer pipeline, named it overreach, and showed that scoped fixer instructions contain it.

• Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan, in Lessons from Building Static Analysis Tools at Google (Communications of the ACM, 2018), described how developers stop trusting automated review findings once false positives waste their time, and how Google’s Tricorder platform enforced a 10 percent ceiling on effective false positives using developers’ own feedback on each result.