Why multi-round LLM code review loops diverge

Why does multi-round LLM code review diverge instead of converging? AR Labs researched that question in September 2026, after seeing review loops in which each round produced more critical findings on smaller issues. The findings bear on how our code review gate runs AI review. The report tests five hypotheses against production post-mortems, benchmark papers and practitioner reports, then ranks mitigations by strength of evidence.

Sources were verified to differing depths; see the confidence section.

Five hypotheses, adjudicated

1. Unbounded fresh-pass objection sampling: strongly corroborated. Greptile production post-mortem: ~19% useful / 2% wrong / 79% nits (https://www.greptile.com/blog/make-llms-shut-up). Benchmarking: automated code review precision <10%, F1 ceiling attributed to false-positive volume (https://arxiv.org/html/2509.01494v1). A generative critic samples objections; a fresh full pass per round draws a fresh sample, which produces the "new findings each round" signature.
2. Fix-loop surface growth: corroborated indirectly. Defect detection degrades sharply with patch size (~400 LOC inflection; SWE-at-Google ch.9, https://abseil.io/resources/swe-book/html/ch09.html). A growing diff adds objection surface while lowering real-defect hit rate. (Extrapolation to LLM loops is inference, not measured.)
3. Scope escape into pre-existing code: practitioner-corroborated. "Comment only on changed lines" is a standard bot countermeasure (https://dev.to/pyor/alert-fatigue-comes-for-code-review-16kj); keep whole-repo context readable but not reportable (Greptile HN thread, https://news.ycombinator.com/item?id=42451968).
4. No materiality standard: strongly corroborated, and the obvious fix fails. LLM-as-judge severity self-rating measured near-random (Greptile); judge scores fragile to authority/verbosity cues (https://arxiv.org/html/2604.16790v1, https://arxiv.org/html/2411.15594v6). A rubric clause like "unproven evidence = CRITICAL" is a standing authority cue pushing uncertainty to the top.
5. Gold-plated upstream contracts: unaddressed by the literature. Nearest work: LLMs systematically over-reject conforming code when judging against NL requirements (https://arxiv.org/html/2603.00539, https://arxiv.org/html/2508.12358v1), which is a different claim. Locally testable: classify each CRITICAL as contract-traceable vs reviewer-invented.

Mechanisms the literature adds

• Prompt-elaboration backfire (largest measured effect). GPT-4o false-rejection: 26.2% simple prompt → 73.2% when explanations required; repair-suggestion prompts amplify further (https://arxiv.org/html/2603.00539). Adversarial framing + justify + rate + fix in one pass is the configuration measured to maximise wrongful blocking.
• Self-evaluation is not a filter. Same-model severity gating inherits near-random calibration.
• Non-convergence is the default shape. Self-Refine gains concentrate in rounds 1–2, saturate by ~3 (https://proceedings.neurips.cc/paper_files/paper/2023/file/91edff07232fb1b55a505a9e9f6c0ff3-Paper-Conference.pdf); recursive refinement need not reach a fixed point; stop on an inter-round delta threshold (https://arxiv.org/html/2607.22653).
• Alert fatigue, agentic variant: a fixer agent cannot dismiss, so noise expresses as compliance churn instead of inattention.
• Judge instability: accuracy tracks the judge's own capability; "reliability without validity" (https://arxiv.org/abs/2609.12002, https://arxiv.org/html/2606.19544v1). An unpinned judge silently moves the merge bar.

Mitigations by strength of evidence

1. Execution-grounded verification: keep an objection only if original vs "fixed" code behave differently under test (FN −30–67pts, FP <5) (https://arxiv.org/html/2603.00539).
2. Hard round cap (2), round 2+ delta-only + regression on closed findings.
3. Calibrate against own accept/dismiss history (embedding-similarity suppression; 19%→55% acted-on within two weeks; vendor self-report).
4. Reportable scope = changed lines; repo context readable, not reportable.
5. Split verdict from rationale; severity emitted after the finding; fixes drafted in a separate pass.
6. Instrument acted-on rate; kill categories under ~30%.
7. Pin the judge per change under review; ensemble only for merge-blocking calls (transfer assumed, not shown).
8. Cap patch size, which bounds mechanism 2 at source.

Confidence and caveats

High: mechanisms 1 and 4, prompt-elaboration backfire (multiple sources with numbers). Medium: 2, 3 (uniform practitioner consensus, adjacent formal evidence). Low/untested: 5. The 26xx-series judge-bias arXiv numbers were verified only at search-snippet fidelity; re-check before citing. Greptile numbers are vendor self-report.

An experiment worth running

Re-run one escalating branch three ways and count CRITICALs per round: a fresh full pass (control), delta-only, and delta-only with explanation stripped from the finding pass. Prediction from the overcorrection numbers: the third arm collapses the count. Separately: classify existing CRITICALs contract-traceable vs reviewer-invented to settle hypothesis 5.