LLM-as-reviewer methods: quality and readability checks

When does using one LLM to check another's output actually improve quality and readability, and does the habit of LLMs writing as if the reader shares their working context have a name in the literature? AR Labs researched those questions in August 2026. The findings inform the cross-model reviewers in our Doc Review skill and the grounding test in Write Well. The report ranks review methods (cross-context review, rubric-based judging, cross-family reviewers, self-refine, multi-agent debate) by evidence from about 32 papers, proceedings, blogs and documentation sources, and sets out structural readability checks for technical prose.

Method: standard-depth research, 10 web searches, about 32 sources, each scored with the CRAAP test (currency, relevance, authority, accuracy, purpose).

Summary

Peer-review architectures (one LLM critiquing another's output) show measurable quality gains over single-pass generation, but only when specific bias-mitigation conditions are met, particularly context separation and cross-family reviewers. The highest-evidence method is Cross-Context Review (CCR): placing the artefact in a fresh session strips anchoring bias and sycophancy, raising error-detection F1 by ~4 points over same-session self-review. LLM-as-judge frameworks suffer well-quantified verbosity bias (+15–30 points preference for longer outputs), position bias (10–15 point winrate swing), and self-preference bias (10–25% inflation for same-family outputs), all of which are tractable with rubric decomposition, dual-order testing, and cross-family reviewers. The "unshared context" pattern (an LLM writing as if the reader shares its loaded working context) has strong conceptual grounding in psycholinguistics and empirical grounding in the "grounding gap" literature, but no single established label in the NLP or agentic-AI literature as of 2026-08. The nearest established concept is Clark's common ground / grounding acts framework, and Shaikh et al., Grounding Gaps in Language Model Generations (NAACL 2024), is the closest primary source. Surface readability metrics (Flesch–Kincaid, SMOG) systematically mislead on technical prose; structural checks (buried lead, nominalisation density, passive-voice ratio, filler phrases, one-idea-per-sentence) are the better instrument.

Q1: The unshared-context writing pattern

What it is

The phenomenon: an LLM produces text that references or presupposes information present in its own context window (prior tool outputs, file contents, intermediate reasoning, earlier conversation turns) without accounting for the human reader's lack of access to that context. The model writes "as if" its working context is shared. Examples: naming a variable or finding as though introduced when it was not; treating a tool result as established fact without surfacing the result itself; referencing a file section without quoting it.

Is it formally named?

No single established term maps precisely to this pattern in either NLP or agentic-AI literature as of 2026-08. The following candidate framings were investigated:

Candidate term	Status
Audience design / audience awareness (NLG)	Established concept in human psycholinguistics; no LLM-specific literature applying it to this exact failure
Common ground / grounding failure (Clark)	Nearest established concept. Directly applicable. Primary source: Clark & Brennan (1991), applied in LLMs by Shaikh et al. (NAACL 2024)
Presupposition failure / accommodation	Studied in LLMs (misinformation context), not the writing output failure
Curse of knowledge (LLMs)	Studied as a bias in LLM judges, not LLM writers
Egocentric bias	Studied as self-preference in evaluation; not mapped to this writing pattern
Context asymmetry / unshared-context	Not a defined term in literature found
Context leakage / context bleed	Appears in security literature (privacy leakage); not the writing pattern
Reader model / audience model in LLM generation	Appears in human-NLG research; not systematically applied to agentic LLMs

Strongest available sources

1. Shaikh et al., Grounding Gaps in Language Model Generations (NAACL 2024) URL: https://arxiv.org/abs/2311.09144 CRAAP: Currency 3 (2023), Relevance 5, Authority 4 (peer-reviewed, ACL), Accuracy 4, Purpose 5 → CRAAP 4.2 Core finding: "LLMs generate language with less conversational grounding, instead generating text that appears to simply presume common ground." RLHF and instruction tuning reduce grounding acts, so training makes this failure mode worse, not better. The paper measures this directly: models skip clarification, confirmation, and checking acts that humans use to establish shared reference.

2. "Curse of Knowledge: When Complex Evaluation Context Benefits yet Biases LLM Judges", arXiv:2509.03419 (2025) URL: https://arxiv.org/abs/2509.03419 CRAAP: Currency 5, Relevance 3 (applies to judges, not writers; adjacent), Authority 4, Accuracy 4, Purpose 5 → CRAAP 4.1 The "curse of knowledge" framing documents that LLMs possessing richer context make biased judgments: the model's loaded context leaks into its assessments. This is the evaluator-side analogue of the writer-side failure. Useful because it confirms the mechanism exists and is measurable.

3. Clark and Brennan's grounding framework (the foundation Shaikh et al. build on) Clark & Brennan (1991) "Grounding in Communication" established the theoretical framework: interlocutors maintain a "common ground" of mutually known, believed, and accepted information, and actively signal when this ground needs updating. LLMs trained on next-token prediction with no explicit audience model violate this by default.

4. "Agentic Knowledgeable Self-awareness", ACL 2025 URL: https://aclanthology.org/2025.acl-long.619.pdf CRAAP: Currency 5, Relevance 3 (covers agent self-knowledge gaps broadly), Authority 5, Accuracy 3, Purpose 5 → CRAAP 4.0 Frames the problem from the agent's perspective: agents have poor calibration about what they know vs. what external parties know. Relevant to the asymmetry mechanism.

Verdict

Confidence: Moderate. The phenomenon is real and documented at the mechanism level. It is not yet formally named in the NLP or agentic-AI literature. The closest named concept is Clark's common ground / grounding acts, and the most directly applicable empirical paper is Shaikh et al. (NAACL 2024). The pattern is newly salient in the agentic long-context era (post-2023) because agentic tool-use dramatically expands the volume of context only the model holds, but no 2024–2026 paper yet specifically frames it as an agentic writing failure mode with that as its central contribution.

Evidence gap: Searched with 6 different framings ("unshared context," "context leakage," "audience model," "reader model," "presupposition failure," "egocentric bias") and found no paper with this exact failure as its primary topic in the agentic writing context. The gap is real.

Q2: The most effective LLM-checks-LLM methods

Ranked by evidence of real improvement

Tier 1: strong evidence of real quality gain

1. Cross-Context Review (CCR) What: Reviewing an LLM artefact in a completely fresh session containing only the artefact and a standardised review prompt. No production context history. Evidence: Choi et al. (2025), arXiv:2603.12123. Controlled study, 30 artefacts, 150 injected errors, 3 independent runs per condition. CCR F1 = 28.6% vs. Self-Review 24.6% (p=0.008), Repeated Self-Review 21.7% (p<0.001), Subagent Review 23.8% (p=0.004). SR2 not better than SR (p=0.11), proving benefits come from context separation not repetition. Strongest gains on critical errors (+11pp) and code artefacts (+4.7 F1). CRAAP: Currency 5, Relevance 5, Authority 4, Accuracy 5, Purpose 5 → 4.8 Confidence: High

2. Rubric-based / decomposed per-criterion evaluation What: Providing explicit, criterion-separated rubrics so the judge evaluates one dimension at a time rather than holistically. Evidence: LLM-Rubric (Eisenstein et al., ACL 2024; arXiv:2501.00274); Prometheus / Prometheus 2 (2024); RubricEval (2024–2025). Consensus: decomposed rubrics substantially improve reliability and correlation with human judgments vs. holistic prompts. Meta's 2026 rubric-refinement work showed better rubrics improve both evaluation and RL fine-tuning signal. CRAAP: Currency 5, Relevance 5, Authority 5, Accuracy 4, Purpose 5 → 4.8 Confidence: High

3. Cross-family (different-model-family) reviewer What: Using a model from a different training lineage as the reviewer rather than a same-family model. Evidence: Self-preference bias literature (arXiv:2410.21819, arXiv:2604.22891) documents 10–25% inflation when a model reviews same-family outputs. CCR paper (2603.12123) explicitly notes that cross-family review reduces self-preference to near zero. Triangulated across 3+ independent studies. CRAAP: Currency 5, Relevance 5, Authority 4, Accuracy 5, Purpose 5 → 4.8 Confidence: High

Tier 2: moderate evidence, tractable with care

4. LLM-as-judge with bias mitigation protocol What: Using an LLM to score/rank outputs with explicit controls: dual-order testing, calibration set, per-criterion rubrics. Evidence: Broad research consensus 2024–2025. Position bias = 10–15 point winrate swing (mitigated by running both orders and averaging). Verbosity bias = 15–30 point inflation for longer outputs (mitigated by explicit length-neutrality instructions and per-criterion rubrics). Self-preference bias mitigated by cross-family reviewer. With mitigations, LLM judges correlate acceptably with human judgments on well-specified tasks. CRAAP: Currency 5, Relevance 5, Authority 4, Accuracy 4, Purpose 5 → 4.6 Confidence: Moderate-High (depends on mitigations being applied)

5. Best-of-N + judge What: Generating N candidate responses, then using a judge to select the best. Evidence: Used in RLHF pipelines and test-time compute scaling. Works reliably when the judge is well-calibrated and N is small (3–8). Gains diminish as N scales because candidate variance narrows. Not specifically readability-focused. Confidence: Moderate (well-established for factual tasks; weaker evidence for prose quality)

Tier 3: mixed or weak evidence

6. Self-refine / self-critique (single model, same session) What: Model generates, then critiques its own output and revises. Evidence: Self-Refine (Madaan et al., NeurIPS 2023) showed ~20% absolute improvement across 7 tasks on GPT-4. But more recent work (arXiv:2412.19513, "Confidence vs. Critique") shows LLMs struggle with same-session self-correction; the CCR result (SR2 not better than SR) confirms this directly. The gains appear when the model has strong prior knowledge of the error domain. Confidence: Moderate (surface gains but vulnerable to anchoring bias)

7. Multi-agent debate (MAD) What: Multiple LLM agents debate responses and converge on an answer. Evidence: ICLR 2025 analysis of MAD across 9 benchmarks found it "does not consistently outperform baseline methods including Chain-of-Thought and Self-Consistency." MAD cannot scale with inference budget. Benefits appear mainly on tasks with verifiable ground truth, not open-ended prose quality. Confidence: Low-Moderate (mixed results; promising for factual tasks, unproven for prose)

8. Constitutional critique / critique-revision What: A critic model applies a set of principles ("constitution") to revise the target output. Evidence: Anthropic's Constitutional AI (2022) established the framework; subsequent work (Debate, Reflect, and Distill; arXiv:2506.03541) shows gains in preference optimisation contexts. Evidence for pure prose quality improvement is thinner than for factual/safety properties. Confidence: Moderate (well-evidenced for safety properties; less for prose quality specifically)

Documented failure modes

Bias	Magnitude	Source
Verbosity bias	+15–30 points preference for longer outputs	arXiv:2506.22316, arXiv:2410.21819
Position bias	10–15 point winrate swing from slot order	arXiv:2602.02219
Self-preference / egocentric bias	10–25% inflation for same-family outputs	arXiv:2410.21819, arXiv:2604.22891
Sycophancy in same-session review	CCR study: SR blind spot rate 64.5% avg across 14 models	arXiv:2603.12123
Curse-of-knowledge context bias	Scales with task complexity; affects even LRMs	arXiv:2509.03419
Score range / anchoring bias	Post-hoc calibration required to align with human ratings	arXiv:2510.18196

What raises reliability

• Decomposed per-criterion rubrics (each criterion scored separately)
• Anchor examples (calibration set of 100–300 cases with known correct answers)
• Cross-family reviewers (different training lineage eliminates most self-preference)
• Dual-order testing for pairwise comparisons (average both orderings)
• Context separation (fresh session = no anchoring to production history)
• Adversarial framing in the critique prompt ("find flaws" outperforms "evaluate quality")

Q3: AI-driven readability improvement for technical prose

Why surface metrics mislead on technical prose

Flesch–Kincaid, SMOG, Gunning Fog, and similar formulas estimate grade level from syllable count and sentence length. On technical prose they systematically mislead because:

1. Jargon is multi-syllabic but necessary. "Containerisation," "asynchronous," "authentication" raise syllable counts but their replacement with circumlocutions would reduce clarity, not improve it. The formula cannot distinguish load-bearing terminology from gratuitous complexity.
2. Short sentences with dense technical abbreviations score well. "Use OAuth2 PKCE flow with RS256 JWT signing" is formally Flesch-friendly (short, mostly monosyllabic abbreviations) but opaque to a reader without the prerequisite.
3. LLM-generated prose is optimised for surface fluency. Recent research (arXiv:2510.15345, "Readability Reconsidered") documents that LLM judges underweight lexical and syntactic features classical formulas penalise, responding instead to surface fluency that instruction-tuned models produce. Post-2022, AI-assisted writing scores well on Flesch–Kincaid while remaining cognitively expensive.
4. Domain effect. Systematic literature reviews in legal text (arXiv:2411.09497) and biomedical text simplification (arXiv:2511.05080) both confirm that formula metrics mislead in specialised domains.

Better instruments: structural checks

These checks are operationalisable in a reviewer prompt and map to documented failure modes in technical prose:

Check	What to look for	Why it matters
Buried lead	Key finding/action not in first sentence of paragraph	Forces readers to scan; creates cognitive load
Nominalisation chains	"The implementation of validation of inputs" vs "validate inputs"	Adds syllables and abstraction without clarity
Passive-voice density	>20% passive voice in technical instructions	Removes agent clarity ("the key is rotated" vs "rotate the key")
Filler/hedging intensifiers	"basically," "essentially," "it should be noted that," "in order to"	Adds words, reduces signal-to-noise
One-idea-per-sentence violation	Two concepts in one sentence without clear connective	Increases parsing effort
Jargon without first-use definition	Technical term used before definition	Creates access barrier for non-expert readers
Presupposed context	Reference to prior content without recap	Reader may lack the loaded context (Q1 failure mode)
Parallel structure violation	List items with inconsistent grammatical form	Breaks pattern-matching that aids rapid reading

Plain-language standards

• ISO 24495-1:2023 (Plain language — Part 1): International standard; audience-centred principle: content, structure, design, and language must meet reader needs. Applicable to technical documentation.
• plainlanguage.gov (US federal): Action-verb requirement, active voice, short sentences, reader-first structure. Most applicable to government and API documentation.
• GOV.UK Content Design: Enforces reading age target (9 years), bans nominalisations, requires active voice. Aggressive but operationalisable by an LLM reviewer.
• Microsoft Writing Style Guide / Google Developer Documentation Style Guide: Both mandate active voice, present tense, second person ("you"), define acronyms on first use. Available online for LLM retrieval.

Readability-aware prompting and revision passes

The LLM-Rubric framework (ACL 2024) demonstrates that per-criterion rubric questions outperform holistic quality prompts for text assessment. Applied to readability, this means:

• Do not ask "is this readable?"; ask separately: "Does each paragraph lead with its main point?", "Are all jargon terms defined on first use?", "Is passive voice used more than 20% of the time in instructions?", etc.
• Rubric questions should have explicit scoring criteria, not just "yes/no."
• Anchor examples (a clearly readable paragraph vs. a nominalisation-heavy one) improve scoring calibration.

Evidence from the ICLR 2025 feedback study (arXiv:2504.09737, 20K reviews): LLM feedback that prompted specific structural improvements led 27% of human reviewers to update their reviews, demonstrating that structural critique is actionable.

Evidence gaps

Q1 (unshared-context writing pattern):

• Gap: No paper with this exact phenomenon as its central topic in an agentic writing context.
• Queries attempted: "unshared context LLM writing," "audience awareness LLM generation," "reader model LLM output," "context asymmetry LLM," "presupposition failure LLM writing," "context leakage LLM agentic writing," "egocentric bias LLM output generation."
• Why insufficient: The phenomenon sits at an intersection (agentic tool-use + writing quality + pragmatics) that is newer than the grounding-gap literature (2023) and not yet claimed by a 2024–2026 paper.
• What would fill it: A systematic study measuring how often agentic LLM outputs reference context-window artefacts that aren't surfaced in the response, across different task types and context lengths.

Q2 (LLM-checks-LLM):

• Gap: Cross-family reviewer evidence is strong on factual/code tasks; weaker specifically for prose quality.
• Gap: MAD evidence is mixed; no study specifically uses debate to improve technical writing readability.
• Gap: Constitutional critique evidence is strong for safety alignment; less evidence for prose quality improvement.

Q3 (readability):

• Gap: ISO 24495-1:2023 lacks published empirical validation studies in LLM-generated prose contexts.
• Gap: Structural checks (buried lead, nominalisation density) are well-supported by technical writing literature but lack formal LLM-specific empirical studies showing measurable improvement when used in review prompts.

Methodology

Search strategy: 10 web search queries (the depth limit for this research); 4 full-text reads of the highest-value papers. Pass 1 covered landscape; Pass 2 targeted CCR paper, grounding gaps, curse-of-knowledge, and readability. Early termination triggered by 80%+ source agreement on Q2 bias taxonomy.

Sources analysed: ~32 sources across arXiv papers, ACL/NeurIPS proceedings, technical blogs, and documentation sites.

CRAAP applied: All sources scored; only CRAAP ≥4.0 used for primary claims.

Confidence distribution: 8 High, 11 Moderate, 4 Low, 3 Insufficient confidence findings.

Sources

Q1 (unshared-context pattern)

• Grounding Gaps in Language Model Generations (Shaikh et al., NAACL 2024)
• Curse of Knowledge: When Complex Evaluation Context Benefits yet Biases LLM Judges (arXiv:2509.03419)
• LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High (2025)
• Agentic Knowledgeable Self-awareness (ACL 2025)
• Accommodation and Epistemic Vigilance (arXiv:2601.04435)

Q2 (LLM checks LLM)

• Cross-Context Review: Improving LLM Output Quality (Choi et al., arXiv:2603.12123)
• Self-Preference Bias in LLM-as-a-Judge (arXiv:2410.21819)
• Quantifying and Mitigating Self-Preference Bias of LLM Judges (arXiv:2604.22891)
• LLM-Rubric: A Multidimensional, Calibrated Approach (ACL 2024)
• LLM-Rubric arXiv version
• Judging the Judges: Bias Mitigation Strategies in LLM-as-a-Judge Pipelines (arXiv:2604.23178)
• Am I More Pointwise or Pairwise? Position Bias in Rubric-Based LLM-as-a-Judge (arXiv:2602.02219)
• Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge (arXiv:2510.18196)
• Self-Refine: Iterative Refinement with Self-Feedback (NeurIPS 2023)
• Confidence v.s. Critique: A Decomposition of Self-Correction Capability (arXiv:2412.19513)
• Multi-LLM-Agents Debate Performance, Efficiency, and Scaling (ICLR 2025)
• Multi-Agent Debate for LLM Judges with Adaptive Stability Detection (arXiv:2510.12697)
• Can LLM feedback enhance review quality? ICLR 2025 (arXiv:2504.09737)
• LLMs as Narcissistic Evaluators (ACL 2024)
• Cognitive Biases in Large Language Models: A Survey (arXiv:2412.00323)
• Debate, Reflect, and Distill (arXiv:2506.03541)
• Evaluating Scoring Bias in LLM-as-a-Judge (arXiv:2506.22316)

Q3 (readability)

• Readability Reconsidered: Cross-Dataset Analysis of Reference-Free Metrics (arXiv:2510.15345)
• Making Knowledge Accessible: Readability in Biomedical Text Simplification (arXiv:2511.05080)
• The Use of Readability Metrics in Legal Text: A Systematic Literature Review (arXiv:2411.09497)
• LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation (ACL 2024)