Teach that fluent is not the same as true
Polished, confident prose is exactly what a language model produces when it is wrong. Give students marking codes and make them annotate the answer.
Oct 20, 2026 · 1 min read
Students rate confident prose as credible, and language models produce nothing else. Break that link with a marking exercise: hand them an AI answer and a set of audit codes, and have them annotate it line by line for unsourced statistics, false certainty and flattened complexity. Wineburg and McGrew (2019) found that professional fact-checkers ignore how a page looks and judge it by what lies behind it, while novices anchor on surface polish. (Manning's ai-output-critical-audit-designer skill supplies the codes, a rubric and a teacher modelling script.)
In practice
Annotation key for AI-generated text:
* unsourced statistic or factual claim
! false certainty (confident, no hedge,
no source)
= list without explanation or depth
? complexity or counterargument omitted
~ vague language doing no real work
Mark every paragraph. Then rate the whole text
weak / moderate / strong for accuracy and depth.- Use simple codes: star any unsourced statistic, exclamation mark for false certainty, question mark where complexity was flattened, wave for vague filler.
- Model the first annotation yourself as a think-aloud, explicitly separating how the text sounds from whether it holds up.
- Score the text weak, moderate or strong on clarity, accuracy, depth and breadth; the grade goes to the audit, not to the AI.
- Run the exercise on a genuinely good AI answer occasionally; the lesson is calibration, not blanket distrust.
Quick takeaway: polish is a style property, not an evidence property; make students mark the difference.