Calibration

Calibration is the recurring session where reviewers grade the same conversation independently, then compare and reconcile the differences until the rubric means the same thing to everyone. It is the maintenance that keeps a QA program honest.

There's no single formula, though teams track inter-rater agreement or score variance as a health signal.

What it hides: calibration reveals uncomfortable truths — that your rubric is ambiguous, or that reviewers carry different standards — and skipping it doesn't make the disagreement go away, it just moves it off the record. A QA score without calibration behind it is precision without accuracy.