Inter-rater reliability for pilot and full analysis reviewer groups.

<p>All inter-rater reliabilities were measured using Krippendorff's alpha for ordinal data. *Includes only reviewers whose average scores (across all concepts) were within one standard deviation of the all-reviewer mean within each reviewer group.</p><p>^ Excludes concepts for which >20% of all reviewers deviated from the two most popular scores (i.e., high-disagreement concepts), and includes only reviewers whose average scores (across all included concepts) were within one standard deviation of the resulting all-reviewer mean within each reviewer group.</p><p>Inter-rater reliability for pilot and full analysis reviewer groups.