The Value of Bayes Theorem in the Interpretation of Subjective Diagnostic Findings: What Can We Learn from Agreement Studies?

The performance characteristics of a detectionsystem are usually described in terms of accu-racy and reliability. Accuracy (or validity) is defined as the ability of a detection system to correctly identify the true underlying state.1 Accuracy is often described in terms of sensitivity (Sn) and specificity (Sp), which are the probabilities of correctly classi-fying a diseased and nondiseased subject, respec-tively. Reliability (or precision) is defined as the ability of a test to give consistent results in repeated applications, regardless of the actual state of the disease.2 Two common measures of reliability are within-observer (dwo) and between-observer (dbo) disagreement. Within-observer disagreement is the probability of 2 inconsistent observations on a ran-dom subject by the same observer. Between-observe