Computer-aided detection (CAD) in lung cancer screening at chest MDCT: ROC analysis of CAD versus radiologist performance

To evaluate the performance of a computer-aided detection (CAD) algorithm in the detection of pulmonary nodules on high-resolution multidetector row computed tomography images in a large, homogeneous screening population, and to evaluate the effect of the system output on the performance of radiologists, using receiver operating characteristic analysis. Three radiologists with variable experience (1 to 7 y), independently read the 200 computed tomography scans and assigned each nodule candidate a confidence score (1-2-3: unlikely, probably, and definitely a nodule). CAD was applied to all scans; successively readers reevaluated all findings of the CAD, assigning, in consensus, a confidence score (1 to 3). The reference standard was established by the consensus of 2 experienced radiologists with 30 and 15 years of experience. Results were used to generate an free-response receiver operating characteristic analysis. The reference standard showed 125 nodules. Sensitivity for readers I-II-III was 57%, 68%, and 46%. A double reading resulted in an increase in sensitivity up to 75%. With CAD, sensitivity was increased to 94%, 96%, and 94% for readers I, II, and III. The area under the free-response receiver operating characteristic curve (Az) was 0.72, 0.82, 0.55, and 0.84 for readers I, II, III, and the CAD, when considering all nodules. Differences between readers I-II and CAD were not significant (P=0.9). There was a significant difference between reader III and the CAD. For nodules <6-mm Az was 0.40, 0.47, 0.14, and 0.72 for readers I, II, III, and the CAD. Differences between all readers and the CAD were significant (P