Re: The Central Role of Receiver Operating Characteristic (ROC) Curves in Evaluating Tests for the Early Detection

Receiver operating characteristic (ROC) curves are a powerful and fl exible tool to identify differentially expressed genes in microarray experiments. This use was discussed by Baker ( 1) in refer-ence to a study by Pepe et al. ( 2), and was described by us ( 3), to illustrate a method of selecting cut-off values to sep-arate different groups of cancer patients. Moreover, Baker indicated an innovative strategy to identify cancer patients that was based on grouping genes selected on the basis of the partial area under the ROC curve (pAUC) and suggested that multiple methods could be developed to generate classifi cation rules. In our opin-ion, this strategy is promising and should not be limited to the framework of the early detection of cancer. In fact, using the more conventional whole AUC for gene selection, bidirectional rules can be obtained, allowing identifi cation of different classes of cancer patients or of subjects differentially exposed to carci-nogenic compounds. To assess the latter hypothesis, we developed a classifi cation method that combined cut-off values at the highest observed specifi city, as illustrated in Fig. 1, using two hypothetical genes. In the fi rst step, the combination of a few genes with high AUC is identifi ed, which allows the best separation between the two groups. In the next steps, the same procedure is repeated recursively on the remaining genes to generate a panel of classifi cation rules. For each class, a score proportional to the AUC is then assigned to each sample. The scores generated at each step are summed with the previous scores to yield, in each rule, the genes with the highest AUC. This method was applied to the data from two online databases ( 4, 5), and the results were validated by complete leave-one-out cross-validation ( 6). The fi rst data set ( 4), generated from 34 current smok-ers and 23 nonsmokers, used Affymetrix chips and included 7329 genes