correspondence RE: Net Risk Reclassification P Values: Valid or Misleading?

Biomedical research of the last few decades has been character-ized by an overreliance on statistical testing. Researchers like many zeros in their P values to convince their audience on the validity of their claims from empirical observations. The area of predictive modeling has been no exception. When the focus shifted from association, eg, expressed by hazard ratios, to the added predictive ability of new biomarkers, P values remained at the forefront in reporting. A contributing factor is the difficult Figure 1. Distributions of model performance improvement statistics in 5000 simulations where four uninformative markers are added to a baseline clinical risk score. All assumptions follow Pepe et al. (6). Event rate 10%. Results based on samples of 420 individuals drawn from a population of 10 000 (as in Pepe et al. [6]) and samples of 4200 individuals drawn from a population of 100 000 (10 times the sizes of Pepe et al. [6]). Left panels illustrate distributions on development samples and right panels show distribution based on models derived on development samples and applied to validation data samples. NRI(0.10) indicates category-based NRI with single risk threshold at 0.10 (equal to event rate), NRI(0.05, 0.20) indicates category-based NRI with categories defined by 0.05 and 0.20 as risk thresholds. TR = training dataset; TS = test dataset. c