Controlling True Positive Rate in ROC Analysis

ROC analysis is a widely used method for evaluating the performance of classifiers. In analysis involving scarce data sets leave-one-out resampling techniques might be appropriate. This introduces a problem in terms of com-puting average ROC curves necessary to determine vari-ance in the true positive and negative rates. A method to determine decision regions for a specified true positive rate is presented. The method is based on estimating the class specific probability density functions for the two classes. The functions are discretised. Dividing these yields a func-tion where values above or below a specific threshold value corresponds to deciding class one or two respectively. It is shown how a gradual lowering of the threshold value cor-responds to an increase in the true positive rate, and how a true positive rate can be specified and the corresponding threshold determined. An example with simulated data is used to demonstrate the method. 1