Cataloged from PDF version of article.The restricted Neyman–Pearson (NP) approach is studied for composite hypothesis-testing problems in the presence of uncertainty in the prior probability distribution under the alternative hypothesis. A restricted NP decision rule aims to maximize the average detection probability under the constraints on the worst-case detection and false-alarm probabilities, and adjusts the constraint on the worst-case detection probability according to the amount of uncertainty in the prior probability distribution. In this study, optimal decision rules according to the restricted NP criterion are investigated. Also, an algorithm is provided to calculate the optimal restricted NP decision rule. In addition, i...