Semiparametric estimation of probabilistic index models : efficiency and bias

Many well-known rank tests can be viewed as score tests under probabilistic index models (PIMs), that is, regression models for the conditional probability that the outcome of one randomly chosen subject exceeds the outcome of another independently chosen subject. PIMs provide a natural regression framework for nonparametric rank tests. In addition, PIMs supplement these tests with effect sizes and ease the development of more flexible tests, such as tests that allow for covariate adjustment. Inferences for PIMs are currently based on an estimator, referred to as the standard estimator, that is derived heuristically. By appealing to semiparametric theory and a Hoeffding decomposition, we rigorously derive the class of all consistent and asymptotically normal estimators for the parameters indexing a PIM. We identify the (locally) semiparametric efficient estimator in this class, and derive a second estimator with a smaller second-order finite-sample bias. The properties of the estimators are evaluated theoretically and empirically. The heuristic standard estimator turns out to be the preferred estimator in practice, because it is computationally superior to both the efficient and the bias-reduced estimators, and only suffers from a minor loss in efficiency. We also propose a partition strategy to further improve the computational performance of the standard estimator