Convex optimization, shape constraints, compound decisions, and empirical Bayes rules

Abstract. Estimation of mixture densities for the classical Gaussian com-pound decision problem and their associated (empirical) Bayes rules is considered from two new perspectives. The first, motivated by Brown and Greenshtein (2009), introduces a nonparametric maximum likelihood estimator of the mixture density subject to a monotonicity constraint on the resulting Bayes rule. The second, motivated by Jiang and Zhang (2009), proposes a new approach to computing the Kiefer-Wolfowitz non-parametric maximum likelihood estimator for mixtures. In contrast to prior methods for these problems, our new approaches are cast as con-vex optimization problems that can be efficiently solved by modern inte-rior point methods. In particular, we show that the reformulation of the Kiefer-Wolfowitz estimator as a convex optimization problem reduces the computational effort by several orders of magnitude for typical problems, by comparison to prior EM-algorithm based methods, and thus greatly ex-pands the practical applicability of the resulting methods. Our new pro-cedures are compared with several existing empirical Bayes methods in simulations employing the well-established design of Johnstone and Sil-verman (2004). Some further comparisons are made based on prediction of baseball batting averages. A Bernoulli mixture application is briefly considered in the penultimate section