Constraints versus Priors

Abstract: Many scientific problems have unknown parameters that are thought to lie in some known set. For instance, the amount of energy absorbed by an x-ray specimen must be between 0 and 100 % of the incident energy. Similar constraints arise in expressing “epistemic” uncertainty. Such prior information can be handled directly by frequentist methods. Bayesian methods require supplementing the constraint with a prior probability distribution for the parameter. This can cause frequentist and Bayesian estimates, and the nominal uncertainties of those estimates, to differ substantially. Moreover, Bayesian and frequentist definitions of uncertainty may sound similar, but they measure quite different things. For instance, Bayesian uncertainties generally involve expectations with respect to the posterior distribution of the parameter, holding the data fixed, while frequentist uncertainties generally involve expectations with respect to the distribution of the data, holding the parameter fixed. This paper gives simple examples where “uninformative ” priors are in fact extremely informative, and sketches how to measure how much information the prior adds to the constraint