What If... ? Robust Prediction Intervals for Unbalanced Samples

A confidence interval is a standard way of expressing our uncertainty about the value of a population parameter. In survey sampling most methods of confidence interval estimation rely on “reasonable” assumptions to be true in order to achieve nominal coverage levels. Typically these correspond to replacing complex sample statistics by large sample approximations and invoking central limit behaviour. Unfortunately, coverage of these intervals in practice is often much less than anticipated, particularly in unbalanced samples. This paper explores an alternative approach, based on a generalisation of quantile regression analysis, to defining an interval estimate that captures our uncertainty about an unknown population quantity. These quantile-based intervals seem more robust and stable than confidence intervals, particularly in unbalanced situations. Furthermore, they do not involve estimation of second order quantities like variances, which is often difficult and time-consuming for non-linear estimators. We present empirical results illustrating this alternative approach and discuss implications for its use