Estimating Subjective Probabilities of Bounded Continuous Distributions Using the Ratio Judgment and Scaling (RJS) Method

Subjective probabilities play a major role in many aspects of judgment and decision making research and in numerous decision analyses applications. There are many diverse methods to elicit subjective probability distributions of continuous variables. However, these methods are susceptible to multiple cognitive biases, such as representativeness, anchoring, confirmation, etc., and depend on the partition of the target variable (Fox & Clemen, 2005). I propose a new methodology for estimating subjective probability distributions of continuous variables that can reduce the impact of many of these biases. In this methodology – labeled Ratio Judgment and Scaling (RJS) – judges are asked to compare all pairs of possible outcomes and identify in each pair which is more likely, and by how much. These relative judgments are easier and more natural, and I hypothesize that they are less susceptible to many of the biases that plague the traditional methods. I did a series of studies to validate and test the RJS new procedure: (1) A simulation study that identified optimal ways of discretizing / binning a continuous distribution for the purpose of applications of the RJS methodology, and (2) two experimental studies contrasting the new RJS method with direct elicitation methods for various partitions. I found that the judges’ estimates of probabilities based on the RJS method were highly consistent, more accurate and less sensitive to the partition procedure than their counterparts based on direct estimates of the same variables. Moreover, the participants rated favorably the procedure. These results provide theoretical and empirical support for the feasibility and validity of the new method and demonstrate its superiority over standard methods. These results combined with the judges’ direct feedback suggest that the RJS methodology is a viable candidate for replacing standard elicitation methods