Guess what I think: Essays on the wisdom in meta-predictions

People’s self-reported beliefs, judgments and experiences are highly subjective and can be unreliable. However, such information is also very valuable. Researchers would like to know the true motives of people to understand their behavior. Practitioners can use subjective data to make better decisions. Furthermore, subjective judgments are useful in forecasting. Previous work suggests that the “Wisdom of Crowds” is an effective solution for predicting uncertain quantities. This dissertation develops novel methods to elicit and aggregate subjective information effectively. All methods are based on the following idea: What people think about other people’s judgments (“meta-prediction”) is related to their own judgment on the matter. Chapter 2 proposes a new forecast aggregation algorithm that improves the “Wisdom of Crowds” on the likelihood of an event. Simple average of forecasts could be biased due to common information among the forecasters. The algorithm uses meta-predictions to remove a potential bias in the collective forecast. Chapter 3 presents another solution to the same problem. Forecasters are incentivized such that the collective forecast becomes unbiased. Chapter 4 develops an incentive mechanism to elicit subjective information truthfully. The incentives are based on people’s meta-predictions. Truth-telling participants expect higher rewards. This motivation can improve the quality of the self-reported information. Chapter 5 focuses again on the “Wisdom of Crowds.” The chapter proposes a new algorithm to transform the average probability forecast. The transformed forecasts are much closer to the true probability of the uncertain event. Each chapter in this dissertation introduces a new incentive mechanism or an algorithm. Therefore, this dissertation makes methodological contributions to the literature on elicitation and aggregation of subjective information. Furthermore, each chapter presents experimental results to demonstrate practical effectiveness. The findings suggest that the meta-predictions can be useful. It is also easy to collect meta-predictions in simple surveys. Thus, this dissertation motivates subsequent work that could use meta-predictions even more extensively