Mechanism Design for eliciting probabilistic estimates from multiple suppliers with unknown costs and limited precision

This paper reports on the design of a novel two-stage mechanism, based on strictly proper scoring rules, that allows a centre to acquire a costly probabilistic estimate of some unknown parameter, by eliciting and fusing estimates from multiple suppliers. Each of these suppliers is capable of producing a probabilistic estimate of any precision, up to a privately known maximum, and by fusing several low precision estimates together the centre is able to obtain a single estimate with a specified minimum precision. Specifically, in the mechanism’s first stage M from N agents are pre-selected by eliciting their privately known costs. In the second stage, these M agents are sequentially approached in a random order and their private maximum precision is elicited. A payment rule, based on a strictly proper scoring rule, then incentivises them to make and truthfully report an estimate of this maximum precision, which the centre fuses with others until it achieves its specified precision. We formally prove that the mechanism is incentive compatible regarding the costs, maximum precisions and estimates, and that it is individually rational. We present empirical results showing that our mechanism describes a family of possible ways to perform the pre-selection in the first stage, and formally prove that there is one that dominates all others