Managing AI Advice in Crowd Decision-Making

In this paper, we examine how advice from an AI algorithm should be provided to decision-makers that work in a crowd setting. With a theoretical model and numerical experiments we show that the harmful effect of incorrect advice relative to the beneficial effect of correct advice increases with increasing crowd size. Thus, for larger crowds, more advice should be withheld so that it does not negatively affect the crowd accuracy. We propose a mechanism for AI advice personalization that takes the crowd size into account. In an experimental study where subjects classified images, we demonstrate that the crowd size-dependent advice personalization reduces the detrimental effects of incorrect advice and leads to an increase in crowd accuracy