It's new, but is it good? How generalization and uncertainty guide the exploration of novel options

How do people decide whether to try out novel options as opposed to tried-and-testedones? We argue that they infer a novel option’s reward from contextual informationlearned from functional relations and take uncertainty into account when making adecision. We propose a Bayesian optimization model to describe their learning and decisionmaking. This model relies on similarity-based learning of functional relationships betweenfeatures and rewards, and a choice rule that balances exploration and exploitation bycombining predicted rewards and the uncertainty of these predictions. Our model makestwo main predictions. First, decision makers who learn functional relationships willgeneralize based on the learned reward function, choosing novel options only if theirpredicted reward is high. Second, they will take uncertainty about the function intoaccount, and prefer novel options that can reduce this uncertainty. We test thesepredictions in two preregistered experiments in which we examine participants’ preferencesfor novel options using a feature-based multi-armed bandit task in which rewards are anoisy function of observable features. Our results reveal strong evidence for functionalexploration and moderate evidence for uncertainty-guided exploration