Emotion-Based Reinforcement Learning

Studies have shown that counterfactual reasoning can shape human decisions. However, there is a gap in the litera-ture between counterfactual choices in description-based and experience-based paradigms. While studies using description-based paradigms suggest participants maximize expected sub-jective emotion, studies using experience-based paradigms as-sume that participants learn the values of options and se-lect what maximizes expected utility. In this study, we used computational modeling to test 1) whether participants make emotion-based decisions in experience-based paradigms, and 2) whether the impact of regret depends on its degree of unex-pectedness as suggested by the current regret theory. The re-sults suggest that 1) participants make emotion-based choices even in experience-based paradigms, and 2) the impact of re-gret is greater when it is expected than when it is unexpected. These results challenge the current theory of regret and suggest that reinforcement learning models may need to use counter-factual value functions when full information is provided