Implicit Value Updating Explains Transitive Inference Performance: The Betasort Model

<div><p>Transitive inference (the ability to infer that <i>B</i> > <i>D</i> given that <i>B</i> > <i>C</i> and <i>C</i> > <i>D</i>) is a widespread characteristic of serial learning, observed in dozens of species. Despite these robust behavioral effects, reinforcement learning models reliant on reward prediction error or associative strength routinely fail to perform these inferences. We propose an algorithm called <i>betasort</i>, inspired by cognitive processes, which performs transitive inference at low computational cost. This is accomplished by (1) representing stimulus positions along a unit span using beta distributions, (2) treating positive and negative feedback asymmetrically, and (3) updating the position of every stimulus during every trial, whether that stimulus was visible or not. Performance was compared for rhesus macaques, humans, and the betasort algorithm, as well as <i>Q</i>-learning, an established reward-prediction error (RPE) model. Of these, only <i>Q</i>-learning failed to respond above chance during critical test trials. Betasort’s success (when compared to RPE models) and its computational efficiency (when compared to full Markov decision process implementations) suggests that the study of reinforcement learning in organisms will be best served by a feature-driven approach to comparing formal models.</p>