Online Optimization of Teaching Sequences with Multi-Armed Bandits

We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activi-ties to maximize skills acquired by each student, taking into account limited time and motivational resources. At a given point in time, the system tries to propose to the student the activity which makes him progress best. We introduce two algorithms that rely on the empirical estimation of the learning progress, one that uses information about the dif-ficulty of each exercise RiARiT and another that does not use any knowledge about the problem ZPDES. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by spe-cific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this op-timization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the ex-pert and allowing the system to deal with didactic gaps in its knowledge. 1