Exploration/exploitation in adaptive recommender systems

ABSTRACT: Interactive information systems are often designed on the basis of little knowledge about users goals and about the final content of the information base. In addition users vary widely in their interests. This makes it useful to give such systems the ability to dynamically adapt to its users. Here we focus on ”recommending ” systems that help a user navigate through the information system. In particular we consider methods for automatically improving the systems recommendation policy on the basis of feedback from the users. We approach this problem from the framework of reinforcement learning. One key idea in reinforcement learning is that exploration of unknown areas of a domain is needed to acquire an optimal policy. We demonstrate with a toy problem that this is an essential element for recommender systems that automatically improve their recommending policy