On Discovering non-Obvious Recommendations: Using Unexpectedness and Neighborhood Selection Methods in Collaborative Filtering Systems

This paper proposes a number of studies in order to move the field of recommender systems beyond the traditional paradigm and the classical perspective of rating prediction accuracy. We contribute to existing helpful but less explored recommendation strategies and propose new approaches tar-geting to more useful recommendations for both users and businesses. Working toward this direction, we discuss the studies we have conducted so far and present our future re-search plans. The overall goal of this research program is to expand our focus from even more accurate rating pre-dictions toward a more holistic experience for the users, by providing them with non-obvious but high quality recom-mendations and avoiding the over-specialization and con-centration bias problems. In particular, we propose a new probabilistic neighborhood-based method as an improvement of the standard k-nearest neighbors approach, alleviating some of the most common problems of collaborative filtering recommender systems, based on classical metrics of disper-sion and diversity as well as some newly proposed metrics. Furthermore, we propose a concept of unexpectedness in rec-ommender systems and operationalize it by suggesting var-ious mechanisms for specifying the expectations of the users and proposing a recommendation method for providing the users with unexpected but high quality personalized recom-mendations that fairly match their interests. Besides, in or-der to generate utility-based recommendations for Massive Open Online Courses (MOOCs) that better serve the edu-cational needs of students, we study the satisfaction of users with online courses vis-à-vis student retention. Finally, we summarize the conclusions of the conducted studies, discuss the limitations of our work and also outline the managerial implications of the proposed stream of research