A Framework for Offline Evaluation of Recommender Systems based on Probabilistic Relational Models

Recommender systems and their evaluation have been widely studied topics since more than past two decades. Implementation of such systems can be found in numerous commercial and non-commercial software. However, most of the existing open-source/free libraries for recommender systems still deal with single-table data whereas recent studies on recom-mender systems focus on the use of relational (multi-table, multi-entity) data. In our earlier work (Chulyadyo and Leray [2014]), we had presented a personalized recommender system that works with relational data, and is based on Probabilistic Relational Models (PRM), a framework for mod-eling uncertainties present on relational data. With the aim to benefit from existing software for evaluating recommender systems, we propose a framework for evaluating such PRM-based recommender systems in this report. The basic idea is to delegate the tasks of evaluation to an external library while the focus for the implementation of the recommender system under study is only on learning a recommendation model and making recommendations from it. Our proposed evaluation framework is generic, and should not be limited only to PRM-based recommender systems. Any recommender systems dealing with relational data can follow this evaluation framework