Attention Mechanisms in Hierarchical Session-Based Recommendation

The use of attention mechanisms in different applications of recurrent neural networks has yielded significantly higher accuracies, but their use in session-based recommender systems is largely unexplored. In addition to increasing accuracy, attention mechanisms also allow for easy visualization of which input items impact the prediction of new outputs, which can lead to a better understanding of how users act, their habits and which past items are important for predicting future ones. In this project we explore different ways of including attention mechanisms in session-based recommender systems with hierarchical recurrent neural networks to improve accuracy. Four experimental models have been developed that apply attention mechanisms in different ways. These models have been tested on two separate datasets. We show that the attention mechanisms used in problems such as text translation and document classification yield a small increase in accuracy in the recommender system problem, with a slightly higher increase when combined with the use of a simple temporal attention mechanism. While we have been unable to achieve the same accuracy improvements as attention mechanisms have achieved in other problem domains, our improvements together with attention weight visualizations suggest that a temporal attention mechanism can work in recommender systems and that a more sophisticated temporal attention mechanism could increase accuracy even more