Building Rich Recommender Systems by Modeling Visual, Sequential, and Relational Signals

Modeling and predicting user behavior in recommender systems are challenging as there are various types of signals at play. Traditional item recommendation algorithms mainly focus on modeling signals that indicate users’ preferences, e.g. purchases, clicks, ratings. Despite their success, they typically ignore other signals that are also important for the recommendation task, e.g. visual signals for understanding users’ finer-grained preferences toward appearances of items, sequential signals for exploiting the recommendation context, or relational signals encoding the complex and useful relationships amongst items.Modeling these signals is non-trivial because it requires one to tackle not only ‘standard’ recommender systems challenges such as dealing with large, sparse, and long-tailed datasets, but also new challenges from dealing with the signals themselves, such as the need to model content in terms of its visual appearance, highly subjective sequential behavior, and the heterogeneous ‘relatedness’ amongst items.In this dissertation, we tackle these challenges by building novel models that are scalable, accurate, and intuitive. Empirical results on a wide spectrum of large, real-world datasets indicate that our methods can not only quantitatively improve recommendation accuracy, but also enrich the system’s ability to understand visual interactions (e.g. visual preferences and fashion trends), context (e.g. the impact of recent actions on the future), and complicated relationships amongst items (e.g. complementary or substitutable)