A Scalable Recommender System for Automatic Playlist Continuation

As major companies like Spotify, Deezer and Tidal look to improve their music streamingproducts, they repeatedly opt for features that engage with users and lead to a morepersonalised user experience. Automatic playlist continuation enables these platforms tosupport their users with a seamless and smooth interface to enjoy music, own their experience,and discover new songs and artists.This report details a recommender system that enables automatic playlist continuation;providing the recommendation of music tracks to users who are creating new playlists or curatingexisting ones. The recommendation framework given in this report is able to provide accurateand pertinent track recommendation, but also addresses issues of scalability, practicalimplementation and decision transparency, so that commercial enterprises can deploy such asystem more easily and develop a winning strategy for their user experience. Furthermore, therecommender system does not require any rich and varied supply of user data, instead requiringonly basic information as input such as the title of the playlist, the tracks currently in the playlist,and the artists associated with those tracks.To accomplish these goals, the system relies on user-based collaborative filtering; a simple, wellestablishedmethod of recommendation, supported by web-scraping and topic modellingalgorithms that creatively use the supplied data to paint a more holistic image of what kind ofplaylist the user would like. This system was developed using data from the Million PlaylistDataset, released by Spotify in 2018 as part of the Recommender Systems Challenge, evaluatedusing R-precision, normalised discounted cumulative gain, and a proprietary evaluation metriccalled Recommended Song Clicks, that reflects the number of times a user would have to refreshthe list of recommendations provided if the current Spotify user interface was used tocommunicate them. Over an 80:20 train-test split, the scores were: 0.343, 0.224, and 15.73