Multi-dimensional federated learning in recommender systems

A wide range of web services like e-commerce, job-searching, and target advertising heavily rely on recommender systems that find products of interest to fulfill users' diverse and complicated demands. To better model the user preferences and provide satisfactory recommendations, there has been an increasing research focus on constructing more accurate and complete user representations that exploit the user's personal information including the profile and the behavior history. Inevitably, this would induce privacy risks for users in two main aspects: 1) collection of users' sensitive attributes like gender and address; 2) untrustworthy exchange of user data among services. Thus, this natural conflict between user privacy and recommendation accuracy has drawn lots of attention in recent years. One of the most widely studied and verified solution is the federated learning technique. The general idea behind federated learning is to maintain users' critical data on edge devices (e.g. mobile phones) and communicate only the model parameters to the central server. However, a standard federated learning system is designed for a single task with a simple learning objective. In reality, a user typically interacts with various applications every day with heterogeneous intentions. In this dissertation, I will extend the federated learning system to this realistic but more complex multi-objective setting, where multiple federated learning agents collaborate in an environment with multiple central servers and a number of distributed edge devices: a) the set of centralized services possess the collective knowledge of all users but each service collect its own domain data, and b) the set of personalized smart assistants on every user's edge devices maintain the complete data of users and help provide satisfactory personal experiences through interactions with different services. The resulting framework, as a multi-dimensional federated learning paradigm, instantiates the direct intersection between the two trends: demands for a more complete user profile and more protection of user privacy. In the foreseeable future, this problem would become a fundamental piece of the omnipotent personal intelligent assistant. Formally, this problem is a multi-task federated optimization problem, and I identify several main challenges in this work: First, task dimensions are heterogeneous not only in their inherent design of user profiles and learning objectives but also heterogeneous in terms of the users involved in the learning process; Second, the cross-dimension nature induces requirements of extra privacy control among services, in addition to each task's privacy protection of users. Third, the global objective of one service may violate user privacy or conflict with those of other services even in the federated environment. Fourth, collaboration among multiple distributed learning systems may induce extra communication and computation overhead, which makes the entire system unstable and slow. I will present the general framework of multi-dimensional federated learning, and illustrate several solution techniques including privacy-based information management, indirect transfer, and objective transformation that address the aforementioned challenges.Ph.D.Includes bibliographical reference