Hierarchical Multi-task Learning

Traditionally, machine learning research has adopted methods that were designed to learn one or a set of machine learning tasks independently. However, motivated by our brain's learning mechanism to transfer knowledge from past and other related experiences, recent research has developed and studied methods incorporating target task relationships in the learning algorithms. The area of machine learning in which multiple target tasks are solved simultaneously while exploiting their similarities and underlying structures is known as multi-task learning. Multi-task learning methods (MTL) have proven effective in learning improved machine learning models by facilitating the transfer of knowledge through simultaneously learning a set of target tasks. However, the success of existing multi-task learning methods depends on the extent of the similarity between the target tasks. When tasks are not sufficiently similar, the negative transfer that impacts the quality of the learned models may occur. Therefore, new techniques were adopted that took advantage of task clusters, task-task relatedness, or an asymmetric knowledge transfer. However, none of these techniques are adequate when applied to a large number of heterogeneous tasks organized in a complex hierarchical structure. The abundance of such hierarchies in many domains, including health-care, document classification, and image classification, motivates the development of a new class of multi-task learning methods that can take advantage of these complex hierarchical task relationships. In this thesis, we explore and develop supervised multi-task learning methods that leverage existing task hierarchies to guide the transfer of knowledge between related tasks and evaluate these methods in the context of healthcare applications. First, we propose a simple, yet flexible, approach for learning low-dimensional representations of patients' electronic health records data that are able to overcome challenges related to learning of the models for multiple target tasks from such data. Second, we propose new hierarchical multi-task learning methods that enable the transfer of knowledge in the form of parameter transfer. Third, we study and present new feature-based hierarchical multi-task learning methods that utilize feature transfer instead of parameter transfer solutions to further improve the performance of the models. Finally, we discuss the open questions and problems and provide ideas for future research directions