Curriculum Learning of Multiple Tasks

Sharing information between multiple tasks enables al-gorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally re-lated to each other, hence it could be advantageous to trans-fer information only between the most related tasks. In this work we propose an approach that processes mul-tiple tasks in a sequence with sharing between subsequent tasks instead of solving all tasks jointly. Subsequently, we address the question of curriculum learning of tasks, i.e. finding the best order of tasks to be learned. Our approach is based on a generalization bound criterion for choosing the task order that optimizes the average expected classi-fication performance over all tasks. Our experimental re-sults show that learning multiple related tasks sequentially can be more effective than learning them jointly, the order in which tasks are being solved affects the overall perfor-mance, and that our model is able to automatically discover a favourable order of tasks. 1