Multitask Learning using Nonparametrically Learned Predictor Subspaces

Given several related learning tasks, we propose a nonparametric Bayesian learn-ing model that captures task relatedness by assuming that the task parameters (i.e., weight vectors) share a latent subspace. More specifically, the intrinsic dimen-sionality of this subspace is not assumed to be known a priori. We use an infinite latent feature model- the Indian Buffet Process- to automatically infer this num-ber. We also propose extensions of this model where the subspace learning can incorporate (labeled, and additionally unlabeled if available) examples, or the task parameters share a mixture of subspaces, instead of sharing a single subspace. The latter property can allow learning nonlinear manifold structure underlying the task parameters, and can also help in preventing negative transfer from outlier tasks.