Multi-level Lasso for Sparse Multi-task Regression

We present a flexible formulation for variable selection in multi-task regression to allow for discrepancies in the estimated sparsity pat-terns accross the multiple tasks, while lever-aging the common structure among them. Our approach is based on an intuitive de-composition of the regression coefficients into a product between a component that is com-mon to all tasks and another component that captures task-specificity. This decomposition yields the Multi-level Lasso objective that can be solved efficiently via alternating opti-mization. The analysis of the “orthonormal design ” case reveals some interesting insights on the nature of the shrinkage performed by our method, compared to that of related work. Theoretical guarantees are provided on the consistency of Multi-level Lasso. Simula-tions and empirical study of micro-array data further demonstrate the value of our frame-work. 1