Local Rademacher Complexity for Multi-Label Learning

We analyze the local Rademacher complexity of empirical risk minimization-based multi-label learning algorithms, and in doing so propose a new algorithm for multi-label learning. Rather than using the trace norm to regularize the multi-label predictor, we instead minimize the tail sum of the singular values of the predictor in multi-label learning. Benefiting from the use of the local Rademacher complexity, our algorithm, therefore, has a sharper generalization error bound. Compared with methods that minimize over all singular values, concentrating on the tail singular values results in better recovery of the low-rank structure of the multi-label predictor, which plays an important role in exploiting label correlations. We propose a new conditional singular value thresholding algorithm to solve the resulting objective function. Moreover, a variance control strategy is employed to reduce the variance of variables in optimization. Empirical studies on real-world data sets validate our theoretical results and demonstrate the effectiveness of the proposed algorithm for multi-label learning.National Key Technology Research and Development Program [2015BAF15B00]; National Natural Science Foundation of China [61375026]; Australian Research Council [DP-140102164, FT-130101457, LP-140100569]SCI(E)EIARTICLExuchang@pku.edu.cn; tliang.liu@gmail.com; dacheng.tao@uts.edu.au; xuchao@cis.pku.edu.cn31495-15072