Recycling weak labels for multiclass classification

This paper explores the mechanisms to efficiently combine annotations of different quality for multiclass classification datasets, as we argue that it is easier to obtain large collections of weak labels as opposed to true labels. Since labels come from different sources, their annotations may have different degrees of reliability (e.g., noisy labels, supersets of labels, complementary labels or annotations performed by domain experts), and we must make sure that the addition of potentially inaccurate labels does not degrade the performance achieved when using only true labels. For this reason, we consider each group of annotations as being weakly supervised and pose the problem as finding the optimal combination of such collections. We propose an efficient algorithm based on expectation-maximization and show its performance in both synthetic and real-world classification tasks in a variety of weak label scenarios.This work was supported by FEDER/Ministry of Science, Innovation and Universities - State Agency of Research [grant TEC2017-83838-R] and also by the EU Horizon-2020 Research and Innovation programme [grant 824988, Project MUSKETEER]. The work of MPN and RSR was supported by the SPHERE Next Steps Project funded by the UK Engineering and Physical Sciences Research Council (EPSRC) [grant EP/R005273/1]. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research