Dissimilarity Measure Machines

This paper presents a dissimilarity-based discriminative framework for learning from data coming in the form of probability distributions. Departing from the use of positive kernel-based methods, we build upon embeddings based on dissimilarities tailored for distribution. We enable this by extending \citet{balcan2008theory}'s theory of learning with similarity functions to the case of distribution-shaped data. Then, we show that several learning guarantees of the dissimilarity still hold when estimated from empirical distributions. Algorithmically, the proposed approach consists in building features from pairwise dissimilarities and in learning a linear decision function in this new feature space. Our experimental results show that this dissimilarity-based approach works better than the so-called support measure machines or the sliced Wasserstein kernel, and that among several dissimilarities including Kullback-Leibler divergence and Maximum Mean Discrepancy, the entropy-regularized Wasserstein distance provides the best compromise between computational efficiency and accuracy