CLUSTERING, CODING, AND THE CONCEPT OF SIMILARITY

ABSTRACT: This paper develops a theory of cluster-ing and coding which combines a geometric model with a probabilistic model in a principled way. The geomet-ric model is a Riemannian manifold with a Riemannian metric, gij(x), which we interpret as a measure of dis-similarity. The probabilistic model consists of a stochas-tic process with an invariant probability measure which matches the density of the sample input data. The link between the two models is a potential function, U(x), and its gradient, ∇U(x). We use the gradient to de-fine the dissimilarity metric, which guarantees that our measure of dissimilarity will depend on the probability measure. Finally, we use the dissimilarity metric to de-fine a coordinate system on the embedded Riemannian manifold, which gives us a low-dimensional encoding o