A MCMC approach to Hierarchical MixtureModelling

Abstract There are many hierarchical clustering algorithms available, but theselack a firm statistical basis. Here we set up a hierarchical probabilistic mixture model, where data is generated in a hierarchical tree-structuredmanner. Markov chain Monte Carlo (MCMC) methods are demonstrated which can be used to sample from the posterior distribution over treescontaining variable numbers of hidden units. 1 Introduction Over the past decade or two mixture models have become a popular approach to clusteringor competitive learning problems. They have the advantage of having a well-defined objective function and fit in with the general trend of viewing neural network problems in astatistical framework. However, one disadvantage is that they produce a "flat " cluster structure rather than the hierarchical tree structure that is returned by some clustering algorithmssuch as the agglomerative single-link method (see e.g. [12]). In this paper I formulate a hierarchical mixture model, which retains the advantages of the statistical framework, butalso features a tree-structured hierarchy. The basic idea is illustrated in Figure 1(a). At the root of the tree (level 1) we have a singlecentre (marked with