Large scale hyperspectral data segmentation by random spatial subspace clustering

A novel method called spatial subspace clustering (SpatSC) for 1D hyperspectral data segmentation problem, e.g. hyperspectral data taken from a drill hole, exploring spatial information has been proposed in [1]. The purpose of this exercise is to improve interpretability of the hyperspectral data. The spatial subspace clustering has two major components in its formulation, i.e. data self reconstruction and fused lasso. The first component is mainly to separate different subspaces where data lie on or close to, while the second is to exploit the spatial smoothness based on the observation of stratification of rocks. It produces interpretable and consistent clusters by utilizing the spatial information. However, the implementation of SpatSC requires an optimization of N2 variables, where N is the number of samples in the data set. When N is large, for example, tens of thousands for a typical drill hole data set, the algorithm is no longer suitable for personal computers. To alleviate the computational intensity, we propose to run SpatSC on a randomly chosen calibration set from crude spatial clustering, which is only a small proportion of the whole data set. The final clustering result is then propagated combining the crude spatial clustering and SpatSC results on calibration set. By doing so, the computation cost is reduced by an order of two magnitude compare to the original SpatSC. We applied this random spatial subspace clustering algorithm on real thermal infrared drill hole data set to show its effectiveness