Non-linear Data Structure Extraction using Simple Hebbian Networks

We present a class of neural networks algorithms based on simple Hebbian learning which allow the finding of higher order structure in data. The neural networks use negative feedback of activation to self-organise; such networks have previously been shown to be capable of performing Principal Component Analysis (PCA). In this paper, this is extended to Exploratory Projection Pursuit (EPP) which is a statistical method for investigating structure in high-dimensional data sets. As opposed to previous proposals for networks which learn using Hebbian learning, no explicit weight normalisation, decay or weight clipping is required. The results are extended to multiple units and related to both the statistical literature on EPP and the neural network literature on Non-linear PCA. 1 Introduction Cross-fertilisation between the fields of artificial neural networks and statistics has recently proved fruitful. In unsupervised learning, the realisation that simple neural network architectures ar..