Design and analysis of discriminant pattern classifiers

In recent years pattern recognition has evolved to a mature discipline and has been successfully applied to various problems. A fundamental part of an automatic pattern recognition system is classification, where a pattern vector is assigned to one of a finite number of classes. This thesis reports on the development and design of pattern classifier algorithms, with particular emphasis on statistical algorithms which employ discriminant functions. The first part of this research work investigates the use of linear discriminant functions as pattern classifiers. A comparison of some well known methods, including Perceptron, Widrow-Hoff and Ho-Kashyap, is presented. Using generalised linear modelling a new method of training discriminant functions is developed. In this method the linear discriminant function is transformed by a non-linear link function which associates with each pattern vector a measure which is bounded in the range of 0 to 1 according to the class membership of the pattern. In simulations the GLM approach is applied both to synthetic data and to experimental data from a binary pattern matching problem. It is seen that GLM exhibits faster and more reliable convergence than existing linear discriminant approaches. Extensions of this method to Piecewise linear discriminant functions and to polynomial discriminant functions are explored. Application of self-organising methods for efficient generation of polynomial discriminant functions is also investigated. In the second part of the work a review of neural networks is presented, followed by an analysis and formulation of a popular neural network training algorithm, namely Backpropagation (BP). The capabilities and deficiencies of BP and its variations are experimentally evaluated by computer simulations. An alternative formulation based on Empirical Maximum Likelihood (EML) is also proposed. This approach is shown to have a simpler error landscape in comparison to the original BP based on mean square error. Simulations show that the EML approach generally provides faster convergence, involves fewer calculations per iteration than conventional BP, and results in equally good classification performance