Modelling functions from sample data with classification applications

In this thesis we investigate various aspects of the pattern recognition problem solving process. Pattern recognition can be viewed as a decision making process where the underlying density functions or discriminant functions of the application have to be estimated often in a high dimensional space. We consider two main types of estimators: the feed-forward neural network and the nearest neighbor method.In the first part of the thesis we investigate the optimization problem that is solved when using feed-forward neural networks for function approximation. We find that the feed-forward neural network optimization problem is very ill-conditioned and can influence the solution process severely. We also show how the feed-forward neural network function and its gradient can be implemented using automatic differentiation techniques.The second part of the thesis is concerned with the nearest neighbor method. We present two new continuous, supervised learning methods: a novel memory-based learning technique and an approximate nearest neighbor method, both approximating the convergence properties of the nearest neighbor method. These methods can be used in continuous learning areas such as speech, hand-writing and financial applications. We also present a fast approximate search method for high dimensional spaces that is based on the k-d-tree. A lower bound on the performance of the method is derived and some results are shown for a uniform distribution. An application of this method to a speech data set shows very promising results.U of I OnlyETDs are only available to UIUC Users without author permissio