Pattern Recognition Technologies and Applications

Preface The history of automated pattern recognition can be traced back to the advent of modern computing mid-way through the 20th century. Since that time, the popularity and growth of the pattern recognition field has been fueled by its scientific significance and its applicability to the real world. Pattern recognition is a very challenging and multidisciplinary research area attracting researchers and practitioners from many fields, including computer science, computational intelligence, statistics, engineering, and medical sciences, to mention just a few. Pattern recognition is a process described as retrieving a pattern from a database of known patterns. It has numerous real-world applications in areas such as security, medicine, information processing, and retrieval. Some pattern recognition applications in areas such as handwriting recognition, document retrieval, speech recognition, signature verification, and face recognition are the main focus of the current research activities in the pattern recognition and computational intelligence communities around the globe. Researchers and developers are facing many challenges to applying pattern recognition techniques in many real-world applications. This book consists of 17 peer-reviewed chapters that describe theoretical and applied research work in this challenging area. The state of the art in areas such as handwriting recognition, signature verification, speech recognition, human detection, gender classification, morphological structures for image classification, logic synthesis for image and signal processing, occlusion sequence mining, probabilistic neural networks for EMG patterns, multi-objective clustering ensembles, evolutionary ensembles, support vector machines for biomedical data, and unified support vector machines is presented in various chapters of this book. The first two chapters focus on off-line cursive handwriting recognition. In Chapter I, Verma and Blumenstein review existing handwriting recognition techniques and present the current state of the art in cursive handwriting recognition. Standard handwriting recognition processes are presented, and each process is described in detail. Some novel segmentation strategies and a segmentation-based approach for automated recognition of unconstrained cursive handwriting are also presented. In Chapter II, Uchida investigates the theoretical and practical importance of elastic matching for handwriting recognition. He argues that the use of elastic matching techniques instead of rigid matching techniques improves the robustness of handwriting recognition systems. In addition, the optimized matching represents the deformation of handwritten characters and, thus, is useful for statistical analysis of the deformation. Elastic matching is formulated as an optimization problem of planar matching, or pixel-to-pixel correspondence, between two character images under a certain matching model such as affine and nonlinear. The next two chapters focus on off-line signature verification. In Chapter III, Batista, Rivard, Sabourin, and Granger present the current state of art in automatic signature verification. Automatic signature verification is a biometric method that can be applied in all situations where handwritten signatures are used, such as cashing a check, signing a credit card, and authenticating a document. They review existing approaches in the literature and present a survey of the most important techniques used for feature extraction and verification in this field. They also present strategies used for problems such as limited amounts of data and show important challenges and some new research directions. xiv In Chapter IV, Madasu and Lovell present an off-line signature verification and forgery detection system based on fuzzy modeling. The various handwritten signature characteristics and features are first studied and encapsulated to devise a robust verification system. The verification of genuine signatures and detection of forgeries is achieved via angle features extracted using a grid method. The derived features are fuzzified by an exponential membership function, which is modified to include two structural parameters. The structural parameters are devised to take into account the possible variations due to handwriting styles and to reflect other factors affecting the scripting of a signature. The proposed system has been tested on a large database of signatures comprising more than 1,200 signature images obtained from 40 volunteers. Chapters V and VI focus on speech recognition. In Chapter V, Suárez-Guerra and Oropeza-Rodriguez present the state of the art in automatic speech recognition. Speech recognition is very challenging for researchers in many fields, including computer science, mathematical statistics, applied artificial intelligence, and linguistics. The unit of essential information used to characterize the speech signal in the most widely used ASR systems is the phoneme. However, several researchers recently have questioned this representation and demonstrated the limitations of the phonemes, suggesting that ASR with better performance can be developed replacing the phoneme by triphones and syllables as the unit of essential information used to characterize the speech signal. This chapter presents an overview of the most successful techniques used in ASR systems, together with some recently proposed ASR systems that intend to improve the characteristics of conventional ASR systems. In Chapter VI, Leedham, Pervouchine, and Zhong investigate features of handwriting and speech and their effectiveness at determining whether the identity of a writer or speaker can be identified from handwriting or speech. For handwriting, some of the subjective and qualitative features used by document examiners are investigated in a scientific and quantitative manner based on the analysis of three characters (d, y, and f) and the grapheme th. For speech, several frequently used features are compared for their strengths and weaknesses in distinguishing speakers. The results show that some features do have good discriminative power, while others are less effective. Acceptable performance can be obtained in many situations using these features. However, the effect of handwriting forgery/disguise or conscious speech imitation/alteration on these features is not investigated. New and more powerful features are needed in the future if high accuracy person identification can be achieved in the presence of disguise or forgery. In Chapter VII, Yu, Pham, and Yan present a new pattern recognition method using morphological structure. First, smooth linearization is introduced based on various chain codes. Second, morphological structural points are described in terms of smooth followed contours and linearized lines, and then the patterns of morphological structural points and their properties are given. Morphological structural points are basic tools for pattern recognition-based morphological structure. Furthermore, how the morphological structure can be used to recognize and classify images is presented. One application is document image processing and recognition, analysis, and recognition of broken handwritten digits. Another one is dynamic analysis and recognition of cell-cycle screening based on morphological structures. In Chapter VIII, Shan, Bigdeli, Lovell, and Chen propose a variability compensation technique that synthesizes realistic frontal face images from nonfrontal views. It is based on modeling the face via active appearance models and estimating the pose through a correlation model. The proposed technique is coupled with adaptive principal component analysis (APCA), which was previously shown to perform well in the presence of both lighting and expression variations. The proposed recognition techniques, although advanced, are not computationally intensive. So they are quite well suited to the embedded system environment. Indeed, the authors have implemented an early prototype of a face recognition module on a mobile camera phone so the camera could be used to identify the person holding the phone. xv In Chapter IX, Guha, Mukerjee, and Venkatesh present complex multi-object interactions resulting in occlusion sequences that are a visual signature for the event. In this chapter, multi-object interactions are tracked using a set of qualitative occlusion primitives derived on the basis of the persistence hypothesis— objects continue to exist even when hidden from view. Variable length temporal sequences of occlusion primitives are shown to be well correlated with many classes of semantically significant events. In surveillance applications, determining occlusion primitives is based on foreground blob tracking and requires no prior knowledge of the domain or camera calibration. New foreground blobs are identified as putative objects that may undergo occlusions, split into multiple objects, merged back again, and so forth. Significant activities are identified through temporal sequence mining, which bear a high correlation with semantic categories. Thus, semantically significant event categories can be recognized without assuming camera calibration or any environmental/object/action model prior. In Chapter X, Jia and Zhang review human detection techniques. Human detection is the first step for a number of applications such as smart video surveillance, driving assistance systems, and intelligent digital content management. It is a challenging problem due to the variance of illumination, color, scale, pose, and so forth. This chapter reviews various aspects of human detection in static images and focuses on learning-based methods that build classifiers using training samples. There are usually three modules for these methods: feature extraction, classifier design, and merging of overlapping detections. The chapter reviews most of the existing methods for each module and analyzes their respective pros and cons. The contribution includes two aspects: first, the performance of existing feature sets on human detection are compared; second, a fast human detection system based on the histogram of oriented gradients features and a cascaded Adaboost classifier is proposed. This chapter is useful for both algorithm researchers and system designers in the computer vision and pattern recognition communities. In Chapter XI, Tivive and Bouzerdoum present a brain-inspired pattern recognition architecture. With the ever-increasing utilization of imagery in scientific, industrial, civilian, and military applications, visual pattern recognition has been thriving as a research field and has become an essential enabling technology for many applications. In this chapter, a brain-inspired pattern recognition architecture that easily can be adapted to solve various real-world visual pattern recognition tasks is presented. The architecture has the ability to extract visual features from images and classify them within the same network structure; in other words, it integrates the feature extraction stage with the classification stage, and both stages are optimized with respect to one another. The main processing unit for feature extraction is governed by a nonlinear biophysical mechanism known as shunting inhibition, which plays a significant role in visual information processing in the brain. The proposed architecture is applied to four real-world visual pattern recognition problems; namely, handwritten digit recognition, texture segmentation, automatic face detection, and gender recognition. Experimental results demonstrate that the proposed architecture is very competitive with and sometimes outperforms existing state-of-the-art techniques for each application. In Chapter XII, Rawski, Selvaraj, Falkowski, and Luba present the discussion on efficiency of various implementation methodologies of DSP algorithms targeting modern FPGA architectures. Nowadays, programmable technology provides the possibility of implementing digital systems with the use of specialized embedded DSP blocks. In the first place, however, this technology gives the designer the possibility to increase the efficiency of designed systems by exploitation of parallelisms of implemented algorithms. Moreover, it is possible to apply special techniques such as distributed arithmetic (DA). Since in this approach general-purpose multipliers are replaced by combinational LUT blocks, it is possible to construct digital filters of very high performance. Additionally, application of the functional decomposition- based method to LUT block optimization and mapping has been investigated. The chapter presents results of the comparison of various design approaches in these areas. xvi In Chapter XIII, Zhou and Wang present an approach to class-dependent feature selection and a novel support vector machine (SVM). The relative background and theory are presented for describing the proposed method, and real applications of the method on several biomedical datasets are demonstrated. The authors hope that this chapter can provide readers with a different view of the feature selection method and also the classifier so as to promote more promising methods and applications. In Chapter XIV, Shilton and Palaniswami present a unified introduction to support vector machine (SVM) methods for binary classification, one-class classification, and regression. The SVM method for binary classification (binary SVC) is introduced first and then extended to encompass one-class classification (clustering). Next, using the regularized risk approach as a motivation, the SVM method for regression (SVR) is described. These methods are then combined to obtain a single, unified SVM formulation that encompasses binary classification, one-class classification, and regression (as well as some extensions of these), and the dual formulation of this unified model is derived. A mechanical analogy for the binary and one-class SVCs is given to provide an intuitive explanation of the operation of these two formulations. Finally, the unified SVM is extended to implement general cost functions, and an application of SVM classifiers to the problem of spam e-mail detection is considered. In Chapter XV, Faceli, Carvalho, and Souto investigate multi-objective clustering ensembles for clustering techniques. Clustering is an important tool for data exploration. Several clustering algorithms exist, and new algorithms are frequently proposed in the literature. These algorithms have been very successful in a large number of real-world problems. However, there is no clustering algorithm, optimizing only a single criterion, able to reveal all types of structures (homogeneous or heterogeneous) present in a dataset. In order to deal with this problem, several multi-objective clustering and cluster ensemble methods have been proposed in the literature, including a multi-objective clustering ensemble algorithm. In this chapter, an overview of these methods, which, to a great extent, are based on the combination of various aspects from traditional clustering algorithms, is presented. In Chapter XVI, Duell and Yao present negative correlation learning in evolutionary ensembles with suitable speciation techniques. Negative correlation learning (NCL) is a technique that attempts to create an ensemble of neural networks whose outputs are accurate but negatively correlated. The motivation for such a technique can be found in the bias-variance-covariance decomposition of an ensemble of the learner’s generalisation error. NCL is also increasingly used in conjunction with an evolutionary process, which gives rise to the possibility of adapting the structures of the networks at the same time as learning the weights. This chapter examines the motivation and characteristics of the NCL algorithm. Some recent work relating to the implementation of NCL in a single objective evolutionary framework for classification tasks is presented, and the authors examine the impact of two different speciation techniques: implicit fitness sharing and an island model population structure. The choice of such speciation techniques can have a detrimental effect on the ability of NCL to produce accurate and diverse ensembles and should, therefore, be chosen carefully. This chapter also provides an overview of other researchers’ work with NCL and gives some promising future research directions. In Chapter XVII, Tsuji, Bu, and Fukuda present a recurrent probabilistic neural network for EMG pattern recognition. In the field of pattern recognition, probabilistic neural networks (PNNs) have been proven as an important classifier. For pattern recognition of EMG signals, the characteristics usually used are amplitude, frequency, and space. However, significant temporal characteristics exist in the transient and nonstationary EMG signals, which cannot be considered by traditional PNNs. In this chapter, a recurrent PNN called recurrent log-linearized Gaussian mixture network (R-LLGMN) is introduced for EMG pattern recognition, with the emphasis on utilizing temporal characteristics. The structure of R-LLGMN is based on the algorithm of a hidden Markov model (HMM), which is a routinely used technique for modeling stochastic time series. Since R-LLGMN inherits advantages from both HMM and xvii neural computation, it is expected to have a higher representation ability and show better performance when dealing with time series such as EMG signals. Experimental results show that R-LLGMN can achieve high discriminant accuracy in EMG pattern recognition. Brijesh Verma, Central Queensland University, Australia Michael Blumenstein, Griffith University, Australia Editor