The concepts of hidden Markov model in speech recognition

The speech recognition field is one of the most challenging fields that has faced scientists for a long time. The complete solution is still far from reach. The efforts are concentrated with huge funds from the companies to different related and supportive approaches to reach the final goal. Then, apply it to the enormous applications that are still waiting for the successful speech recognisers that are free from the constraints of speakers, vocabularies or environment. This task is not an easy one due to the interdisciplinary nature of the problem and as it requires speech perception to be implied in the recogniser (Speech Understanding Systems) which in turn point strongly to the use of intelligence within the systems. The bare techniques of recognisers (without intelligence) are following wide varieties of approaches with different claims of success by each group of authors who put their faith in their favourite way. However, the sole technique that gains the acceptance of the researchers to be the state of the art is the Hidden Markov Model (HMM) technique. HMM is agreed to be the most promising one. It might be used successfully with other techniques to improve the performance, such as hybridising the HMM with Artificial Neural Networks (ANN) algorithms. This does not mean that the HMM is pure from approximations that are far from reality, such as the successive observations independence, but the results and potential of this algorithm is reliable. The modifications on HMM take the burden of releasing it from these poorly representative approximations hoping for better results. In this report we are going to describe the backbone of the HMM technique with the main outlines for successful implementation. The representation and implementation of HMM varies in one way or another but the main idea is the same as well as the results and computation costs, it is a matter of preferences to choose one. Our preference here is that adopted by Ferguson and Rabiner et al. In this report we will describe the Markov Chain, and then investigate a very popular model in the speech recognition field (the Left-Right HMM Topology). The mathematical formulations needed to be implemented will be fully explained as they are crucial in building the HMM. The prominent factors in the design will also be discussed. Finally we conclude this report by some experimental results to see the practical outcomes of the implemented model.Unpublished[1] J.D. Ferguson, “Hidden Markov Analysis: An Introduction“, in Hidden Markov Models for Speech, Institute of Defence Analyses, Princeton, NJ, 1980. [2] S. E. Levinson, L. R. Rabiner, and M. M. Sondhi, “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition”, The Bell System Technical Journal, vol. 62, no.4, pp.1035-1073, Apr. 1983. [3] L. R. Rabiner, B. H. Juang, “An Introduction to Hidden Markov Models”, IEEE ASSP Magazine, pp. 4 – 16, Jan. 1986. [4] L. R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, vol. 77, no. 2, pp. 257-286, 1989 [5] L. R. Rabiner, B. H. Juang, “Fundamentals of Speech Recognition”, Prentice Hall, Englewood Cliffs, New Jersey, 1993. [6] L.E. Baum and T. Petrie, “Statistical Inference for Probabilistic Functions of Finite State Markov Chains”, Ann. Math. Stat., vol. 37, pp 1554-1563, 1966. [7] L. E. Baum, T. Petrie, G. Soules, and N. Weiss, “A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains”, Ann. Math. Stat., vol. 41, no.1, pp 164-171, 1970. [8] L. E. Baum, “An Inequality and Associated Miximization Technique in Statistical Estimation for Probabilistic Functions of Markov Processes”, Proc. Symp. On Inequalities, vol. 3, pp 1-7, Academic Press, New York and London 1972. [9] J. K. Baker, “The Dragon System – An Overview”, IEEE Trans. Acoustic, Speech, and Signal Processing, vol. ASSP-23, no. 1, pp. 24-29, Feb. 1975. [10] F. J. Owens, “Signal Processing of Speech “, Macmillan Press Ltd., London, 1993. [11] E. Harborg, “Hidden Markov Models Applied to Automatic Speech Recognition”, PhD Thesis, Norwegian Institute of Technology, Trondheim, Aug. 1990. [12] L. E. Baum and J. A. Egon, “An inquality with applications to statistical Estimation for Probabilistic Functions of Markov Process and to a Model for Ecology”, Bull. Amer. Meteorol. Soc., vol.73, pp.360-363, 1967. [13] G. D. Forney, “The Viterbi Algorithm”, Proc. IEEE, vol. 61, pp.268-278, Mar, 1973. [14] Y. Linde, A. Buzo, R. M. Gray “An ALgorithm for Vector Quantizer Design”,IEEE Trans. on Comm. vol. COM-28, no.1, pp. 84-95, 1980. [15] R. M. Gray, “Vector Quantization”, IEEE ASSP Magazine, vol. 1, no. 2, pp. 4 – 29, 1984. [16] F. Jelinek, “Statistical Methods for Speech Recognition”, The MIT Press, 1998