Scalable HMM-based inference engine in large vocabulary continuous speech recognition

Parallel scalability allows an application to efficiently uti-lize an increasing number of processing elements. In this pa-per we explore a design space for parallel scalability for an in-ference engine in large vocabulary continuous speech recog-nition (LVCSR). Our implementation of the inference engine involves a parallel graph traversal through an irregular graph-based knowledge network with millions of states and arcs. The challenge is not only to define a software architecture that exposes sufficient fine-grained application concurrency, but also to efficiently synchronize between an increasing number of concurrent tasks and to effectively utilize parallelism op-portunities in today’s highly parallel processors. We propose four application-level implementation al-ternatives we call “algorithm styles ” and construct highly optimized implementations on two parallel platforms: an Intel Core i7 multicore processor and a NVIDIA GTX280 manycore processor. The highest performing algorithm style varies with the implementation platform. On a 44-minute speech data set, we demonstrate substantial speedups of 3.4× on Core i7 and 10.5 × on GTX280 compared to a highly optimized sequential implementation on Core i7 without sac-rificing accuracy. The parallel implementations contain less than 2.5 % sequential overhead, promising scalability and significant potential for further speedup on future platforms. 1