Offline handwriting recognition with multidimensional recurrent neural networks

Offline handwriting recognition—the automatic transcription of images of hand-written text—is a challenging task that combines computer vision with sequence learning. In most systems the two elements are handled separately, with sophisti-cated preprocessing techniques used to extract the image features and sequential models such as HMMs used to provide the transcriptions. By combining two re-cent innovations in neural networks—multidimensional recurrent neural networks and connectionist temporal classification—this paper introduces a globally trained offline handwriting recogniser that takes raw pixel data as input. Unlike competing systems, it does not require any alphabet specific preprocessing, and can therefore be used unchanged for any language. Evidence of its generality and power is pro-vided by data from a recent international Arabic recognition competition, where it outperformed all entries (91.4 % accuracy compared to 87.2 % for the competition winner) despite the fact that neither author understands a word of Arabic.