On-Line Cursive Handwriting Recognition Using Hidden Markov Models and Statistical Grammars

On-Line Cursive Handwriting Recognition Using Hidden Markov Models and Statistical Grammars John Makhoul, Thad Starner~, Richard Schwartz, and George Chou BBN Systems and Technologies 70 Fawcett Street Cambridge, MA 02138 Email: Makhoul @bbn.com ABSTRACT The BYBLOS continuous speech recognition system is applied to on-line cursive handwriting recognition. By exploiting similarities between on-line cursive handwriting and continuous speech recogni- tion, we can use the same base system adapted to handwriting feature vectors instead of speech. The use of hidden Markov models obvi- ates the need for segmentation of the handwritten script sentences before recognition. To test our system, we collected handwritten sentences using text from the ARPA Airline Travel Information Ser- vice (ATIS) and the ARPA Wall Street Journal (WSJ) corpora. In an initial experiment on the ATIS data, a word error rate of 1.1% was achieved with a 3050-word lexicon, 52-character set, collected from one writer. In a subsequent writer-dependent test on the WSJ data, error rates ranging between 2%-5% were obtained with a 25,595- word lexicon, 86-character set, collected from six different writers. Details of the recognition system, the data collection process, and analysis of the experiments are presented. I. INTRODUCTION The segmentation of written words into component characters is of- ten the first step of handwriting recognition systems [1]. In some cases, segmentation is forced on the user by providing boxes for the writing of discrete letters. However, in modem continuous speech recognition efforts, segmentation of phonemes is not performed be- fore either of the training or the recognition steps. Instead, segmenta- tion occurs simultaneously with recognition. If such a system could be adapted for handwriting, the very difficult and time consuming issue of segmentation could be avoided. This paper addresses such a system, where automatic recognition of on-line cursive handwriting is achieved by the use of continuous speech recognition methods. In this context, on-line refers to the situation where the time sequence of samples comprising the script is known (as with pen computers). The recognition of the on-line handwriting is performed through the use of hidden Markov models and statistical grammars in a manner very similar to several modem speech recognizers. In fact, we show that, with essentially no modification, a speech recognition system can perform accurate on-line handwriting recognition with the input features being those of writing instead of speech. Hidden Markov models have intrinsic properties which make them very attractive for handwriting recognition. For training, all that is necessary is a data stream and its transcription (the text matching the handwriting). The training process automatically aligns the com- ponents of the transcription to the data. Thus, no special effort is needed to label training data. Segmentation, in the traditional sense, is avoided altogether. Recognition is performed on another data stream. Again, no explicit segmentation is necessary. The segmen- teui~rently with the MIT Media Lab. tation of words into characters or even sentences into words occurs naturally by incorporating the use of a lexicon and a language model into the recognition process. The result is a …

Content truncated to 500 words