An LDA-based Topic Selection Approach to Language Model Adaptation for Handwritten Text Recognition

Proceedings of Recent Advances in Natural Language Processing, pages 646–653, Hissar, Bulgaria, Sep 7–9 2015. An LDA-based Topic Selection Approach to Language Model Adaptation for Handwritten Text Recognition Jafar Tanha, Jesse de Does and Katrien Depuydt Institute for Dutch Lexicology (INL) jafar.tanha.pnu@gmail.com, {jesse.dedoes, katrien.depuydt}@inl.nl Abstract Typically, only a very limited amount of in-domain data is available for training the language model component of an Hand- written Text Recognition (HTR) system for historical data. One has to rely on a combination of in-domain and out-of- domain data to develop language models. Accordingly, domain adaptation is a cen- tral issue in language modeling for HTR. We pursue a topic modeling approach to handle this issue, and propose two algo- rithms based on this approach. The first algorithm relies on posterior inference for topic modeling to construct a language model adapted to the development set, and the second algorithm proceeds by iterative selection, using a new ranking criterion, of topic-dependent language models. Our experimental results show that both ap- proaches clearly outperform a strong base- line method. 1 Introduction Huge amounts of handwritten historical docu- ments are nowadays being published by on-line digital libraries as document images. The content of these documents is of great interest to histori- ans, linguists and literary scholars alike. How- ever, if the transcription of the documents is not available for information retrieval, we can hardly consider this content to be accessible for research. Full manual transcription is slow and costly, but the development of efficient and cost-effective ap- proaches for the indexing, search and full tran- scription of historical handwritten document im- ages can benefit from modern Handwritten Text Recognition (HTR) technology (S´anchez et al., 2013). An indispensable component of state-of-the-art HTR is language modeling (Pl¨otz and Fink, 2009) (Espana-Boquera et al., 2011), which is neces- sary to guide the decoding process by ranking and constraining the possible recognition hypotheses. Language modeling has proven extremely suc- cessful in improving results of Automatic Speech Recognition (Chelba et al., 2012), which is a very similar task from the technical point of view. Highly effective language models in this field have been developed from huge language corpora. cf. for instance (Chelba et al., 2012). Language mod- els are usually constructed from large text cor- pora which – ideally – are in-domain, linguisti- cally close to the language of the document col- lection which is being processed. However, for HTR of historical documents, obtaining effective models is much less straightforward: models built from the strictly in-domain data are generally un- satisfactory because not enough data can be ob- tained to avoid overfitting, and in order to exploit the larger pool of out-domain data one has to sur- mount two difficulties: (1) indiscriminate use of out-of-domain data may not benefit, in fact even deteriorate system performance and (2) the use of the complete out-domain data for training may in- crease the complexity of the system, making the decoding process almost untractable (Axelrod et al., 2011; Tanha et al., 2014). The above-mentioned issues …

Content truncated to 500 words