Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient?

Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient? Piotr ˙Zelasko AGH University of Science and Technology, Kraków, www.dsp.agh.edu.pl, pzelasko@agh.edu.pl Abstract In this paper, the problem of recovery of morphological information lost in abbreviated forms is addressed with a focus on highly inflected languages. Evidence is presented that the correct inflected form of an expanded abbreviation can in many cases be deduced solely from the morphosyntactic tags of the context. The prediction model is a deep bidirectional LSTM network with tag embedding. The training and evaluation data are gathered by finding the words which could have been abbreviated and using their corresponding morphosyntactic tags as the labels, while the tags of the context words are used as the input features for classification. The network is trained on over 10 million words from the Polish Sejm Corpus and achieves 74.2% prediction accuracy on a smaller, but more general National Corpus of Polish. The analysis of errors suggests that performance in this task may improve if some prior knowledge about the abbreviated word is incorporated into the model. Keywords: abbreviation expansion, morphosyntactic tags, strongly inflected langauges 1. Introduction In this paper, we address the problem of recovery of mor- phological information lost in abbreviated forms, focusing on highly inflected languages. Significance of the problem can be shown in the domain of text normalization, espe- cially in cases of expanding numerals and abbreviations to their full forms. Although in English the translation of to- ken "14" to "fourteen", or token "yr." to "year" appears triv- ial, it is not so in a highly inflected language. For example in Polish, the abbreviation "r." could be expanded into one of {rok, roku, rokowi, rokiem}, all of which mean "year" in English, but are inflected differently, based on the context in which they are used. Correct choice of the grammatical number and case is essential towards successful text nor- malization, which is an indispensable part of any text-to- speech (TTS) system (Sproat and Jaitly, 2016), and can also be useful in text preprocessing for language model (LM) training (Pakhomov, 2002). Under the assumption that an abbreviation is not ambigu- ous, the conversion of a token into its abbreviated form can be seen as a removal of morphological information from the token. Nonetheless, a person (e.g. a native speaker) reading the text is able to deduce the information missing from the abbrevation by using the context. Furthermore, we suspect that it is possible to abstract away from concrete words and infer the missing morphological information based only on morphological and syntactic properties of the context. Con- sequently, we formulate a hypothesis, that based purely on the morphosyntactic information present in other words in the sentence (i.e. the context), the correct morphosyntac- tic tag can be inferred for the abbreviated word in a highly inflected language. Given recent major achievements of deep neural networks (DNN) in Natural Language Processing (NLP) tasks (Sun- dermeyer et al., 2012) (Huang et al., 2015), we apply a bidirectional recurrent neural …

Content truncated to 500 words