Acronym recognition and processing in 22 languages

Proceedings of Recent Advances in Natural Language Processing, pages 237–244, Hissar, Bulgaria, 7-13 September 2013. Acronym Recognition and Processing in 22 Languages Maud Ehrmann Department of Computer Science Sapienza University of Rome ehrmann@di.uniroma1.it Ralf Steinberger European Commission Joint Research Centre IPSC-GlobeSec ralf.steinberger@jrc.ec.europa.eu Leonida Della Rocca European Commission Joint Research Centre IPSC-GlobeSec leonida.della- rocca@ext.jrc.ec.europa.eu Hristo Tannev European Commission Joint Research Centre IPSC-GlobeSec hristo.tannev @jrc.ec.europa.eu Abstract We are presenting work on recognising acro- nyms of the form Long-Form (Short-Form) such as “International Monetary Fund (IMF)” in millions of news articles in twenty-two lan- guages, as part of our more general effort to recognise entities and their variants in news text and to use them for the automatic analysis of the news, including the linking of related news across languages. We show how the ac- ronym recognition patterns, initially developed for medical terms, needed to be adapted to the more general news domain and we present evaluation results. We describe our effort to automatically merge the numerous long-form variants referring to the same short-form, while keeping non-related long-forms sepa- rate. Finally, we provide extensive statistics on the frequency and the distribution of short- form/long-form pairs across languages. 1 Introduction and Motivation An acronym is an abbreviation formed from the initial letters of the various word elements and read as a single word.1 Acronyms are formed to speed up and ease communication, mainly to create words for concepts frequently used or dif- 1 See http://dictionary.reference.com/help/faq/ language/t08.html to distinguish acronyms from relat- ed concepts such as initials and contractions. ficult to describe. Like entities, acronyms have a high reference value, in the sense that they most of the time act as reference anchors of textual content. However, they are not always explicitly defined, which can cause comprehension prob- lems, both for humans and machines. In addition, due to the large number of acronyms – we found over one million when analysing our news data set – the same short-form (SF) can have several conceptually different long-forms (LF) (see Ta- ble 1). Even for the same SF-LF pair, many LF variants may exist. In addition to simple wording differences, there can be grammatical inflection forms and cross-lingual variants. Acronyms are productive words, i.e. new ac- ronyms are created every day, requiring frequent updating of any acronym database. In the first month of applying the tool to our large through- put of multilingual news articles, we identified 66,000 acronyms (before merging variants, i.e. unique SF-LF pairs). After only five months of analysis, the monthly number of newly identified acronym pairs has halved and the number of newly found acronyms seems to be stabilising around this value. We are adding these new ac- ronyms to our multilingual dataset every day and we plan to publicly release the more frequently occurring ones in regular intervals as part of the multilingual name variant resource JRC-Names (Steinberger et al. 2011), which currently pre- dominantly contains person names. This dataset 237 can be used for named entity recognition and other natural language …

Content truncated to 500 words