Predicting Inflectional Paradigms and Lemmata of Unknown Words for Semi-automatic Expansion of Morphological Lexicons

Proceedings of Recent Advances in Natural Language Processing, pages 379–387, Hissar, Bulgaria, Sep 7–9 2015. Predicting Inflectional Paradigms and Lemmata of Unknown Words for Semi-automatic Expansion of Morphological Lexicons Nikola Ljubeˇsi´c University of Zagreb nljubesi@ffzg.hr Miquel Espl`a-Gomis Universitat d’Alacant mespla@dlsi.ua.es Filip Klubiˇcka University of Zagreb fklubick@ffzg.hr Nives Mikeli´c Preradovi´c University of Zagreb nmikelic@ffzg.hr Abstract In this paper we describe a semi-automated approach to extend morphological lexicons by defining the prediction of the correct inflectional paradigm and the lemma for an unknown word as a supervised ranking task trained on an already existing lexicon. While most ranking approaches rely only on heuristics based on a single informa- tion source, our predictor uses hundreds of features calculated on the candidate stem, corpus evidence and statistics calculated from the existing lexicon. On the exam- ple of the Croatian language we show that our approach significantly outperforms a heuristic-based baseline, yielding correct candidates in 77% of cases on the first po- sition and in 95% of cases on the first five positions. 1 Introduction Morphological lexicons are a vital resource in au- tomatic processing of morphologically rich lan- guages and their construction is a tedious and costly process. The most reasonable approach to organizing inflectional morphological lexicons of morpho- logically rich languages is to define inflectional paradigms and assign them to corresponding lex- emes. In this way, every entry in the morphological lexicon becomes a pair (l, p) of a lemma l and a paradigm p which allow to derive all the possible surface forms of a given word. In this paper we frame the problem of assisting a process of extending an existing morphological lexicon as a supervised ranking problem. Namely, for each unknown word of a language we generate all possible pairs (l, p) and rank them with the goal of positioning existing pairs as high as possible. The result of the raking process is presented to a linguist through a graphical interface for labelling the correct pairs (l, p).drastically lower than would be the case if the lexicon was built manually. 2 Related Work A significant amount of research has focused on the problem of enhancing the process of produc- ing morphological resources. The most widely used approach is ranking pairs (l, p) of lemmas and paradigms by various scoring functions which rely on corpus evidence, the most popular being the coverage of all inflected forms derived from a pair (l, p) in a given monolingual corpus (Cl´ement et al., 2004; Tadi´c and Oliver, 2004; Sagot, 2005; ˇSnajder et al., 2008; Espl`a-Gomis et al., 2011). While our approach follows the same ranking paradigm, we argue that a significant amount of additional in- formation can be gained from corpora and other information sources, supervised machine learning being the obvious solution for combining those. The approach by Lind´en (2009) does not rely on corpus evidence only, but uses the existing lexicon as well, showing that by combining corpus and lexicon evidence significant gains can be achieved. The first approach to exploit machine learning …

Content truncated to 500 words