Enhancing Sequence-to-Sequence Neural Lemmatization with External Resources

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3112–3122 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3112 Enhancing Sequence-to-Sequence Neural Lemmatization with External Resources Kirill Milintsevich Institute of Computer Science University of Tartu Tartu, Estonia kirill.milintsevich@ut.ee Kairit Sirts Institute of Computer Science University of Tartu Tartu, Estonia kairit.sirts@ut.ee Abstract We propose a novel hybrid approach to lemma- tization1 that enhances the seq2seq neural model with additional lemmas extracted from an external lexicon or a rule-based system. Dur- ing training, the enhanced lemmatizer learns both to generate lemmas via a sequential de- coder and copy the lemma characters from the external candidates supplied during run-time. Our lemmatizer enhanced with candidates ex- tracted from the Apertium morphological ana- lyzer achieves statistically significant improve- ments compared to baseline models not utiliz- ing additional lemma information, achieves an average accuracy of 97.25% on a set of 23 UD languages, which is 0.55% higher than obtained with the Stanford Stanza model on the same set of languages. We also com- pare with other methods of integrating exter- nal data into lemmatization and show that our enhanced system performs considerably better than a simple lexicon extension method based on the Stanza system, and it achieves comple- mentary improvements w.r.t. the data augmen- tation method. 1 Introduction State-of-the-art lemmatization systems are based on attentional sequence-to-sequence neural archi- tectures operating on characters that transform the surface word form into its lemma (Kanerva et al., 2018; Qi et al., 2018). Like any other supervised learning model, these systems are dependent on the amount and quality of the existing training data. Attempts to develop even more accurate lemmati- zation systems can focus on improving the model’s architecture or obtaining additional data. While annotating additional data is an ongoing process for many smaller languages in the Universal Depen- dencies (UD) collection, there are also other data 1https://github.com/501Good/ lexicon-enhanced-lemmatization sources available that can be useful for improving lemmatization systems. In particular, we refer to ex- isting rule-based morphological analyzers, lexicons, and other such resources. Three potential sources for extracting additional lemma candidates are Apertium, Unimorph, and UD Lexicons initiatives. Apertium2 is an open- source rule-based machine translation platform (Forcada et al., 2011). It also includes rule-based morphological analyzers based on finite-state trans- ducers that cover 80 languages. Unimorph3 is a project aimed at collecting annotated morphological inflection data, including lemmas, from Wiktionary (Kirov et al., 2016), a free open dictionary for many languages. Currently, the Unimorph project covers 110 languages. UD Lexicons4 is a collection of 53 morphological lexicons in CoNLL-UL format covering 38 languages. UD Lexicons mostly use Apertium and Giellatekno systems to generate the annotations (Sagot, 2018). Several previous works have proposed methods to improve lemmatization systems by augmenting the training data with additional instances (Bergma- nis and Goldwater, 2019; Kanerva et al., 2020). In this paper, we propose another approach that both modifies the model architecture and leverages addi- tional data. Unlike previous work where the model gains …

Content truncated to 500 words