Augmenting a BiLSTM Tagger with a Morphological Lexicon and a Lexical Category Identification Step

Proceedings of Recent Advances in Natural Language Processing, pages 1161–1168, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_133 1161 Augmenting a BiLSTM Tagger with a Morphological Lexicon and a Lexical Category Identification Step Steinþór Steingrímsson1,2 Örvar Kárason1 Hrafn Loftsson1 1Department of Computer Science, Reykjavik University, Iceland 2The Árni Magnússon Institute for Icelandic Studies, Reykjavik, Iceland {steinthor18,orvark13,hrafn}@ru.is Abstract Previous work on using BiLSTM mod- els for PoS tagging has primarily focused on small tagsets. We evaluate BiLSTM models for tagging Icelandic, a morpho- logically rich language, using a relatively large tagset. Our baseline BiLSTM model achieves higher accuracy than any previ- ously published tagger not taking advan- tage of a morphological lexicon. When we extend the model by incorporating such data, we outperform previous state-of-the- art results by a significant margin. We also report on work in progress that attempts to address the problem of data sparsity in- herent in morphologically detailed, fine- grained tagsets. We experiment with train- ing a separate model on only the lexical category and using the coarse-grained out- put tag as an input for the main model. This method further increases the accuracy and reduces the tagging errors by 21.3% compared to previous state-of-the-art re- sults. Finally, we train and test our tagger on a new gold standard for Icelandic. 1 Introduction Bidirectional long short-term memory (BiLSTM) models have in recent years been shown to be ef- fective for various sequential labelling tasks, in- cluding Part-of-Speech (PoS) tagging (Ling et al., 2015; Plank et al., 2016). BiLSTMs are an extension of general LSTMs (Hochreiter and Schmidhuber, 1997) that perform better on sequences where the complete input se- quence is available. Two LSTMs are trained on the input sequence, one on its natural reading order and the other on its reverse (Graves and Schmidhuber, 2005). In addition to word em- beddings (WE), character embeddings were first used for tagging with BiLSTMs by Dos Santos and Zadrozny (2014). This entails not only ex- amining the sequence of words in a sentence dur- ing training but also the sequences of characters within those words. In this paper we use BiLSTM models, with both word and character embeddings, to train a PoS tag- ger for a morphologically rich language, Icelandic, with a fine-grained tagset of 565 morphosyntactic tags. Only a small portion of previous work using neural networks for PoS tagging has focused on languages with rich morphology and large tagsets, e.g. Sagot and Martínez Alonso (2017). Various taggers have been developed for Ice- landic: data-driven taggers (Helgadóttir, 2005), a rule-based tagger (IceTagger) (Loftsson, 2008), and a hybrid tagger (Loftsson et al., 2009). Prior to the work presented here, an averaged perceptron tagger, IceStagger (Loftsson and Östling, 2013), was the current state-of-the-art tagger, achieving an accuracy of 93.84% by employing a morpho- logical lexicon and external word embeddings. This paper presents the first deep neural net- work tagger for Icelandic. We evaluate three mod- els. First, we confirm the effectiveness of a Bi- LSTM model for PoS tagging using a fine-grained tagset. …

Content truncated to 500 words