Modeling Composite Labels for Neural Morphological Tagging

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 368–379 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 368 Modeling Composite Labels for Neural Morphological Tagging Alexander Tkachenko Institute of Computer Science University of Tartu Tartu, Estonia aleksandr.tkatsenko@ut.ee Kairit Sirts Institute of Computer Science University of Tartu Tartu, Estonia kairit.sirts@ut.ee Abstract Neural morphological tagging has been re- garded as an extension to POS tagging task, treating each morphological tag as a mono- lithic label and ignoring its internal structure. We propose to view morphological tags as composite labels and explicitly model their internal structure in a neural sequence tag- ger. For this, we explore three different neural architectures and compare their performance with both CRF and simple neural multiclass baselines. We evaluate our models on 49 lan- guages and show that the neural architecture that models the morphological labels as se- quences of morphological category values per- forms significantly better than both baselines establishing state-of-the-art results in morpho- logical tagging for most languages.1 1 Introduction The common approach to morphological tagging combines the set of word’s morphological fea- tures into a single monolithic tag and then, sim- ilar to POS tagging, employs multiclass sequence classification models such as CRFs (Müller et al., 2013) or recurrent neural networks (Labeau et al., 2015; Heigold et al., 2017). This approach, how- ever, has a number of limitations. Firstly, it ig- nores the intrinsic compositional structure of the labels and treats two labels that differ only in the value of a single morphological category as completely independent; compare for instance la- bels [POS=NOUN,CASE=NOM,NUM=SG] and [POS=NOUN,CASE=NOM,NUM=PL] that only differ in the value of the NUM category. Secondly, it introduces a data sparsity issue as the less fre- quent labels can have only few occurrences in the 1The source code is available at https://github.com/AleksTk/seq-morph-tagger training data. Thirdly, it excludes the ability to pre- dict labels not present in the training set which can be an issue for languages such as Turkish where the number of morphological tags is theoretically unlimited (Yuret and Türe, 2006). To address these problems we propose to treat morphological tags as composite labels and explic- itly model their internal structure. We hypothesise that by doing that, we are able to alleviate the spar- sity problems, especially for languages with very large tagsets such as Turkish, Czech or Finnish, and at the same time also improve the accuracy over a baseline using monolithic labels. We explore three different neural architectures to model the compo- sitionality of morphological labels. In the first ar- chitecture, we model all morphological categories (including POS tag) as independent multiclass clas- sifiers conditioned on the same contextual word representation. The second architecture organises these multiclass classifiers into a hierarchy—the POS tag is predicted first and the values of morpho- logical categories are predicted conditioned on the value of the predicted POS. The third architecture models the label as a sequence of morphological category-value pairs. All our models share the same …

Content truncated to 500 words