Explicitly modeling case improves neural dependency parsing

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 356–358 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 356 Explicitly modeling case improves neural dependency parsing Clara Vania and Adam Lopez Institute for Language, Cognition and Computation School of Informatics University of Edinburgh c.vania@ed.ac.uk, alopez@inf.ed.ac.uk Abstract Neural dependency parsing models that com- pose word representations from characters can presumably exploit morphosyntax when mak- ing attachment decisions. How much do they know about morphology? We investigate how well they handle morphological case, which is important for parsing. Our experiments on Czech, German and Russian suggest that adding explicit morphological case—either or- acle or predicted—improves neural depen- dency parsing, indicating that the learned rep- resentations in these models do not fully en- code the morphological knowledge that they need, and can still benefit from targeted forms of explicit linguistic modeling. 1 Introduction Parsing morphologically rich languages (MRLs) is difficult due to the complex relationship of syn- tax to morphology. But the success of neural net- works offer an appealing solution to this problem by computing word representation from charac- ters. Character-level models (Ling et al., 2015; Kim et al., 2016) learn relationship between sim- ilar word forms and have shown to be effective for parsing MRLs (Ballesteros et al., 2015; Dozat et al., 2017; Shi et al., 2017; Bj¨orkelund et al., 2017). Does that mean that we can do away with explicit modeling of morphology altogether? Con- sider two challenges in parsing MRLs raised by Tsarfaty et al. (2010, 2013): • Can we represent words abstractly so as to reflect shared morphological aspects between them? • Which types of morphological information should we include in the parsing model? It is tempting to hypothesize that character-level models effectively solve the first problem. For the second, Tsarfaty et al. (2010) and Seeker and Kuhn (2013) reported that morphological case is beneficial across morphologically rich languages with extensive case systems, where case syn- cretism is pervasive and often hurts parsing perfor- mance. But these studies focus on vintage parsers; do neural parsers with character-level representa- tions also solve this second problem? We attempt to answer this question by asking whether an explicit model of morphological case helps dependency parsing, and our results show that it does. Furthermore, a pipeline model in which we feed predicted case to the parser out- performs multi-task learning in which case predic- tion is an auxiliary task. These results suggest that neural dependency parsers do not adequately in- fer this crucial linguistic feature directly from the input text. 2 Dependency Parsing Model We use a neural graph-based dependency parser similar to that of Kiperwasser and Goldberg (2016) and Zhang et al. (2017) for all our exper- iments. We treat our parser as a black box and ex- periment only with the input representations of the parser. Let w = w1, . . . , w|w| be an input sentence of length |w| and let w0 denote an artificial ROOT token. For each …

Content truncated to 500 words