On the Frailty of Universal POS Tags for Neural UD Parsers

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 69–96 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 69 On the Frailty of Universal POS Tags for Neural UD Parsers Mark Anderson Carlos Gómez-Rodríguez Universidade da Coruña, CITIC FASTPARSE Lab, LyS Research Group, Departamento de Ciencias de la Computación y Tecnologías de la Información Campus Elviña, s/n, 15071 A Coruña, Spain {m.anderson,carlos.gomez}@udc.es Abstract We present an analysis on the effect UPOS accuracy has on parsing performance. Re- sults suggest that leveraging UPOS tags as fea- tures for neural parsers requires a prohibitively high tagging accuracy and that the use of gold tags offers a non-linear increase in perfor- mance, suggesting some sort of exceptionality. We also investigate what aspects of predicted UPOS tags impact parsing accuracy the most, highlighting some potentially meaningful lin- guistic facets of the problem. 1 Introduction Part-of-speech (POS) tags and dependency parsing have formed a long-standing union in NLP. But equally long-standing has been the question of its efficacy. Prior to the prevalence of deep learning in NLP, they were shown to be useful for syntac- tic disambiguation in certain contexts (Voutilainen, 1998; Dalrymple, 2006; Alfared and Béchet, 2012). However, for neural network implementations, es- pecially those which utilise character embeddings, POS tags have been shown to be much less useful (Ballesteros et al., 2015; de Lhoneux et al., 2017). Others have found that POS tags can still have a positive impact when using character representa- tions given that the accuracy of the predicted POS tags used is sufficiently high (Dozat et al., 2017). Smith et al. (2018) undertook a systematic study of the impact of features for Universal Dependency (UD) parsing and found that using universal POS (UPOS) tags does still offer a marginal improve- ment for their transition-based neural parser. The use of fine-grained POS tags still seems to garner noticeable improvements (Ammar et al., 2016). Latterly, POS tags have been commonly utilised implicitly for neural network parsers in multi- learning frameworks where they can be leveraged without the cost of error-propagation (Zhang and Weiss, 2016; Yang et al., 2017; Li et al., 2018; Nguyen and Verspoor, 2018; Zhang et al., 2020). Beyond multi-learning systems, Strzyz et al. (2019) introduced dependency parsing as sequence la- belling by encoding dependencies using relative positions of UPOS tags, thus explicitly requiring them at runtime. We follow the work of Smith et al. (2018) and evaluate the interplay of word embeddings, charac- ter embeddings, and POS tags as features for two modern parsers, one a graph-based parser, Biaffine, and the other a transition-based parser, UUParser (Dozat and Manning, 2017; Smith et al., 2018). Similar to Zhang et al. (2020), we focus on the contribution of POS tags but evaluate UPOS tags. Contribution We analyse the effect UPOS ac- curacy has on two dependency parser systems for a number of UD treebanks. Our results suggest that in order to leverage UPOS tags as explicit fea- tures for these neural parsers, a prohibitively high tagging accuracy …

Content truncated to 500 words