Parsing as Tagging

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5225–5231 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5225 Parsing as Tagging Robert Vacareanu1, George C. G. Barbosa2, Marco A. Valenzuela-Esca´rcega2, Mihai Surdeanu2 1 Technical University of Cluj-Napoca, 2 University of Arizona 1Cluj-Napoca, Cluj, Romania, 2Tucson, AZ, USA robert.vacareanu@gmail.com {gcgbarbosa, marcov, msurdeanu}@email.arizona.edu Abstract We propose a simple yet accurate method for dependency parsing that treats parsing as tagging (PaT). That is, our approach addresses the parsing of dependency trees with a sequence model implemented with a bidirectional LSTM over BERT embeddings, where the “tag” to be predicted at each token position is the relative position of the corresponding head. For example, for the sentence John eats cake, the tag to be predicted for the token cake is -1 because its head (eats) occurs one token to the left. Despite its simplicity, our approach performs well. For example, our approach outperforms the state-of-the-art method of (Fern´andez-Gonz´alez and G´omez-Rodr´ıguez, 2019) on Universal Dependencies (UD) by 1.76% unlabeled attachment score (UAS) for English, 1.98% UAS for French, and 1.16% UAS for German. On average, on 15 UD languages, our method with minimal tuning performs comparably with this state-of-the-art approach, being only 0.16% UAS, and 0.82% LAS behind. Keywords: dependency parsing, sequence methods 1. Introduction There exists a trend in syntactic dependency parsing to- wards simpler and simpler approaches. For example, early dependency parsing algorithms had a runtime complex- ity of O(n3) (Eisner, 1996). These were followed by maximum spanning tree algorithms with a complexity of O(n2) (McDonald et al., 2005), and shift-reduce methods, which are linear in sentence length (O(n)) (Nivre, 2003). This quest for simplicity continued with the recent works of Ma et al. (2018) and Fern´andez-Gonz´alez and G´omez- Rodr´ıguez (2019), which replaced the shift-reduce algo- rithm with pointer networks, which are also linear but are arguably simpler than the shift-reduce algorithm. This evolutionary trend begs the question: can we simplify dependency parsing further, without losing considerable performance? This paper indicates that the answer is yes. We propose an extremely simple method for dependency parsing that treats parsing as tagging (PaT). That is, our approach addresses the parsing of dependency trees with a sequence model, where the “tag” to be predicted at each token position is the relative position of the corresponding head. For example, for the sentence John eats cake, the tag to be predicted for the token cake is -1 because its head (eats) occurs one token to the left.1 The contributions of this work are: (1) We propose a simple approach for dependency parsing that operates in three stages, which are all widely used in other natural language processing (NLP) approaches. First, we encode the input tokens using a combination of ran- domly initialized word embeddings, contextualized embed- dings (Devlin et al., 2018), character-level embeddings gen- erated using a convolutional neural network (CNN), and part-of-speech (POS) embeddings. Second, these represen- tations are fed into a BiLSTM. Finally, using the BiLSTM’s …

Content truncated to 500 words