Not All Linearizations Are Equally Data-Hungry in Sequence Labeling Parsing

Proceedings of Recent Advances in Natural Language Processing, pages 978–988 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_111 978 Not All Linearizations Are Equally Data-Hungry in Sequence Labeling Parsing Alberto Mu˜noz-Ortiz1, Michalina Strzyz 1,2, David Vilares1 1Universidade da Coru˜na, CITIC, Spain 2Priberam Labs, Portugal alberto.munoz.ortiz@udc.es michalina.strzyz@priberam.pt, david.vilares@udc.es Abstract Different linearizations have been proposed to cast dependency parsing as sequence labeling and solve the task as: (i) a head selection prob- lem, (ii) finding a representation of the token arcs as bracket strings, or (iii) associating par- tial transition sequences of a transition-based parser to words. Yet, there is little understand- ing about how these linearizations behave in low-resource setups. Here, we first study their data efficiency, simulating data-restricted se- tups from a diverse set of rich-resource tree- banks. Second, we test whether such differ- ences manifest in truly low-resource setups. The results show that head selection encodings are more data-efficient and perform better in an ideal (gold) framework, but that such ad- vantage greatly vanishes in favour of bracket- ing formats when the running setup resembles a real-world low-resource configuration. 1 Introduction Dependency parsing (Mel’cuk et al., 1988; K¨ubler et al., 2009) has achieved clear improvements in recent years, to the point that graph-based (Martins et al., 2013; Dozat et al., 2017) and transition-based (Ma et al., 2018; Fern´andez-Gonz´alez and G´omez- Rodr´ıguez, 2019) parsers are already very accurate on certain setups, such as English news. In this line, Berzak et al. (2016) have pointed out that the performance on these setups is already on par with that expected from experienced human annotators. Thus, the efforts have started to focus on re- lated problems such as parsing different domains or multi-lingual scenarios (Sato et al., 2017; Song et al., 2019; Ammar et al., 2016), creating faster models (Volokh, 2013; Chen and Manning, 2014), designing low-resource and cross-lingual parsing techniques (Tiedemann et al., 2014; Zhang et al., 2019), or infusing syntactic knowledge into models (Strubell et al., 2018; Rotman and Reichart, 2019). This work will lie in the intersection between fast parsing and low-resource languages. Recent work has proposed encodings to cast parsing as sequence labeling (Spoustov´a and Spousta, 2010; Strzyz et al., 2019; G´omez-Rodr´ıguez et al., 2020; Li et al., 2018; Kiperwasser and Ballesteros, 2018). This approach computes a linearized tree of a sen- tence of length n in n tagging actions, providing a good speed/accuracy trade-off. Also, it offers a na¨ıve way to infuse syntactic information as an embedding or feature (Ma et al., 2019; Wang et al., 2019). Such encodings have been evaluated on En- glish and multi-lingual setups, but there is no study about their behaviour on low-resource setups, and what strengths and weaknesses they might exhibit. Contribution We study the behaviour of lin- earizations for dependency parsing as sequence labeling in low-resource setups. First, we explore their data efficiency, i.e. if they can exploit their full potential with less supervised data. To do so, we simulate different data-restricted setups from a diverse set of rich-resource treebanks. Second, we shed light about …

Content truncated to 500 words