Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4092–4099 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4092 Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing Daniel Fern´andez-Gonz´alez and Carlos G´omez-Rodr´ıguez Universidade da Coru˜na, CITIC FASTPARSE Lab, LyS Group Depto. de Ciencias de la Computaci´on y Tecnolog´ıas de la Informaci´on Campus de Elvi˜na, s/n, 15071 A Coru˜na, Spain d.fgonzalez@udc.es, carlos.gomez@udc.es Abstract Sequence-to-sequence constituent parsing re- quires a linearization to represent trees as se- quences. Top-down tree linearizations, which can be based on brackets or shift-reduce ac- tions, have achieved the best accuracy to date. In this paper, we show that these results can be improved by using an in- order linearization instead. Based on this observation, we implement an enriched in- order shift-reduce linearization inspired by Vinyals et al. (2015)’s approach, achieving the best accuracy to date on the English PTB dataset among fully-supervised single-model sequence-to-sequence constituent parsers. Fi- nally, we apply deterministic attention mech- anisms to match the speed of state-of-the- art transition-based parsers, thus showing that sequence-to-sequence models can match them, not only in accuracy, but also in speed. 1 Introduction Sequence-to-sequence (seq2seq) neural architec- tures have proved useful in several NLP tasks, with remarkable success in some of them such as ma- chine translation, but they lag behind the state of the art in others. In constituent parsing, seq2seq models still need to improve to be competitive in ac- curacy and efficiency with their main competitors: transition-based constituent parsers (Dyer et al., 2016; Liu and Zhang, 2017b; Fern´andez-Gonz´alez and G´omez-Rodr´ıguez, 2019). Vinyals et al. (2015) laid the first stone in seq2seq constituent parsing, proposing a lineariza- tion of phrase-structure trees as bracketed se- quences following a top-down strategy, which can be predicted from the input sequence of words by any off-the-shelf seq2seq framework. While this approach is very simple, its accuracy and efficiency are significantly behind the state of the art in the fully-supervised single-model scenario. Most attempts to improve this approach fo- cused on modifying the neural network architec- ture, while keeping the top-down linearization strat- egy. As exceptions, Ma et al. (2017) and Liu and Zhang (2017a) proposed linearizations based on se- quences of transition-based parsing actions instead of brackets. Ma et al. (2017) tried a bottom-up lin- earization, but they obtained worse results than top- down approaches.1 Liu and Zhang (2017a) kept the top-down strategy, but using transitions of the top-down transition system of Dyer et al. (2016) instead of a bracketed linearization, achieving a higher performance. In transition-based constituent parsing, an in- order algorithm has recently proved superior to the bottom-up and top-down approaches (Liu and Zhang, 2017b), but we know of no applications of this approach in seq2seq parsing. Contributions In this paper, we advance the un- derstanding of linearizations for seq2seq parsing, and improve the state of the art, as follows: (1) we show that the superiority of a transition-based top-down linearization over a bracketing-based one observed by Liu and Zhang (2017a) does not …

Content truncated to 500 words