Better, Faster, Stronger Sequence Tagging Constituent Parsers

Proceedings of NAACL-HLT 2019, pages 3372–3383 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3372 Better, Faster, Stronger Sequence Tagging Constituent Parsers David Vilares Universidade da Coru˜na, CITIC Departamento de Computaci´on A Coru˜na, Spain david.vilares@udc.es Mostafa Abdou, Anders Søgaard University of Copenhagen Department of Computer Science Copenhagen, Denmark {abdou,soegaard}@di.ku.dk Abstract Sequence tagging models for constituent pars- ing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propa- gation arising from greedy decoding. To ef- fectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gra- dient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chi- nese Penn Treebanks, and reduce their pars- ing time even further. On the SPMRL datasets, we observe even greater improvements across the board, including a new state of the art on Basque, Hebrew, Polish and Swedish.1 1 Introduction Constituent parsing is a core task in natural lan- guage processing (NLP), with a wide set of ap- plications. Most competitive parsers are slow, however, to the extent that it is prohibitive of downstream applications in large-scale environ- ments (Kummerfeld et al., 2012). Previous ef- forts to obtain speed-ups have focused on creating more efficient versions of traditional shift-reduce (Sagae and Lavie, 2006; Zhang and Clark, 2009) or chart-based parsers (Collins, 1997; Charniak, 2000). Zhu et al. (2013), for example, presented 1After this paper was submitted, Kitaev and Klein (2018b) have improved our results using their previous self-attentive constituent parser (Kitaev and Klein, 2018a) and BERT repre- sentations (Devlin et al., 2018) as input to their system. We will acknowledge these results in the Experiments section. a fast shift-reduce parser with transitions learned by a SVM classifier. Similarly, Hall et al. (2014) introduced a fast GPU implementation for Petrov and Klein (2007), and Shen et al. (2018) signifi- cantly improved the speed of the Stern et al. (2017) greedy top-down algorithm, by learning to predict a list of syntactic distances that determine the or- der in which the sentence should be split. In an alternative line of work, some authors have proposed new parsing paradigms that aim to both reduce the complexity of existing parsers and improve their speed. Vinyals et al. (2015) pro- posed a machine translation-inspired sequence-to- sequence approach to constituent parsing, where the input is the raw sentence, and the ‘transla- tion’ is a parenthesized version of its tree. G´omez- Rodr´ıguez and Vilares (2018) reduced constituent parsing to sequence tagging, where only n tagging actions need to be made, and obtained one of the fastest parsers to date. However, the …

Content truncated to 500 words