Unlexicalized Transition-based Discontinuous Constituency Parsing

Unlexicalized Transition-based Discontinuous Constituency Parsing Maximin Coavoux1∗, Benoˆıt Crabb´e2,3, Shay B. Cohen1 1University of Edinburgh, ILCC 2Universit´e Paris Diderot, Universit´e Sorbonne Paris Cit´e, LLF 3Institut Universitaire de France (IUF) {mcoavoux,scohen}@inf.ed.ac.uk bcrabbe@linguist.univ-paris-diderot.fr Abstract Lexicalized parsing models are based on the assumptions that (i) constituents are organized around a lexical head and (ii) bilexical statistics are crucial to solve ambiguities. In this paper, we introduce an unlexicalized transition-based parser for discontinuous constituency struc- tures, based on a structure-label transition system and a bi-LSTM scoring system. We compare it with lexicalized parsing models in order to address the question of lexical- ization in the context of discontinuous con- stituency parsing. Our experiments show that unlexicalized models systematically achieve higher results than lexicalized models, and provide additional empirical evidence that lex- icalization is not necessary to achieve strong parsing results. Our best unlexicalized model sets a new state of the art on English and German discontinuous constituency treebanks. We further provide a per-phenomenon analysis of its errors on discontinuous constituents. 1 Introduction This paper introduces an unlexicalized parsing model and addresses the question of lexicaliza- tion, as a parser design choice, in the context of transition-based discontinuous constituency pars- ing. Discontinuous constituency trees are con- stituency trees where crossing arcs are allowed in order to represent long-distance dependencies, and in general phenomena related to word order variations (e.g., the left dislocation in Figure 1). Lexicalized parsing models (Collins, 1997; Charniak, 1997) are based on the assumptions that (i) constituents are organized around a lexical ∗Work partly done at Universit´e Paris Diderot. head and (ii) bilexical statistics are crucial to solve ambiguities. In a lexicalized Probabilistic Context- Free Grammar (PCFG), grammar rules involve nonterminals annotated with a terminal element that represents their lexical head, for example: VP[saw] −→ VP[saw] PP[telescope]. The probability of such a rule models the likeli- hood that telescope is a suitable modifier for saw. In contrast, unlexicalized parsing models re- nounce modeling bilexical statistics, based on the assumptions that they are too sparse to be estimated reliably. Indeed, Gildea (2001) observed that removing bilexical statistics from Collins’ (1997) model lead to at most a 0.5 drop in F-score. Furthermore, Bikel (2004) showed that bilexical statistics were in fact rarely used during decoding, and that when used, they were close to that of back- off distributions used for unknown word pairs. Instead, unlexicalized models may rely on grammar rule refinements to alleviate the strong independence assumptions of PCFGs (Klein and Manning, 2003; Matsuzaki et al., 2005; Petrov et al., 2006; Narayan and Cohen, 2016). They sometimes rely on structural information, such as the boundaries of constituents (Hall et al., 2014; Durrett and Klein, 2015; Cross and Huang, 2016b; Stern et al., 2017; Kitaev and Klein, 2018). Although initially coined for chart parsers, the notion of lexicalization naturally transfers to transition-based parsers. We take lexicalized to denote a model that (i) assigns a lexical head to each constituent and (ii) uses heads of consti- tuents as features to score parsing actions. Head assignment …

Content truncated to 500 words