Max-Margin Incremental CCG Parsing

Max-Margin Incremental CCG Parsing Miloˇs Stanojevi´c School of Informatics University of Edinburgh m.stanojevic@ed.ac.uk Mark Steedman School of Informatics University of Edinburgh steedman@inf.ed.ac.uk Abstract Incremental syntactic parsing has been an ac- tive research area both for cognitive scientists trying to model human sentence processing and for NLP researchers attempting to com- bine incremental parsing with language mod- elling for ASR and MT. Most effort has been directed at designing the right transition mech- anism, but less has been done to answer the question of what a probabilistic model for those transition parsers should look like. A very incremental transition mechanism of a recently proposed CCG parser when trained in straightforward locally normalised discrimina- tive fashion produces very bad results on En- glish CCGbank. We identify three biases as the causes of this problem: label bias, expo- sure bias and imbalanced probabilities bias. While known techniques for tackling these bi- ases improve results, they still do not make the parser state of the art. Instead, we tackle all of these three biases at the same time using an improved version of beam search optimisa- tion that minimises all beam search violations instead of minimising only the biggest viola- tion. The new incremental parser gives better results than all previously published incremen- tal CCG parsers, and outperforms even some widely used non-incremental CCG parsers. 1 Introduction It has been known for a long time that human sen- tence processing is highly incremental (Marslen- Wilson, 1973), with early formation of semantic representations. A parser that is able to form rep- resentation early must have some notion of partial structure such as “S missing an object NP”. Also, such parser needs to be able to combine partial structures into bigger partial structures. These two properties are at the core of Combinatory Catego- rial Grammar (CCG) (Ades and Steedman, 1982; Steedman, 2000). CCG represents partial con- stituents using complex categories. For example S/NP is the category of a transitive sentential pre- fix such as I like or I think I like requiring an object NP on its right. Such prefix categories are con- structed using combinatory rules such as function composition. In this way we can form (mostly) left-branching derivation trees that can be parsed incrementally even with simple transition mecha- nisms such as shift-reduce parsers. Still, left branching structures are not sufficient to solve all the problems of incremental sentence processing. Right adjuncts are particularly prob- lematic. They appear on the right of the head that they modify which means that they need to be predicted, but at the same time they are op- tional which makes it impossible to predict them with confidence. Stanojevi´c and Steedman (2019) tackle this issue by using incremental tree-rotation and revealing operations that allow adjuncts not to be predicted, but still be easy to attach to the head in case they appear. They show great improvement in the incrementality of this approach as measured by connectedness (the average stack size). However, Stanojevi´c and Steedman (2019) parser is not fully …

Content truncated to 500 words