Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks

Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks Yuanhe Tian♥, Yan Song♠♥†, Fei Xia♥ ♥University of Washington ♠The Chinese University of Hong Kong (Shenzhen) ♥Shenzhen Research Institute of Big Data ♥{yhtian, fxia}@uw.edu ♠songyan@cuhk.edu.cn Abstract Supertagging is conventionally regarded as an important task for combinatory categorial grammar (CCG) parsing, where effective mod- eling of contextual information is highly im- portant to this task. However, existing studies have made limited efforts to leverage contex- tual features except for applying powerful en- coders (e.g., bi-LSTM). In this paper, we pro- pose attentive graph convolutional networks to enhance neural CCG supertagging through a novel solution of leveraging contextual infor- mation. Specifically, we build the graph from chunks (n-grams) extracted from a lexicon and apply attention over the graph, so that differ- ent word pairs from the contexts within and across chunks are weighted in the model and facilitate the supertagging accordingly. The ex- periments performed on the CCGbank demon- strate that our approach outperforms all previ- ous studies in terms of both supertagging and parsing. Further analyses illustrate the effec- tiveness of each component in our approach to discriminatively learn from word pairs to en- hance CCG supertagging.1 1 Introduction Combinatory categorial grammar (CCG) is a lexi- calized grammatical formalism, where the lexical categories (also known as supertags) of the words in a sentence provide informative syntactic and semantic knowledge for text understanding. There- fore, CCG parse often provides useful information for many downstream natural language processing (NLP) tasks such as logical reasoning (Yoshikawa et al., 2018) and semantic parsing (Beschke, 2019). To perform CCG parsing in different languages, most studies conducted a supertagging-parsing pi- pline (Clark and Curran, 2007; Kummerfeld et al., †Corresponding author. 1Our code and models for CCG supertagging are released at https://github.com/cuhksz-nlp/NeST-CCG. 2010; Song et al., 2012; Lewis and Steedman, 2014b; Huang and Song, 2015; Xu et al., 2015; Lewis et al., 2016; Vaswani et al., 2016; Yoshikawa et al., 2017), in which their main focus is the first step, and they generated the CCG parse trees di- rectly from supertags with a few rules afterwards. Building an accurate supertagger in a sequence labeling process requires a good modeling of con- textual information. Recent neural approaches to supertagging mainly focused on leveraging power- ful encoders with recurrent models (Lewis et al., 2016; Vaswani et al., 2016; Clark et al., 2018), with limited attention paid to modeling extra contextual features such as word pairs with strong relations. Graph convolutional networks (GCN) is demon- strated to be an effective approach to model such contextual information between words in many NLP tasks (Marcheggiani and Titov, 2017; Huang and Carley, 2019; De Cao et al., 2019; Huang et al., 2019); thus we want to determine whether this ap- proach can also help CCG supertagging. However, we cannot directly apply conventional GCN models to CCG supertagging because in most of the previous studies the GCN models are built over the edges in the dependency tree of an input sentence. As high-quality dependency parsers are …

Content truncated to 500 words