The Covert Helps Parse the Overt

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 343–353, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics The Covert Helps Parse the Overt Xun Zhang, Weiwei Sun and Xiaojun Wan Institute of Computer Science and Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {zhangxunah,ws,wanxiaojun}@pku.edu.cn Abstract This paper is concerned with whether deep syntactic information can help sur- face parsing, with a particular focus on empty categories. We design new al- gorithms to produce dependency trees in which empty elements are allowed, and evaluate the impact of information about empty category on parsing overt elements. Such information is helpful to reduce the approximation error in a structured pars- ing model, but increases the search space for inference and accordingly the estima- tion error. To deal with structure-based overfitting, we propose to integrate disam- biguation models with and without empty elements, and perform structure regular- ization via joint decoding. Experiments on English and Chinese TreeBanks with dif- ferent parsing models indicate that incor- porating empty elements consistently im- proves surface parsing. 1 Introduction In the last two decades, there was an increas- ing interest in producing rich syntactic annota- tions that are not limited to surface analysis. See, among others, (Callmeier, 2000; Kaplan et al., 2004; Clark and Curran, 2007; Miyao and Tsu- jii, 2008; Zhang et al., 2016). Such analysis, e.g. deep dependency structures (King et al., 2003), is usually coupled with grammars un- der deep formalisms, e.g. Combinatory Cate- gorial Grammar (CCG; Steedman, 2000), Head- driven Phrase-Structure Grammar (HPSG; Pollard and Sag, 1994) and Lexical-Functional Grammar (LFG; Bresnan and Kaplan, 1982). Although deep grammar formalisms allow information beyond local construction to be constructed, it is still not clear whether such additional information is help- ful for surface syntactic analysis. This is partly because analysis grounded on different grammar formalisms, e.g. HPSG and CFG, are not directly comparable. In the Government and Binding (GB; Chom- sky, 1981) theory, empty category is a key con- cept bridging S-Structure and D-Structure, due to its possible contribution to trace movements. Fol- lowing the linguistic insights underlying GB, a tra- ditional dependency analysis can be augmented with empty elements, viz. covert elements (Xue and Yang, 2013). See Figure 1 for an example. The new representation provides a considerable amount of deep syntactic information, while keep- ing intact all dependencies of overt words. Inte- grating both overt and covert elements in one uni- fied representation provides an effective yet light- weight way to achieve deeper language under- standing beyond surface syntax1. Even more im- portant, this modest way to modify tree analysis makes possible fair evaluation of the influence of deep syntactic elements on surface parsing. We study graph-based parsing models for this new representation with a particular focus on the impact of information about the covert on parsing the overt. The major advantage of the graph-based approach to dependency parsing is that its con- strained factorization enables the design of poly- …

Content truncated to 500 words