Empty Category Detection With Joint Context-Label Embeddings

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 263–271, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Empty Category Detection With Joint Context-Label Embeddings Xun Wang, Katsuhito Sudoh and Masaaki Nagata NTT Communication Science Laboratories Kyoto 619-0237, Japan wang.xun,sudoh.katsuhito,nagata.masaaki@lab.ntt.co.jp Abstract This paper presents a novel technique for empty category (EC) detection using dis- tributed word representations. A joint model is learned from the labeled data to map both the distributed representations of the contexts of ECs and EC types to a low dimensional space. In the testing phase, the context of possible EC positions will be projected into the same space for empty category detection. Experiments on Chinese Treebank prove the effectiveness of the proposed method. We improve the precision by about 6 points on a subset of Chinese Treebank, which is a new state-of- the-art performance on CTB. 1 Introduction The empty category (EC) is an important con- cept in linguistic theories. It is used to de- scribe nominal words that do not have ex- plicit phonological forms (they are also called “covert nouns”). This kind of grammatical phe- nomenons is usually caused by the omission or dislocation of nouns or pronouns. Empty cat- egories are the “hidden” parts of text and are essential for syntactic parsing (Gabbard et al., 2006; Yang and Xue, 2010). As a basic prob- lem in NLP, the resolution of ECs also has a huge impact on lots of downstream tasks, such as co-reference resolution (Ponzetto and Strube, 2006; Kong and Ng, 2013), long distance de- pendency relation analysis (Marcus et al., 1993; Xue et al., 2005). Research also uncovers the important role of ECs in machine translation. Some recent work (Chung and Gildea, 2010; Xi- ang et al., 2013) demonstrates the improvements they manage to obtain through EC detection in Chinese-English translation. To resolve ECs, we need to decide 1) the po- sition and type of the EC and 2) the content of the EC (to which element the EC is linked to if plausible). Existing research mainly fo- cuses on the first problem which is referred to as EC detection (Cai et al., 2011; Yang and Xue, 2010), and so is this paper. As ECs are words or phrases inferable from their context, previous work mainly designs features mining the contexts of ECs and then trains classifica- tion models or parsers using these features (Xue and Yang, 2013; Johnson, 2002; Gabbard et al., 2006; Kong and Zhou, 2010). One problem with these human-developed features are that they are not fully capable of representing the seman- tics and syntax of contexts. Besides, the feature engineering is also time consuming and labor in- tensive. Recently neural network models have proven their superiority in capturing features using low dense vector compared with traditional manu- ally designed features in dozens of NLP tasks (Bengio et al., 2006; Collobert and Weston, 2008; Socher et al., 2010; Collobert et al., 2011; Li and Hovy, 2014; …

Content truncated to 500 words