A Corpus to Learn Refer-to-as Relations for Nominals

A Corpus to Learn Refer-to-as Relations for Nominals Wasi Uddin Ahmad and Kai-Wei Chang Department of Computer Science, University of California, Los Angeles, California, USA {wasiahmad, kwchang}@cs.ucla.edu Abstract Continuous representations for words or phrases, trained on large unlabeled corpora are proved very useful for many natural language processing tasks. While these vector representations capture many fine-grained syntactic and semantic regularities among words or phrases, it often lacks coreferential information which is useful for many downstream tasks like information extraction, text summarization etc. In this paper, we argue that good word and phrase embeddings should contain information for identifying refer-to-as relationship and construct a corpus from Wikipedia to generate coreferential neural embeddings for nominals. The term nominal refers to a word or a group of words that functions like a noun phrase. In addition, we use coreference resolution as a proxy to evaluate the learned neural embeddings for noun phrases. To simplify the evaluation procedure, we design a coreferential phrase prediction task where the learned nominal embeddings are used to predict which candidate nominals can be referred to a target nominal. We further describe how to construct an evaluation dataset for such task from well known OntoNotes corpus and demonstrate encouraging baseline results. Keywords: Nominals, Coreference, Refer-to-as relation 1. Introduction Understanding relations between words and phrases is a long-standing problem in natural language processing. Var- ious resources are collected and utilized in order to under- stand different types of relations between words, including synonymy, antonymy (Turney, 2008) and hierarchical re- lationships such as hyponymy and hypernymy (Fu et al., 2014). Coreference (a.k.a refer-to-as) relation is another important type of relations between words and phrases and has a wide range of potential applications. Previous work (Feng et al., 2015) found both textual and visual informa- tion helpful to learn refer-to-as relations between words. However, annotated data for coreferent phrases are missing. In this work, we develop a dataset from Wikipedia which can aid in learning and evaluating refer-to-as relations be- tween a group of words which act as a noun phrase. Fur- thermore, the developed dataset can be leveraged to con- struct semantic space representations for the coreferential nominals. Learning semantic representations for words from large un- labeled corpora (ex., Wikipedia) using co-occurrence statis- tics has a long history in natural language processing (Deer- wester et al., 1990; Lund and Burgess, 1996; Collobert and Weston, 2008). More recent works (Mikolov et al., 2013; Pennington et al., 2014) uses log-bilinear models to learn continuous representations of words on large corpora ef- ficiently. While these vector representations capture fine- grained syntactic and semantic regularities among words or phrases, it often lacks coreferential information. For exam- ple, “phd student” and “graduate fellow” can be co-referred to each other and this relationship should be recognized by semantic representations. Refer-to-as relation information can benefit many natural language processing applications such as question answer- ing (Morton, 1999), information extraction (Humphreys et al., 1997; Zelenko et al., 2004) etc. So, in this paper, we focus on the task of …

Content truncated to 500 words