Annotating Zero Anaphora for Question Answering

Annotating Zero Anaphora for Question Answering Yoshihiko Asao, Ryu Iida, Kentaro Torisawa National Institute of Information and Communications Technology Kyoto 619-0289, Japan {asao, ryu.iida, torisawa}@nict.go.jp Abstract We constructed a large annotated dataset of zero pronouns that correspond to adjuncts marked by -de (translated to English as in, at, by or with) in Japanese. Adjunct zero anaphora resolution plays an important role in extracting information such as location and means from a text. To our knowledge, however, there have been no large-scale dataset covering them. In this paper, focusing on the application of zero anaphora resolution to question answering (QA), we proposed two annotation schemes. The first scheme was designed to efficiently collect zero anaphora instances that are useful in QA. Instead of directly annotating zero anaphora, annotators evaluated QA instances whose correctness hinges on zero anaphora resolution. Over 20,000 instances of zero anaphora were collected with this scheme. We trained a multi-column convolutional neural network with the annotated data, achieving an average precision of 0.519 in predicting the correctness of QA instances of the same type. In the second scheme, zero anaphora is annotated in a more direct manner. A model trained with the results of the second annotation scheme performed better than the first scheme in identifying zero anaphora for sentences randomly sampled from a corpus, suggesting a tradeoff between application-specific and general-purpose annotation schemes. Keywords: zero anaphora, question answering, convolutional neural networks 1. Introduction Zero anaphora refers to anaphora in which the anaphor has a phonetically null form. Zero anaphora resolution is an important sub-problem of many NLP tasks such as question answering (QA). To successfully apply machine learning algorithms to zero anaphora, it is crucial to construct a large and consistent annotated dataset. In this paper, we focus on zero pronouns that correspond to non-obligatory adjuncts marked by postposition -de in Japanese, which mark the location, means, reason or man- ner of an event and can be translated to English prepositions such as in, at, by or with. Although the identification of ad- junct zero pronouns plays an important role in extracting such information as location and means from a text, to our knowledge, there have been no attempts to create a large- scale dataset covering them. While it is possible to exhaustively annotate zero anaphora in a corpus to achieve our goal, such an approach might be inefficient when we are interested only in instances that are useful in a specific application. Given this, we pro- pose two different annotation schemes in this paper. In the first scheme, annotators annotate QA instances that poten- tially involve zero anaphora. In the second scheme, anno- tators directly annotate noun-predicate pairs with regard to whether they are in a zero anaphora relationship. We eval- uated the performance of these two schemes using an exist- ing neural network-based zero anaphora resolution method (Iida et al., 2016). Our experimental results show that the first scheme achieved better performance when it is used to train a module for a QA system, suggesting …

Content truncated to 500 words