Evaluating Automated and Manual Acquisition of Anaphora Resolution Strategies

Evaluating Automated and Manual Acquisition of Anaphora Resolution Strategies Chinatsu Aone and Scott William Bennett Systems Research and Applications Corporation (SRA) 2000 15th Street North Arlington, VA 22201 aonec~sra.corn, bennett~sra.com Abstract We describe one approach to build an au- tomatically trainable anaphora resolution system. In this approach, we use Japanese newspaper articles tagged with discourse information as training examples for a ma- chine learning algorithm which employs the C4.5 decision tree algorithm by Quin- lan (Quinlan, 1993). Then, we evaluate and compare the results of several variants of the machine learning-based approach with those of our existing anaphora resolu- tion system which uses manually-designed knowledge sources. Finally, we compare our algorithms with existing theories of anaphora, in particular, Japanese zero pro- nouns. 1 Introduction Anaphora resolution is an important but still diffi- cult problem for various large-scale natural language processing (NLP) applications, such as information extraction and machine tr~slation. Thus far, no theories of anaphora have been tested on an empir- ical basis, and therefore there is no answer to the "best" anaphora resolution algorithm. I Moreover, an anaphora resolution system within an NLP sys- tem for real applications must handle: • degraded or missing input (no NLP system has complete lexicons, grammars, or semantic knowledge and outputs perfect results), and • different anaphoric phenomena in different do- mains, languages, and applications. Thus, even if there exists a perfect theory, it might not work well with noisy input, or it would not cover all the anaphoric phenomena. 1Walker (Walker, 1989) compares Brennan, Friedman a~ad Pollard's centering approach (Brennan et al., 1987) with Hobbs' algorithm (Hohbs, 1976) on a theoretical basis. These requirements have motivated us to de- velop robust, extensible, and trainable anaphora resolution systems. Previously (Aone and Mc- Kee, 1993), we reported our data-driven multilin- gual anaphora resolution system, which is robust, exteusible, and manually trainable. It uses dis- course knowledge sources (KS's) which are manu- ally selected and ordered. (Henceforth, we call the system the Manually-Designed Resolver, or MDR.) We wanted to develop, however, truly automatically trainable systems, hoping to improve resolution per- formance and reduce the overhead of manually con- structing and arranging such discourse data. In this paper, we first describe one approach we are taking to build an automatically trainable anaphora resolution system. In this approach, we tag corpora with discourse information, and use them as training examples for a machine learning algorithm. (Henceforth, we call the system the Ma- chine Learning-based Resolver, or MLR.) Specifi- cally, we have tagged Japanese newspaper articles about joint ventures and used the C4.5 decision tree algorithm by Quinlan (Quinlan, 1993). Then, we evaluate and compare the results of the MLR with those produced by the MDR. Finally, we compare our algorithms with existing theories of anaphora, in particular, Japanese zero pronouns. 2 Applying a Machine Learning Technique to Anaphora Resolution In this section, we first discuss corpora which we created for training and testing. Then, we describe the learning approach chosen, and discuss training features and training methods …

Content truncated to 500 words