Improving Machine Learning Approaches to Coreference Resolution

Improving Machine Learning Approaches to Coreference Resolution Vincent Ng and Claire Cardie Department of Computer Science Cornell University Ithaca, NY 14853-7501 � yung,cardie � @cs.cornell.edu Abstract We present a noun phrase coreference sys- tem that extends the work of Soon et al. (2001) and, to our knowledge, pro- duces the best results to date on the MUC- 6 and MUC-7 coreference resolution data sets — F-measures of 70.4 and 63.4, re- spectively. Improvements arise from two sources: extra-linguistic changes to the learning framework and a large-scale ex- pansion of the feature set to include more sophisticated linguistic knowledge. 1 Introduction Noun phrase coreference resolution refers to the problem of determining which noun phrases (NPs) refer to each real-world entity mentioned in a doc- ument. Machine learning approaches to this prob- lem have been reasonably successful, operating pri- marily by recasting the problem as a classification task (e.g. Aone and Bennett (1995), McCarthy and Lehnert (1995)). Specifically, a pair of NPs is clas- sified as co-referring or not based on constraints that are learned from an annotated corpus. A separate clustering mechanism then coordinates the possibly contradictory pairwise classifications and constructs a partition on the set of NPs. Soon et al. (2001), for example, apply an NP coreference system based on decision tree induction to two standard coref- erence resolution data sets (MUC-6, 1995; MUC- 7, 1998), achieving performance comparable to the best-performing knowledge-based coreference en- gines. Perhaps surprisingly, this was accomplished in a decidedly knowledge-lean manner — the learn- ing algorithm has access to just 12 surface-level fea- tures. This paper presents an NP coreference system that investigates two types of extensions to the Soon et al. corpus-based approach. First, we propose and evaluate three extra-linguistic modifications to the machine learning framework, which together pro- vide substantial and statistically significant gains in coreference resolution precision. Second, in an attempt to understand whether incorporating addi- tional knowledge can improve the performance of a corpus-based coreference resolution system, we expand the Soon et al. feature set from 12 features to an arguably deeper set of 53. We propose addi- tional lexical, semantic, and knowledge-based fea- tures; most notably, however, we propose 26 addi- tional grammatical features that include a variety of linguistic constraints and preferences. Although the use of similar knowledge sources has been explored in the context of both pronoun resolution (e.g. Lap- pin and Leass (1994)) and NP coreference resolution (e.g. Grishman (1995), Lin (1995)), most previous work treats linguistic constraints as broadly and un- conditionally applicable hard constraints. Because sources of linguistic information in a learning-based system are represented as features, we can, in con- trast, incorporate them selectively rather than as uni- versal hard constraints. Our results using an expanded feature set are mixed. First, we find that performance drops signifi- cantly when using the full feature set, even though the learning algorithms investigated have built-in feature selection mechanisms. We demonstrate em- Computational Linguistics (ACL), Philadelphia, July 2002, pp. 104-111. Proceedings of the 40th Annual Meeting of the …

Content truncated to 500 words