Joint Unsupervised Coreference Resolution with Markov Logic

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 650–659, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Joint Unsupervised Coreference Resolution with Markov Logic Hoifung Poon Pedro Domingos Department of Computer Science and Engineering University of Washington Seattle, WA 98195-2350, U.S.A. {hoifung,pedrod}@cs.washington.edu Abstract Machine learning approaches to coreference resolution are typically supervised, and re- quire expensive labeled data. Some unsuper- vised approaches have been proposed (e.g., Haghighi and Klein (2007)), but they are less accurate. In this paper, we present the first un- supervised approach that is competitive with supervised ones. This is made possible by performing joint inference across mentions, in contrast to the pairwise classification typ- ically used in supervised methods, and by us- ing Markov logic as a representation language, which enables us to easily express relations like apposition and predicate nominals. On MUC and ACE datasets, our model outper- forms Haghigi and Klein’s one using only a fraction of the training data, and often matches or exceeds the accuracy of state-of-the-art su- pervised models. 1 Introduction The goal of coreference resolution is to identify mentions (typically noun phrases) that refer to the same entities. This is a key subtask in many NLP applications, including information extraction, ques- tion answering, machine translation, and others. Su- pervised learning approaches treat the problem as one of classification: for each pair of mentions, predict whether they corefer or not (e.g., McCal- lum & Wellner (2005)). While successful, these approaches require labeled training data, consisting of mention pairs and the correct decisions for them. This limits their applicability. Unsupervised approaches are attractive due to the availability of large quantities of unlabeled text. However, unsupervised coreference resolution is much more difficult. Haghighi and Klein’s (2007) model, the most sophisticated to date, still lags su- pervised ones by a substantial margin. Extending it appears difficult, due to the limitations of its Dirich- let process-based representation. The lack of label information in unsupervised coreference resolution can potentially be overcome by performing joint inference, which leverages the “easy” decisions to help make related “hard” ones. Relations that have been exploited in supervised coreference resolution include transitivity (McCal- lum & Wellner, 2005) and anaphoricity (Denis & Baldridge, 2007). However, there is little work to date on joint inference for unsupervised resolution. We address this problem using Markov logic, a powerful and flexible language that combines probabilistic graphical models and first-order logic (Richardson & Domingos, 2006). Markov logic allows us to easily build models involving rela- tions among mentions, like apposition and predi- cate nominals. By extending the state-of-the-art al- gorithms for inference and learning, we developed the first general-purpose unsupervised learning al- gorithm for Markov logic, and applied it to unsuper- vised coreference resolution. We test our approach on standard MUC and ACE datasets. Our basic model, trained on a minimum of data, suffices to outperform Haghighi and Klein’s (2007) one. Our full model, using apposition and other relations for joint inference, is often as accu- rate as the best supervised …

Content truncated to 500 words