A Two-stage Sieve Approach for Quote Attribution

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 460–470, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics A Two-stage Sieve Approach for Quote Attribution Grace Muzny1, Michael Fang1, Angel X. Chang1,2, and Dan Jurafsky1 1Stanford University, Stanford, CA 94305 2Princeton University, Princeton, NJ 08544 {muzny,mjfang,angelx,jurafsky}@cs.stanford.edu Abstract We present a deterministic sieve-based system for attributing quotations in literary text and a new dataset: QuoteLi31. Quote attribution, determining who said what in a given text, is important for tasks like creating dialogue systems, and in newer areas like computational literary studies, where it creates opportunities to analyze novels at scale rather than only a few at a time. We release QuoteLi3, which con- tains more than 6,000 annotations linking quotes to speaker mentions and quotes to speaker entities, and introduce a new al- gorithm for quote attribution. Our two- stage algorithm first links quotes to men- tions, then mentions to entities. Using two stages encapsulates difficult sub-problems and improves system performance. The modular design allows us to tune either for overall performance or for the high precision appropriate for many use cases. Our system achieves an average F-score of 87.5% across three novels, outperform- ing previous systems, and can be tuned for precision of 90.4% at a recall of 65.1%. 1 Introduction Dialogue, representing linguistic and social rela- tionships between characters, is an important com- ponent of literature. In this paper, we consider the task of quote attribution for literary text: identify- ing the speaker for each quote. This task is im- portant for developing realistic character-specific conversational models (Vinyals and Le, 2015; Li et al., 2016), analyzing discourse structure, and lit- erary studies (Muzny et al., 2016). But identifying speakers can be difficult; authors often refer to the 1Quotes in Literary text from 3 novels. speaker only indirectly via anaphora, or even omit mention of the speaker entirely (Table 1). Prior work has produced important datasets la- beling quotes in novels, providing training data for supervised methods. But some of these model the quote-attribution task at the mention-level (Elson and McKeown, 2010; O’Keefe et al., 2012), and others at the entity-level (He et al., 2013), leading to labels that are inconsistent across datasets. We propose entity-level quote attribution as the end goal but with mention-level quote attribution as an important intermediary step. Our first con- tribution is the QuoteLi3 dataset, a unified combi- nation of data from Elson and McKeown (2010) and He et al. (2013) with the addition of more than 3,000 new labels from expert annotators. This dataset provides both mention and entity labels for Pride and Prejudice, Emma, and The Steppe. Next, we describe a new deterministic system that models quote attribution as a two-step process that i) uses textual cues to identify the mention that corresponds to the speaker of a quote, and ii) re- solves the mention to an entity. This system im- proves over previous work by 0.8-2.1 F1 points and …

Content truncated to 500 words