Learning Extraction Patterns for Subjective Expressions

Learning Extraction Patterns for Subjective Expressions∗ Ellen Riloff School of Computing University of Utah Salt Lake City, UT 84112 riloff@cs.utah.edu Janyce Wiebe Department of Computer Science University of Pittsburgh Pittsburgh, PA 15260 wiebe@cs.pitt.edu Abstract This paper presents a bootstrapping process that learns linguistically rich extraction pat- terns for subjective (opinionated) expressions. High-precision classifiers label unannotated data to automatically create a large training set, which is then given to an extraction pattern learning algorithm. The learned patterns are then used to identify more subjective sentences. The bootstrapping process learns many subjec- tive patterns and increases recall while main- taining high precision. 1 Introduction Many natural language processing applications could benefit from being able to distinguish between factual and subjective information. Subjective remarks come in a variety of forms, including opinions, rants, allega- tions, accusations, suspicions, and speculations. Ideally, information extraction systems should be able to distin- guish between factual information (which should be ex- tracted) and non-factual information (which should be discarded or labeled as uncertain). Question answering systems should distinguish between factual and specula- tive answers. Multi-perspective question answering aims to present multiple answers to the user based upon specu- lation or opinions derived from different sources. Multi- document summarization systems need to summarize dif- ferent opinions and perspectives. Spam filtering systems ∗This work was supported by the National Science Founda- tion under grants IIS-0208798, IIS-0208985, and IRI-9704240. The data preparation was performed in support of the North- east Regional Research Center (NRRC) which is sponsored by the Advanced Research and Development Activity (ARDA), a U.S. Government entity which sponsors and promotes research of import to the Intelligence Community which includes but is not limited to the CIA, DIA, NSA, NIMA, and NRO. must recognize rants and emotional tirades, among other things. In general, nearly any system that seeks to iden- tify information could benefit from being able to separate factual and subjective information. Some existing resources contain lists of subjective words (e.g., Levin’s desire verbs (1993)), and some em- pirical methods in NLP have automatically identified ad- jectives, verbs, and N-grams that are statistically associ- ated with subjective language (e.g., (Turney, 2002; Hatzi- vassiloglou and McKeown, 1997; Wiebe, 2000; Wiebe et al., 2001)). However, subjective language can be ex- hibited by a staggering variety of words and phrases. In addition, many subjective terms occur infrequently, such as strongly subjective adjectives (e.g., preposterous, un- seemly) and metaphorical or idiomatic phrases (e.g., dealt a blow, swept off one’s feet). Consequently, we believe that subjectivity learning systems must be trained on ex- tremely large text collections before they will acquire a subjective vocabulary that is truly broad and comprehen- sive in scope. To address this issue, we have been exploring the use of bootstrapping methods to allow subjectivity classifiers to learn from a collection of unannotated texts. Our re- search uses high-precision subjectivity classifiers to au- tomatically identify subjective and objective sentences in unannotated texts. This process allows us to generate a large set of labeled sentences automatically. The sec- ond emphasis …

Content truncated to 500 words