Linguistic Models for Analyzing and Detecting Biased Language

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1650–1659, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Linguistic Models for Analyzing and Detecting Biased Language Marta Recasens Stanford University recasens@google.com Cristian Danescu-Niculescu-Mizil Stanford University Max Planck Institute SWS cristiand@cs.stanford.edu Dan Jurafsky Stanford University jurafsky@stanford.edu Abstract Unbiased language is a requirement for reference sources like encyclopedias and scientific texts. Bias is, nonetheless, ubiq- uitous, making it crucial to understand its nature and linguistic realization and hence detect bias automatically. To this end we analyze real instances of human edits de- signed to remove bias from Wikipedia ar- ticles. The analysis uncovers two classes of bias: framing bias, such as praising or perspective-specific words, which we link to the literature on subjectivity; and episte- mological bias, related to whether propo- sitions that are presupposed or entailed in the text are uncontroversially accepted as true. We identify common linguistic cues for these classes, including factive verbs, implicatives, hedges, and subjective inten- sifiers. These insights help us develop fea- tures for a model to solve a new prediction task of practical importance: given a bi- ased sentence, identify the bias-inducing word. Our linguistically-informed model performs almost as well as humans tested on the same task. 1 Introduction Writers and editors of reference works such as encyclopedias, textbooks, and scientific articles strive to keep their language unbiased. For ex- ample, Wikipedia advocates a policy called neu- tral point of view (NPOV), according to which articles should represent “fairly, proportionately, and as far as possible without bias, all signifi- cant views that have been published by reliable sources” (Wikipedia, 2013b). Wikipedia’s style guide asks editors to use nonjudgmental language, to indicate the relative prominence of opposing points of view, to avoid presenting uncontroversial facts as mere opinion, and, conversely, to avoid stating opinions or contested assertions as facts. Understanding the linguistic realization of bias is important for linguistic theory; automatically detecting these biases is equally significant for computational linguistics. We propose to ad- dress both by using a powerful resource: edits in Wikipedia that are specifically designed to remove bias. Since Wikipedia maintains a complete revi- sion history, the edits associated with NPOV tags allow us to compare the text in its biased (before) and unbiased (after) form, helping us better under- stand the linguistic realization of bias. Our work thus shares the intuition of prior NLP work apply- ing Wikipedia’s revision history (Nelken and Ya- mangil, 2008; Yatskar et al., 2010; Max and Wis- niewski, 2010; Zanzotto and Pennacchiotti, 2010). The analysis of Wikipedia’s edits provides valu- able linguistic insights into the nature of biased language. We find two major classes of bias-driven edits. The first, framing bias, is realized by sub- jective words or phrases linked with a particular point of view. In (1), the term McMansion, unlike homes, appeals to a negative attitude toward large and pretentious houses. The second class, episte- mological bias, is related to linguistic features that subtly (often via presupposition) focus on …

Content truncated to 500 words