A Tangled Web: The Faint Signals of Deception in Text - Boulder Lies and Truth Corpus (BLT-C)

A Tangled Web: The Faint Signals of Deception in Text Boulder Lies and Truth Corpus (BLT-C) Franco Salvetti†, John B. Lowe‡, and James H. Martin♯ Department of Computer Science, University of Colorado at Boulder, 1111 Engineering Drive, Boulder, CO USA†,♯ Department of Linguistics, University of California at Berkeley, 1203 Dwinelle Hall, Berkeley, CA USA‡ franco.salvetti@colorado.edu†, jblowe@berkeley.edu‡, martin@colorado.edu♯ Abstract We present an approach to creating corpora for use in detecting deception in text, including a discussion of the challenges peculiar to this task. Our approach is based on soliciting several types of reviews from writers and was implemented using Amazon Mechanical Turk. We describe the multi-dimensional corpus of reviews built using this approach, available free of charge from LDC as the Boulder Lies and Truth Corpus (BLT-C). Challenges for both corpus creation and the deception detection include the fact that human performance on the task is typically at chance, that the signal is faint, that paid writers such as turkers are sometimes deceptive, and that deception is a complex human behavior; manifestations of deception depend on details of domain, intrinsic properties of the deceiver (such as education, linguistic competence, and the nature of the intention), and specifics of the deceptive act (e.g., lying vs. fabricating.) To overcome the inherent lack of ground truth, we have developed a set of semi-automatic techniques to ensure corpus validity. We present some preliminary results on the task of deception detection which suggest that the BLT-C is an improvement in the quality of resources available for this task. Keywords: boulder lies and truth corpus, deception detection, corpus construction 1. Introduction Oh! what a tangled web we weave When first we practise to deceive!1 Deception, as a conscious and deliberate act, is a socially pervasive psycholinguistic phenomenon—from lies on the witness stand during legal trials to fabricated online prod- uct reviews. Its detection in human communication has long been of great interest in real-life situations involving law enforcement (Granhag and Str¨omwall, 2004), national security (National Research Council, 2003), and business (Xiao and Benbasat, 2011)—just to mention a few. The techniques employed for the detection of deception are varied, ingenious, and often dramatic—from the ancient Chinese method of spitting dry rice to the modern poly- graph. Deception detection has been the subject of investigation within psychology, social science, and linguistics, where it has mainly been based on qualitative and quantitative ob- servations of gesture, facial expression and voice analysis. Nonetheless, there is only a little scientific work, most of it quite recent, that has been done on the theoretical under- pinnings of systems for automatically detecting deception in written text. One of the principal challenges to making progress on com- putational methods for detecting deception is a peculiar characteristic of the task: human beings are not very good at it. In fact, human performance in detecting deception is no better than chance (Bond and DePaulo, 2006) and sometimes even below chance, due to (among other rea- sons) truth bias (Vrij, 2008). For this reason it is virtually impossible …

Content truncated to 500 words