Verification and Implementation of Language-Based Deception Indicators in Civil and Criminal Narratives

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 41–48 Manchester, August 2008 Verification and Implementation of Language-Based Deception Indicators in Civil and Criminal Narratives Joan Bachenko Deception Discovery Technologies Oxford, NJ 07863 jbachenko@comcast.net Eileen Fitzpatrick Montclair State University Montclair, NJ 07043 fitzpatricke@mail.montclair.edu Michael Schonwetter Deception Discovery Technologies Minneapolis, MN 55416 mschonwetter@synchronvideo.com Abstract Our goal is to use natural language proc- essing to identify deceptive and non- deceptive passages in transcribed narra- tives. We begin by motivating an analy- sis of language-based deception that relies on specific linguistic indicators to discover deceptive statements. The indi- cator tags are assigned to a document us- ing a mix of automated and manual methods. Once the tags are assigned, an interpreter automatically discriminates between deceptive and truthful state- ments based on tag densities. The texts used in our study come entirely from “real world” sources—criminal state- ments, police interrogations and legal tes- timony. The corpus was hand-tagged for the truth value of all propositions that could be externally verified as true or false. Classification and Regression Tree techniques suggest that the approach is feasible, with the model able to identify 74.9% of the T/F propositions correctly. Implementation of an automatic tagger with a large subset of tags performed well on test data, producing an average score of 68.6% recall and 85.3% preci- © 2008. Licensed under the Creative Commons Attri- bution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc- sa/3.0/). Some rights reserved. sion when compared to the performance of human taggers on the same subset. 1. Introduction The ability to detect deceptive statements in text and speech has broad applications in law en- forcement and intelligence gathering. The scien- tific study of deception in language dates at least from Undeutsch (1954, 1989), who hypothesized that it is “not the veracity of the reporting person but the truthfulness of the statement that matters and there are certain relatively exact, definable, descriptive criteria that form a key tool for the determination of the truthfulness of statements”. Reviews by Shuy (1998), Vrij (2000), and De- Paulo et al. (2003) indicate that many types of deception can be identified because the liar’s verbal and non-verbal behavior varies considera- bly from that of the truth teller’s. Even so, the literature reports that human lie detectors rarely perform at a level above chance. Vrij (2000) gives a summary of 39 studies of human ability to detect lies. The majority of the studies report accuracy rates between 45-60%, with the mean accuracy rate at 56.6%. The goal of our research is to develop and implement a system for automatically identifying deceptive and truthful statements in narratives and transcribed interviews. We focus exclusively on verbal cues to deception for this initial experiment, ignoring at present potential prosodic cues (but see Hirschberg et al.). 41 In this paper, we describe a language-based analysis of deception that we have constructed and tested using “real world” sources—criminal narratives, police interrogations and legal testimony. Our analysis comprises two components: a set of deception indicators …

Content truncated to 500 words