Verbal and Nonverbal Clues for Real-life Deception Detection

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2336–2346, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Verbal and Nonverbal Clues for Real-life Deception Detection Ver´onica P´erez-Rosas, Mohamed Abouelenien, Rada Mihalcea, Yao Xiao, CJ Linton, Mihai Burzo University of Michigan (vrncapr,zmohamed,mihalcea,xyaoinum,cjlint,mburzo)@umich.edu Abstract Deception detection has been receiving an increasing amount of attention from the computational linguistics, speech, and multimodal processing communities. One of the major challenges encountered in this task is the availability of data, and most of the research work to date has been con- ducted on acted or artificially collected data. The generated deception models are thus lacking real-world evidence. In this paper, we explore the use of multi- modal real-life data for the task of decep- tion detection. We develop a new decep- tion dataset consisting of videos from real- life scenarios, and build deception tools relying on verbal and nonverbal features. We achieve classification accuracies in the range of 77-82% when using a model that extracts and fuses features from the lin- guistic and visual modalities. We show that these results outperform the human capability of identifying deceit. 1 Introduction As deceptive behavior occurs on a daily basis in different areas of life (Meyer, 2010; Smith et al., 2014), the need arises for automated methodolo- gies to detect deception in an efficient, yet reliable manner. There are many applications that can ben- efit from automatic deception identification, such as airport security screening, crime investigation and interrogation, interviews, advertisement, and others. In many of these settings, the polygraph test has been used as the main method to identify deceptive behavior. However, this method requires the use of skin-contact devices and human exper- tise, making it infeasible for large-scale applica- tions. Moreover, polygraph tests were shown to be misleading in multiple cases (Vrij, 2001; Gannon et al., 2009), as human judgment is often biased. Given the difficulties associated with the use of polygraph-like methods, learning-based ap- proaches have been proposed to address the de- ception detection task using a number of modali- ties, including text (Feng et al., 2012) and speech (Hirschberg et al., 2005; Newman et al., 2003). Unlike the polygraph methods, learning-based methods for deception detection rely mainly on data collected from deceivers and truth-tellers. The data is usually elicited from human contrib- utors, in a lab setting or via crowdsourcing. An important problem identified in this data-driven re- search is the lack of real data. Because of the arti- ficial setting, the subjects may not be emotionally aroused, as they may not take the experiments seri- ously given the lack of motivation and/or penalty. In this paper, we describe what we believe is a first attempt at building a multimodal system that detects deception in real-life settings. We collect a dataset consisting of 118 deceptive and truthful video clips, from real trials and live street inter- views aired in television shows. We use the tran- scription of these videos to extract several linguis- tic features, and we manually …

Content truncated to 500 words