Acoustic-Prosodic and Lexical Cues to Deception and Trust: Deciphering How People Detect Lies

Acoustic-Prosodic and Lexical Cues to Deception and Trust: Deciphering How People Detect Lies Xi (Leslie) Chen, Sarah Ita Levitan, Michelle Levine, Marko Mandic & Julia Hirschberg Department of Computer Science Columbia University New York, NY, USA {xi chen, sarahita, michelle}@cs.columbia.edu, mm5305@columbia.edu, julia@cs.columbia.edu Abstract Humans rarely perform better than chance at lie detection. To better understand human perception of deception, we created a game framework, LieCatcher, to collect ratings of perceived deception using a large corpus of deceptive and truthful interviews. We analyzed the acoustic-prosodic and linguistic characteristics of language trusted and mis- trusted by raters and compared these to char- acteristics of actual truthful and deceptive language to understand how perception aligns with reality. With this data we built classifiers to automatically distinguish trusted from mis- trusted speech, achieving an F1 of 66.1%. We next evaluated whether the strategies raters said they used to discriminate between truthful and deceptive responses were in fact useful. Our results show that, although several prosodic and lexical features were consistently perceived as trustworthy, they were not reliable cues. Also, the strategies that judges reported using in deception detection were not helpful for the task. Our work sheds light on the nature of trusted language and provides insight into the challenging problem of human deception detection. 1 Introduction Humans are notoriously poor lie detectors, most performing at chance level or worse (Bond Jr and DePaulo, 2006). This result has been found across a wide variety of deception detection tasks, in multiple modalities, and in different cultures. Al- though poor performance has been well-attested, very little work has been done to understand why humans perform so poorly at detecting lies. Because humans are so poor at deception detec- tion, there have been many efforts to develop automated methods to detect deception in multiple modalities. Biometric indicators, typically mea- sured by the polygraph (a device used to detect lies by measuring blood pressure, pulse, respiration, and skin conductivity), have been shown to perform poorly at deception detection (Eriksson and Lacerda, 2007). Facial expressions (Ekman, 2009a), gestures and body posture (Lu et al., 2005; Tsechpenakiset al., 2005), and even brain imaging (Meijer and Verschuere, 2017) have been explored as potential indicators of deception. Some of these features are difficult or expensive to capture auto- matically, or are too invasive to be practical for general use. In recent years, automatic deception detection has gained popularity in the speech and NLP communities. Language cues have the advan- tage of being inexpensive, non-invasive, and easy to collect automatically. More importantly, prior research examining linguistic cues to deception has been promising. Researchers have used ma- chine learning to identify deceptive language in various domains, including court testimonies (Fornaciari and Poesio, 2013), hotel reviews (Ott et al., 2011), and interview dialogues (Levitan et al., 2018b). These automated methods have demonstrated that machine learning classifiers can indeed identify deceptive language with accuracy between 70% and 90%, depending on the task—much better than human performanceon the same task. These studies have also identified specific characteristics …

Content truncated to 500 words