Adding Gesture, Posture and Facial Displays to the PoliModal Corpus of Political Interviews

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4320–4326 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4320 Adding Gesture, Posture and Facial Displays to the PoliModal Corpus of Political Interviews Daniela Trotta1, Alessio Palmero Aprosio2, Sara Tonelli2 and Annibale Elia1 1 Universit`a di Salerno 2 Fondazione Bruno Kessler {dtrotta,elia}@unisa.it {aprosio,satonelli}@fbk.eu Abstract This paper introduces a multimodal corpus in the political domain, which on top of transcribed face-to-face interviews presents the annotation of facial displays, hand gestures and body posture. While the fully annotated corpus consists of 3 interviews for a total of 120 minutes, it is extracted from a larger available corpus of 56 face-to-face interviews (14 hours) that has been manually annotated with information about metadata (i.e. tools used for the transcription, link to the interview etc.), pauses (used to mark a pause either between or within utterances), vocal expressions (marking non-lexical expressions such as burp and semi-lexical expressions such as primary interjections), deletions (false starts, repetitions and truncated words) and overlaps. In this work, we describe the additional level of annotation relating to non-verbal elements used by three Italian politicians belonging to three different political parties and who at the time of the talk-show were all candidates for the presidency of the Council of Ministers. We also present the results of some analyses aimed at identifying existing relations between the proxemic phenomena and the linguistic structures in which they occur, in order to capture recurring patterns and differences in the communication strategy. Keywords: multimodal corpora, political communication, multi-layered annotation 1. Introduction In the context of a political interview, the host, typically a journalist, acts as a mediator, a representative of the au- dience (Koutsombogera and Papageorgiou, 2010). This means that, if a politician manages to convince or deal with the criticism that the host addresses, then her/his trustwor- thiness, reliability and credibility will be easily established. In this situation, a politician is judged not only based on one’s arguments and rhetorical choices, but also on the at- titude, self-confidence, and in general on an overall con- vincing behaviour. For example, if a politician seems to be conversationally dominant and manages interruptions to a satisfactory degree, it is more likely that the host, and therefore the audience, will be convinced by the arguments put forward by the interviewee. In televised political interviews, politicians struggle to es- tablish an image for themselves as competent personalities, a goal which is considered as important as the topic under discussion. For this reason, analysing the combination of verbal and non-verbal elements (such as their image, in- cluding their non-verbal interactional behavior) is crucial and could be very interesting for scholars in political sci- ence and communication science, and in general to study consensus mechanisms. Indeed politicians make use of non-verbal means to express positive or negative evalua- tions towards persons or facts and thus raise emotions to the public through means that are absent in speech. At the same time, they have …

Content truncated to 500 words