The POTUS Corpus, a Database of Weekly Addresses for the Study of Stance in Politics and Virtual Agents

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1546–1553 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1546 The POTUS Corpus, a database of weekly addresses for the study of stance in politics and virtual agents Thomas Janssoone∗, K´evin Bailly∗, Ga¨el Richard†, Chlo´e Clavel† ∗Sorbonne Universit´es, UPMC Univ Paris 06, CNRS UMR 7222, ISIR,F-75005 Paris France {name}@isir.upmc.fr †T´el´ecom ParisTech,Universit´e Paris-Saclay,Paris, France {firstname.lastname}@telecom-paris.fr Abstract One of the main challenges in the field of Embodied Conversational Agent (ECA) is to generate socially believable agents. The common strategy for agent behaviour synthesis is to rely on dedicated corpus analysis. Such a corpus is composed of multimedia files of socio-emotional behaviors which have been annotated by external observers. The underlying idea is to identify interaction information for the agent’s socio-emotional behavior by checking whether the intended socio-emotional behavior is actually perceived by humans. Then, the annotations can be used as learning classes for machine learning algorithms applied to the social signals. This paper introduces the POTUS Corpus composed of high-quality audio-video files of political addresses to the American people. Two protagonists are present in this database. First, it includes speeches of former president Barack Obama to the American people. Secondly, it provides videos of these same speeches given by a virtual agent named Rodrigue. The ECA reproduces the original address as closely as possible using social signals automatically extracted from the original one. Both are annotated for social attitudes, providing information about the stance observed in each file. It also provides the social signals automatically extracted from Obama’s addresses used to generate Rodrigue’s ones. Keywords: Multi-modal Social Signal, Signal Processing, Embodied Conversational Agent, Audio Video Corpus, POTUS 1. Introduction ECAs can improve the quality of life in our modern digital society. For instance, they can help soldiers to recover from Post Traumatic Stress Disorder or help a patient to undergo treatment [Truong et al., 2015] if they are empathetic enough to provide support. The main challenge relies on the natu- ralness of interactions between Humans and ECAs. With this aim, an ECA should be able to express different stances towards the user, as for instance dominance for a tutor or friendliness for a companion. To give ECAs the capacity to express emotions and interper- sonal stances is one of the main challenges in their design [Vinciarelli et al., 2009b]. However, this field of research is thriving as more and more databases are available for the processing of social signals [Vinciarelli et al., 2012]. These databases are mainly audiovisual and provide mono-modal Figure 1: Facial Action Unit locations, images are obtained from http://www.cs.cmu.edu/˜face/facs.htm or multi-modal inputs to Machine Learning methods [Pent- land, 2004, Rudovic et al., 2017, Baltruˇsaitis et al., 2017]. Features such as prosodic descriptors or activations of facial muscles, potentially labeled as Action Units (AUs see Figure 1), are extracted to recognize a social expression (emotion, stance, behavior . . . ). Data is usually labeled by an external annotator who rates his/her …

Content truncated to 500 words