Crowdsourced Multimodal Corpora Collection Tool

Crowdsourced Multimodal Corpora Collection Tool Patrik Jonell, Catharine Oertel, Dimosthenis Kontogiorgos, Jonas Beskow, Joakim Gustafson KTH Royal Institute of Technology, Sweden Lindstedsvägen 24, 114 28 Stockholm, Sweden {pjjonell, catha, diko, beskow}@kth.se; jocke@speech.kth.se Abstract In recent years, more and more multimodal corpora have been created. To our knowledge there is no publicly available tool which allows for acquiring controlled multimodal data of people in a rapid and scalable fashion. We therefore are proposing (1) a novel tool which will enable researchers to rapidly gather large amounts of multimodal data spanning a wide demo- graphic range, and (2) an example of how we used this tool for corpus collection of our “Attentive listener” multimodal corpus. The code is released under an Apache License 2.0 and available as an open-source repository, which can be found at https://github.com/kth-social-robotics/multimodal-crowdsourcing-tool. This tool will allow researchers to set-up their own multimodal data collection system quickly and create their own multimodal corpora. Finally, this paper provides a discussion about the advantages and disadvantages with a crowd-sourced data collection tool, especially in comparison to a lab recorded corpora. Keywords: crowdsourcing, human-computer interaction, multimodal corpus 1. Introduction In the last decade more and more efforts have been car- ried out to create multimodal corpora. Efforts have come from diverse fields such as psychology, linguistics and com- puter science. They shared the common goal of enabling researchers to gain a better understanding of how humans converse with one another. The traditional approach has been to record corpora in a lab environment. This approach has several disadvantages, however. One such disadvantage is that due to the con- straint of having to attend the recording on campus, most resulting corpora are quite homogeneous with regards to participants’ demographics (e.g. education level, socio- economic background, native language). Participants might also exhibit some degree of the Hawthorne effect, while be- ing observed in sterile lab environments, leading to biased behaviour (Parsons, 1974). A further disadvantage is the monetary cost of staff, equipment and the time spent when doing in-lab recordings, for example setting up equipment. Due to the constraints mentioned above more and more re- searchers have been turning to crowdsourcing as a means of gathering large quantities of data. In particular, crowd- sourcing has been extensively used for annotations as well as transcriptions (Gruenstein et al., 2009; Hipp et al., 2013; Rashtchian et al., 2010). While it has, to some degree, been used for creation of conversational corpora as well, these corpora have mainly been restricted to speech and text (Fi- latova, 2012; Orkin and Roy, 2009; Breazeal et al., 2013). One of the reasons for the lack of these kinds of corpora might be that there are few tools which facilitate the acqui- sition of such data. While there are tools for collecting con- versational data such as Voxforge1, there are currently, to our knowledge, no publicly available open-source tools that allow for crowdsourcing multimodal corpora while simul- 1http://www.voxforge.org/ Figure 1: Corpus creation using multiple crowdsourcing services where participants record themselves. taneously enabling …

Content truncated to 500 words