Multi-Modal Annotation of Quest Games in Second Life

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 171–179, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Multi-Modal Annotation of Quest Games in Second Life Sharon Gower Small, Jennifer Stromer-Galley and Tomek Strzalkowski ILS Institute State University of New York at Albany Albany, NY 12222 small@albany.edu, jstromer@albany.edu, tomek@albany.edu Abstract We describe an annotation tool developed to as- sist in the creation of multimodal action- communication corpora from on-line massively multi-player games, or MMGs. MMGs typically involve groups of players (5-30) who control their avatars1, perform various activities (quest- ing, competing, fighting, etc.) and communicate via chat or speech using assumed screen names. We collected a corpus of 48 group quests in Second Life that jointly involved 206 players who generated over 30,000 messages in quasi- synchronous chat during approximately 140 hours of recorded action. Multiple levels of co- ordinated annotation of this corpus (dialogue, movements, touch, gaze, wear, etc) are required in order to support development of automated predictors of selected real-life social and demo- graphic characteristics of the players. The anno- tation tool presented in this paper was developed to enable efficient and accurate annotation of all dimensions simultaneously. 1 Introduction The aim of our project is to predict the real world characteristics of players of massively-multiplayer online games, such as Second Life (SL). We sought to predict actual player attributes like age or educa- tion levels, and personality traits including leader- ship or conformity. Our task was to do so using only the behaviors, communication, and interaction among the players produced during game play. To do so, we logged all players’ avatar movements, 1 All avatar names seen in this paper have been changed to protect players’ identities. “touch events” (putting on or taking off clothing items, for example), and their public chat messages (i.e., messages that can be seen by all players in the group). Given the complex nature of interpreting chat in an online game environment, we required a tool that would allow annotators to have a synchro- nized view of both the event action as well as the chat utterances. This would allow our annotators to correlate the events and the chat by marking them simultaneously. More importantly, being able to view game events enables more accurate chat anno- tation; and conversely, viewing chat utterances helps to interpret the significance of certain events in the game, e.g., one avatar following another. For example, an exclamation of: “I can’t do it!” could be simply a response (rejection) to a request from another player; however, when the game action is viewed and the speaker is seen attempting to enter a building without success, another interpretation may arise (an assertion, a call for help, etc.). The Real World (RW) characteristics of SL players (and other on-line games) may be inferred to varying degrees from the appearance of their avatars, the behaviors they engage in, as well as from their on-line chat communications. For exam- ple, the avatar gender generally matches the …

Content truncated to 500 words