Weakly Supervised Role Identification in Teamwork Interactions

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1671–1680, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Weakly Supervised Role Identification in Teamwork Interactions Diyi Yang School of Computer Science Carnegie Mellon University Pittsburgh, PA, 15213, USA diyiy@cs.cmu.edu Miaomiao Wen School of Computer Science Carnegie Mellon University Pittsburgh, PA, 15213, USA mwen@cs.cmu.edu Carolyn Penstein Ros´e School of Computer Science Carnegie Mellon University Pittsburgh, PA, 15213, USA cprose@cs.cmu.edu Abstract In this paper, we model conversational roles in terms of distributions of turn level behaviors, including conversation acts and stylistic markers, as they occur over the whole interaction. This work presents a lightly supervised approach to inducing role definitions over sets of contributions within an extended interaction, where the supervision comes in the form of an out- come measure from the interaction. The identified role definitions enable a map- ping from behavior profiles of each par- ticipant in an interaction to limited sized feature vectors that can be used effectively to predict the teamwork outcome. An em- pirical evaluation applied to two Massive Open Online Course (MOOCs) datasets demonstrates that this approach yields su- perior performance in learning representa- tions for predicting the teamwork outcome over several baselines. 1 Introduction In language technologies research seeking to model conversational interactions, modeling ap- proaches have aimed to identify conversation acts (Paul, 2012; Wallace et al., 2013; Bhatia et al., 2014) on a per turn basis, or to identify stances (Germesin and Wilson, 2009; Mukherjee et al., 2013; Piergallini et al., 2014; Hasan and Ng, 2014) that characterize the nature of a speaker’s ori- entation within an interaction over several turns. What neither of these two perspectives quite offer is a notion of a conversational role. And yet, conversational role is a concept with great utility in current real world applications where language technologies may be applied. Important teamwork is achieved through collab- oration where discussion is an important medium for accomplishing work. For example, distributed work teams are becoming increasingly the norm in the business world where creating innovative products in the networked world is a common practice. This work requires the effective ex- change of expertise and ideas. Open source and open collaboration organizations have suc- cessfully aggregated the efforts of millions of volunteers to produce complex artifacts such as GNU/Linux and Wikipedia. Discussion towards decision making about how to address problems that arise or how to extend work benefit from effective conversational interactions. With a grow- ing interest in social learning in large online platforms such as Massive Open Online Courses (MOOCs), students form virtual study groups and teams to complete a course project, and thus may need to coordinate and accomplish the work through discussion. In all such environments, discussions serve a useful purpose, and thus the effectiveness of the interaction can be measured in terms of the quality of the resulting product. We present a modeling approach that leverages the concept of latent …

Content truncated to 500 words