Improving the Generalizability of Collaborative Dialogue Analysis With Multi-Feature Embeddings

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3551–3565 May 2-6, 2023 ©2023 Association for Computational Linguistics Improving the Generalizability of Collaborative Dialogue Analysis With Multi-Feature Embeddings Ayesha Enayet University of Central Florida Orlando, FL USA ayeshaenayet@knights.ucf.edu Gita Sukthankar University of Central Florida Orlando, FL USA gitars@eecs.ucf.edu Abstract Conflict prediction in communication is inte- gral to the design of virtual agents that sup- port successful teamwork by providing timely assistance. The aim of our research is to an- alyze discourse to predict collaboration suc- cess. Unfortunately, resource scarcity is a prob- lem that teamwork researchers commonly face since it is hard to gather a large number of training examples. To alleviate this problem, this paper introduces a multi-feature embed- ding (MFeEmb) that improves the generaliz- ability of conflict prediction models trained on dialogue sequences. MFeEmb leverages tex- tual, structural, and semantic information from the dialogues by incorporating lexical, dialogue acts, and sentiment features. The use of dia- logue acts and sentiment features reduces per- formance loss from natural distribution shifts caused mainly by changes in vocabulary. This paper demonstrates the performance of MFeEmb on domain adaptation problems in which the model is trained on discourse from one task domain and applied to predict team performance in a different domain. The gener- alizability of MFeEmb is quantified using the similarity measure proposed by Bontonou et al. (2021). Our results show that MFeEmb serves as an excellent domain-agnostic representation for meta-pretraining a few-shot model on col- laborative multiparty dialogues. 1 Introduction For many natural language processing applica- tions, the ability to learn features that generalize well across multiple datasets is a key desidera- tum (Saikia et al., 2020). This paper introduces a new multi-feature embedding, MFeEmb, that increases the generalizability of models learned from collaborative multiparty dialogues. Dialogues are different from single-author documents in that, along with textual information, they contain com- munication patterns that may serve as indicators of social dynamics. Treating a dialogue as a mere text collection ignores valuable information. We advocate exploiting implicit features present in mul- tiparty dialogues that are less vulnerable to distri- bution shifts resulting from task domain changes. This paper demonstrates the usage of MFeEmb on a communication analysis task: conflict pre- diction. Teamwork research faces a challenge of resource scarcity since the human subjects datasets are quite small (less than 100 samples), due to the difficulty of recruiting teams and the time con- suming nature of many group tasks. A variety of social phenomena have been investigated within team communication research, including entrain- ment (Rahimi and Litman, 2020) and emergent leadership (Maese et al., 2021). Frequency of com- munication is not in itself a good predictor of team performance, but a meta-analysis conducted by Marlow et al. (2018) that drew upon data from 150 studies conducted on 9702 teams concluded that high quality communication is positively re- lated to team performance in many task domains. Conversely, process conflict, “disagreement among group members about the content …

Content truncated to 500 words