Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues

Proceedings of NAACL-HLT 2019, pages 1448–1459 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1448 Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues Sungjoon Park 1, Donghyun Kim 2, Alice Oh 1 1 School of Computing, KAIST, Republic of Korea 2 Trost, Humart Company, Inc., Republic of Korea sungjoon.park@kaist.ac.kr, kdhyeon821@gmail.com alice.oh@kaist.edu Abstract The recent surge of text-based online coun- seling applications enables us to collect and analyze interactions between counselors and clients. A dataset of those interactions can be used to learn to automatically classify the client utterances into categories that help counselors in diagnosing client status and pre- dicting counseling outcome. With proper anonymization, we collect counselor-client di- alogues, define meaningful categories of client utterances with professional counselors, and develop a novel neural network model for clas- sifying the client utterances. The central idea of our model, ConvMFiT, is a pre-trained con- versation model which consists of a general language model built from an out-of-domain corpus and two role-specific language models built from unlabeled in-domain dialogues. The classification result shows that ConvMFiT out- performs state-of-the-art comparison models. Further, the attention weights in the learned model confirm that the model finds expected linguistic patterns for each category. 1 Introduction Some mental disorders are known to be treated ef- fectively through psychotherapy. However, people in need of psychotherapy may find it challenging to visit traditional counseling services because of time, money, emotional barriers, and social stigma (Bearse et al., 2013). Recently, technology- mediated psychotherapy services emerged to alle- viate these barriers. Mobile-based psychotherapy programs (Mantani et al., 2017), fully automated chatbots (Ly et al., 2017; Fitzpatrick et al., 2017), and intervention through smart devices (Torrado et al., 2017) are examples. Among them, text- based online counseling services with professional counselors are becoming popular because clients can receive these services without traveling to an office and with reduced financial burden com- pared to traditional face-to-face counseling ses- sions (Hull, 2015). In text-based counseling, the communication environment changes from face-to-face counsel- ing sessions. The counselor cannot read non- verbal cues from their clients, and the client uses text messages rather than spoken utterances to deliver their thoughts and feelings, resulting in changes of dynamics in the counseling relation- ship. Previous studies explored computational ap- proaches to analyzing the dynamic patterns of re- lationship between the counselor and the client by focusing on the language of counselors (Imel et al., 2015; Althoff et al., 2016), clustering topics of client issues (Dinakar et al., 2014), and look- ing at therapy outcomes (Howes et al., 2014; Hull, 2015). Unlike previous studies, we take a computa- tional approach to analyze client responses from the counselor’s perspective. Client responses in counseling are crucial factors for judging the counseling outcome and for understanding the sta- tus of the client. So we build a novel catego- rization scheme of client utterances, and we base our categorization scheme on the cognitive behav- ioral theory (CBT), a widely used theory in psy- chotherapy. Also, in …

Content truncated to 500 words