Selecting Stickers in Open-Domain Dialogue through Multitask Learning

Findings of the Association for Computational Linguistics: ACL 2022, pages 3053 - 3060 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Selecting Stickers in Open-Domain Dialogue through Multitask Learning Zhexin Zhang12∗, Yeshuang Zhu2, Zhengcong Fei2, Jinchao Zhang2, Jie Zhou2 1The CoAI group, DCST; 1Institute for Artificial Intelligence; 1State Key Lab of Intelligent Technology and Systems; 1Beijing National Research Center for Information Science and Technology; 1Tsinghua University, Beijing 100084, China. 2Pattern Recognition Center, WeChat AI, Tencent Inc, China. zx-zhang18@mails.tsinghua.edu.cn, {yshzhu,dayerzhang,withtomzhou}@tencent.com, feizhengcong@ict.ac.cn, Abstract With the increasing popularity of online chat- ting, stickers are becoming important in our online communication. Selecting appropri- ate stickers in open-domain dialogue requires a comprehensive understanding of both di- alogues and stickers, as well as the rela- tionship between the two types of modali- ties. To tackle these challenges, we pro- pose a multitask learning method comprised of three auxiliary tasks to enhance the un- derstanding of dialogue history, emotion and semantic meaning of stickers. Extensive ex- periments conducted on a recent challenging dataset show that our model can better com- bine the multimodal information and achieve significantly higher accuracy over strong base- lines. Ablation study further verifies the ef- fectiveness of each auxiliary task. Our code is available at https://github.com/ nonstopfor/Sticker-Selection. 1 Introduction With the development of mobile messaging apps (e.g., WhatsApp and Messenger), visual content is getting more and more frequently used in our daily conversation, such as emojis and stickers. Com- pared with emojis, stickers are larger images con- sisting of drawing characters, symbolic icons, and text titles, and are hence more expressive and versa- tile (Konrad et al., 2020). Users send stickers along with text to show intimacy, express strong emotion, and experience the enjoyment of creativity (Tang and Hew, 2019). Despite the importance of stickers in daily com- munication, selecting stickers in open-domain di- alogue hasn’t been widely explored. In this pa- per, we address the task of selecting an appropriate sticker from a candidate set for an open-domain multi-turn dialogue. This task is a typical setting for various applications, e.g., automatically recom- mending stickers in messaging apps and building ∗Work done during internship at WeChat AI. 看嘛，这次回来我可能有点忙 我看到箱子了？你还多久回家呢？！ 哈哈是不是回家然后一起出来耍喃？ (I saw the suitcase? When will you be home?! Ha ha let’s hang out together when you arrive home?) (It depends. I may be a little busy this time) 家里回来了两个重量级的人物 放那么多天假你忙啥子嘛 (What are you up to with so many days off) (Two VIPs have returned home) 你不懂 啥子重量级嘛？ (You don’t get it) (What does VIP mean?) 你说了我就懂了嘛 (I will if you explain to me) angry cute brokenhearted goodbye Figure 1: An example of the sticker selection task. Given a dialogue history, the model needs to add a sticker to the last textual message which is the most ap- propriate one among a collection of candidate stickers (the one marked in the red rectangle). The words below in red denote the emotion or meaning of each sticker. more interesting and human-like chatbots which could respond with stickers. As shown in Figure 1, this task …

Content truncated to 500 words