Generating Repetitions with Appropriate Repeated Words

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 852 - 859 July 10-15, 2022 ©2022 Association for Computational Linguistics Generating Repetitions with Appropriate Repeated Words Toshiki Kawamoto, Hidetaka Kamigaito, Kotaro Funakoshi and Manabu Okumura Tokyo Institute of technology {kawamoto,kamigaito,funakoshi,oku}@lr.pi.titech.ac.jp Abstract A repetition is a response that repeats words in the previous speaker’s utterance in a dialogue. Repetitions are essential in communication to build trust with others, as investigated in lin- guistic studies. In this work, we focus on repe- tition generation. To the best of our knowledge, this is the first neural approach to address repe- tition generation. We propose Weighted Label Smoothing, a smoothing method for explicitly learning which words to repeat during fine- tuning, and a repetition scoring method that can output more appropriate repetitions during decoding. We conducted automatic and human evaluations involving applying these methods to the pre-trained language model T5 for gen- erating repetitions. The experimental results indicate that our methods outperformed base- lines in both evaluations. 1 Introduction Dialogues can build a trusting relationship with others, thus are essential in our daily lives (Schein, 1993; Searle and Vanderveken, 1985). There are several types of responses in dialogues, and the one we focus on is repetitions (Tannen, 1987). A repe- tition is a response that uses the previous speaker’s words or phrases. Figure 1 shows an example. The phrases "a bear" and "came out" are repeated. Rep- etitions frequently appear in a conversation with diverse roles, e.g., to indicate attentive listening, confirm the previous utterance, and show agree- ment or sympathy (Machi, 2019; Shimojima et al., 2002). Many linguistic studies investigating repe- titions have concluded that they are important for building and strengthening relationships between speakers (Tannen et al., 1989; Johnstone, 2002; Norrick, 1987; Brown, 1999). From the above lin- guistic point of view, we can say that repetitions are indispensable in dialogues. Repetitions are similar to paraphrases and re- flections, which are component skills of counsel- Speaker: When I was driving, a bear suddenly came out . Listener: Oh. A bear came out !? Figure 1: Example repetition. Listener’s response uses words from previous speaker’s utterance. Yellow words indicate those that are repeated and green words indicate those in the repetition. ing (Theron, 2008), in terms of using the previ- ous speaker’s utterance. Paraphrases and reflec- tions have been generated using a template-based method (Han et al., 2013). While many studies have tackled general re- sponse generation with neural network-based frameworks (Adiwardana et al., 2020; Zhang et al., 2020), less attention has been paid to repetitions. This might be because they are buried in a huge amount of response data. Therefore, we focus on automatically generating repetitions. To the best of our knowledge, this is the first study on generating repetitions with a neural approach. We used the pre-trained language model T5 (Raffel et al., 2019) for generating repetitions because it has performed well in language generation in past few years (e.g., …

Content truncated to 500 words