How Do We Answer Complex Questions: Discourse Structure of Long-form Answers

39129

Authors

nan

Abstract

AbstractLong-form answers, consisting of multiple sentences, can provide nuanced and comprehensive answers to a broader set of questions. To better understand this complex and understudied task, we study the functional structure of long-form answers collected from three datasets, ELI5, WebGPT and Natural Questions. Our main goal is to understand how humans organize information to craft complex answers. We develop an ontology of six sentence-level functional roles for long-form answers, and annotate 3.9k sentences in 640 answer paragraphs. Different answer collection methods manifest in different discourse structures. We further analyze model-generated answers – finding that annotators agree less with each other when annotating model-generated answers compared to annotating human-written answers. Our annotated data enables training a strong classifier that can be used for automatic analysis. We hope our work can inspire future research on discourse-level modeling and evaluation of long-form QA systems.

Keywords

nan

Full Content

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 298 - 307 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Multi-Party Empathetic Dialogue Generation: A New Task for Dialog Systems Lingyu Zhu1, Zhengkun Zhang1, Jun Wang2 Hongbin Wang3, Haiying Wu3, Zhenglu Yang1∗ 1TKLNDST, CS, Nankai University, 2MS, Ludong University, 3Mashang Consumer Finance Co., Ltd., China {zhulingyu,zhangzk2017,junwang}@mail.nankai.edu.cn, {hongbin.wang03,haiying.wu02}@msxf.com, yangzl@nankai.edu.cn Abstract Empathetic dialogue assembles emotion un- derstanding, feeling projection, and appropri- ate response generation. Existing work for em- pathetic dialogue generation concentrates on the two-party conversation scenario. Multi- party dialogues, however, are pervasive in re- ality. Furthermore, emotion and sensibility are typically confused; a refined empathy analysis is needed for comprehending fragile and nu- anced human feelings. We address these is- sues by proposing a novel task called Multi- Party Empathetic Dialogue Generation in this study. Additionally, a Static-Dynamic model for Multi-Party Empathetic Dialogue Genera- tion, SDMPED, is introduced as a baseline by exploring the static sensibility and dynamic emotion for the multi-party empathetic dia- logue learning, the aspects that help SDMPED achieve the state-of-the-art performance. 1 Introduction Empathetic conversation studies have been coming to the forefront in recent years owing to the in- creasing interest in dialogue systems. Empathetic dialogues not only provide dialogue partners with highly relevant contents but also project their feel- ings and convey a special emotion, that is, empathy. As revealed by previous studies (Fraser et al., 2018; Zhou et al., 2020), empathy can enhance conver- sation quality and transmit appropriate emotional responses to partners. Accordingly, most, if not all, existing work focuses on taking an emotional perspective in dialogue studies (Levinson et al., 2000; Kim et al., 2004; Bertero et al., 2016; Fraser et al., 2018; Rashkin et al., 2019). Although the empathetic conversation has re- ceived extensive attention, its exploration is still limited to the scenario with only two parties. In fact, multi-party chatting scenes are common in seminar discussions, conferences, and group chats. *Corresponding author. ﹏﹏ I have been fighting for so long. And I don’t want to talk this with my family or friends. I’m confused and lonely. [Sad] Oh, I am so sorry. It’s too bad for you. [Worried] Don’t worry! [Calm] Your words make me feel better. Thank you so much. Means so much. [Relxed] Weak Sensibility Moderate Sensibility Strong Sensibility ﹏﹏﹏﹏ You are not alone. Just be relaxed, I went through this and I truly understand how bad it is. [Supportive] Gee! Believe you can achieve it! [Optimistic] High Low Sensibility: Emotion: Negative Positive Happy to hear that. Working together, we can make the future better! [Happy] ﹏﹏﹏﹏﹏ Speaker 1 Speaker 1 Speaker 2 Speaker 3 Speaker 4 Speaker 4 Speaker 3 Figure 1: An empathetic dialogue example of multi- party. When people with different sensibilities respond to the same requests for help, their emotions and empa- thy differ. Different shades of red and blue denote the degree of positive and negative emotions, and different shades of green denote the degree of …

Content truncated to 500 words

Labeled Text

AbstractLong-form answers, consisting of multiple sentences, can provide nuanced and comprehensive answers to a broader set of questions. To better understand this complex and understudied task, we study the functional structure of long-form answers collected from three datasets, ELI5, WebGPT and Natural Questions. Our main goal is to understand how humans organize information to craft complex answers. We develop an ontology of six sentence-level functional roles for long-form answers, and annotate 3.9k sentences in 640 answer paragraphs. Different answer collection methods manifest in different discourse structures. We further analyze model-generated answers – finding that annotators agree less with each other when annotating model-generated answers compared to annotating human-written answers. Our annotated data enables training a strong classifier that can be used for automatic analysis. We hope our work can inspire future research on discourse-level modeling and evaluation of long-form QA systems.

Publication Details

Paper ID:
39129
Year:
2022
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan