Towards End-to-End Open Conversational Machine Reading

Findings of the Association for Computational Linguistics: EACL 2023, pages 2064–2076 May 2-6, 2023 ©2023 Association for Computational Linguistics Towards End-to-End Open Conversational Machine Reading Sizhe Zhou1,3, Siru Ouyang2,3, Zhuosheng Zhang2,3 Hai Zhao2,3, ∗ 1 UM-SJTU Joint Institute, Shanghai Jiao Tong University 2 Department of Computer Science and Engineering, Shanghai Jiao Tong University 3 Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University {sizhezhou,oysr0926,zhangzs}@sjtu.edu.cn,zhaohai@cs.sjtu.edu.cn Abstract In open-retrieval conversational machine reading (OR-CMR) task, machines are required to do multi-turn question answering given dialogue history and a textual knowledge base. Existing works generally utilize two independent modules to approach this problem’s two successive sub-tasks: first with a hard-label decision making and second with a question generation aided by various entailment reasoning methods. Such usual cascaded modeling is vulnerable to error propagation and prevents the two sub-tasks from being consistently optimized. In this work, we instead model OR-CMR as a unified text-to-text task in a fully end-to-end style. Experiments on the ShARC and OR-ShARC dataset show the effectiveness of our proposed end-to-end framework on both sub-tasks by a large margin, achieving new state-of-the- art results. Further ablation studies support that our framework can generalize to different backbone models. 1 Introduction In a multi-turn dialogue comprehension scenario, machines are expected to answer high-level questions through interactions with human beings until enough information is gathered to derive a satisfying answer (Zhu et al., 2018; Zhang et al., 2018; Zaib et al., 2020; Huang et al., 2020; Fan et al., 2020; Gu et al., 2021). As a specific and challenging dialogue comprehension task, conversational machine reading (CMR) (Saeidi et al., 2018) requires machines to understand the given user’s initial setting and dialogue history before the machine itself is able to give a final answer or inquire for more clarifications according to rule texts (see Figure 1). In terms of acquisition of rule texts which are the main reference for tackling the CMR, there ∗ Corresponding author. This paper was partially supported by Key Projects of National Natural Science Foundation of China (U1836222 and 61733011). Problem Setting: Problem Objective: Closed-Book Setting (only one gold rule): Open-Retrieval Setting (multiple retrieved rules): + Answer: Yes Inquiry Question: Is this a personal independence payment? User Scenario: The rent is $300 above the average in the area. User Initial Question: Can I ask for mandatory reconsideration for this benefit? Dialogue History: Answer: No Single Gold Rule Text: You can ask for mandatory reconsideration for benefits including: • Maternity Allowance. • Pension Credit. • Personal Independence Payment (PIP). • Sure Start Maternity Grant. Retrieved Rule Text 1: Eligible applicants may obtain direct loans… Retrieved Rule Text 2: Housing Benefit You dont need to ask for mandatory reconsideration… Retrieved Rule Text 3: Appeal to the tribunal… Follow-up Question 1: Is this benefit Maternity Allowance? Follow-up Answer 1: No Follow-up Question 2: Is this benefit a Pension Credit? Follow-up Answer 2: No Figure 1: CMR and OR-CMR Task Overview is closed-book setting where the rule texts are …

Content truncated to 500 words