PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6970 - 6984 December 7-11, 2022 ©2022 Association for Computational Linguistics PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance ∗ Yang Deng1, Wenqiang Lei2,†, Wenxuan Zhang3, Wai Lam1, Tat-Seng Chua4 1The Chinese University of Hong Kong, 2Sichuan University, 3DAMO Academy, Alibaba Group, 4Sea-NExT Joint Lab, National University of Singapore {ydeng,wlam}@se.cuhk.edu.hk, {wenqianglei,isakzhang}@gmail.com Abstract To facilitate conversational question answer- ing (CQA) over hybrid contexts in finance, we present a new dataset, named PACIFIC. Com- pared with existing CQA datasets, PACIFIC exhibits three key features: (i) proactivity, (ii) numerical reasoning, and (iii) hybrid context of tables and text. A new task is defined ac- cordingly to study Proactive Conversational Question Answering (PCQA), which combines clarification question generation and CQA. In addition, we propose a novel method, namely UniPCQA, to adapt a hybrid format of input and output content in PCQA into the Seq2Seq problem, including the reformulation of the numerical reasoning process as code genera- tion. UniPCQA performs multi-task learning over all sub-tasks in PCQA and incorporates a simple ensemble strategy to alleviate the error propagation issue in the multi-task learning by cross-validating top-k sampled Seq2Seq out- puts. We benchmark the PACIFIC dataset with extensive baselines and provide comprehensive evaluations on each sub-task of PCQA. 1 Introduction Financial question answering (QA) systems aim to answer user’s instant queries by selecting appropri- ate information from financial documents, which often contain a hybrid of tabular and textual con- tent, and performing complex quantitative analysis. Existing studies on financial QA (Zhu et al., 2021; Chen et al., 2021b; Zhu et al., 2022; Li et al., 2022a) mainly focus on building single-turn QA systems to passively respond to user queries. However, in real-world information-seeking applications (Za- mani et al., 2022), the system is expected to (i) answer highly context-dependent questions in a ∗ This work is substantially supported by a grant from the Research Grant Council of the Hong Kong Special Ad- ministrative Region, China (Project Code: 14200719) and the Sea-NExT Joint Lab. Work done when Yang Deng was a visiting research scholar in the Sea-NExT Joint Lab. † Corresponding author. multi-turn conversation, and (ii) proactively assist users in performing complicated information seeks. In an interactive setting, users tend to ask follow- up or co-referencing questions (Kundu et al., 2020; Liu et al., 2021) without repeating previous infor- mation, and provide a succinct or brief query that may be ambiguous or lack the necessary content. Especially in financial QA, the user queries often contain multiple constraints from different aspects for the concerned objective, as the examples shown in Fig. 1. Even just missing one constraint may cause ambiguity. Therefore, a proactive conversa- tional system that can help clarify the ambiguity is of great importance in financial QA. To this end, this paper introduces a new dataset to promote research into ProActive ConversatIonal question answering in FInanCe, named PACIFIC. PACIFIC is constructed by using the QA pairs in an expert-annotated …

Content truncated to 500 words