Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 850–862 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 850 Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks Endri Kacupaj1, Joan Plepi2, Kuldeep Singh3, Harsh Thakkar4, Jens Lehmann1,5, and Maria Maleshkova1 1Smart Data Analytics Group, University of Bonn, Bonn, Germany 2Technische Universit¨at Darmstadt, Darmstadt, Germany 3Zerotha Research and Cerence GmbH, Germany 4Zerotha Research and Osthus GmbH, Germany 5Fraunhofer IAIS, Dresden, Germany {kacupaj,jens.lehmann,maleshkova}@cs.uni-bonn.de joan.plepi@tu-darmstadt.de kuldeep.singh1@cerence.com harsh.thakkar@osthus.com jens.lehmann@iais.fraunhofer.de Abstract This paper addresses the task of (complex) conversational question answering over a knowledge graph. For this task, we propose LASAGNE (muLti-task semAntic parSing with trAnsformer and Graph atteNtion nEtworks). It is the first approach, which employs a transformer architecture extended with Graph Attention Networks for multi-task neural semantic parsing. LASAGNE uses a transformer model for generating the base logical forms, while the Graph Attention model is used to exploit correlations between (entity) types and predicates to produce node representations. LASAGNE also includes a novel entity recognition module which detects, links, and ranks all relevant entities in the question context. We evaluate LASAGNE on a standard dataset for complex sequential question answering, on which it outperforms existing baseline averages on all question types. Specifically, we show that LASAGNE improves the F1-score on eight out of ten question types; in some cases, the increase in F1-score is more than 20% compared to the state of the art. 1 Introduction Since their inception in the late 2000s, pub- licly available Knowledge Graphs (e.g., DBpe- dia (Lehmann et al., 2015) and Yago (Suchanek et al., 2007)) have been widely used as a source of knowledge in several natural language process- ing (NLP) tasks such as entity linking, relation extraction, fact-checking, and question answering. Question answering (QA), in particular, is an essen- tial task that maps a user natural language question Figure 1: Conversational Question Answering task with examples similar to CSQA dataset (Saha et al., 2018). to a query over a knowledge graph (KG) to retrieve the correct answer (Singh et al., 2018). With the increasing popularity of intelligent personal assis- tants (e.g., Alexa, Siri), the research focus has been shifted to conversational question answering that involves multi-turn dialogues, incorporating the phenomenon of anaphora and ellipses (Christmann et al., 2019; Shen et al., 2019)(c.f. Figure 1). Conversational QA is often realised by using semantic parsing approaches, mapping an utter- ance to a logic form for extracting answers from a KG (Guo et al., 2018; Shen et al., 2019). The state of the art for semantic parsing approaches de- composes the semantic parsing process into two stages (Shen et al., 2019). First, a logical form is generated based on low-level features and then 851 the missing details are filled by considering both the question and the template. Other approaches (Dong and Lapata, 2016; Guo et al., 2018; Liang et al., 2017) first employ an entity linking model to identify entities in the question and subsequently use …

Content truncated to 500 words