SPARQL-to-Text Question Generation for Knowledge-Based Conversational Applications

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 131–147 November 20–23, 2022. ©2022 Association for Computational Linguistics 131 SPARQL-to-Text Question Generation for Knowledge-Based Conversational Applications Gwénolé Lecorvé Morgan Veyret Quentin Brabant Lina M. Rojas-Barahona Orange – Lannion, France {gwenole.lecorve, morgan.veyret, quentin.brabant, linamaria.rojasbarahona}@orange.com Abstract This paper focuses on the generation of nat- ural language questions based on SPARQL queries, with an emphasis on conversational use cases (follow-up question-answering). It studies what can be achieved so far based on current deep learning models (namely pre- trained T5 and BART models). To do so, 4 knowledge-based QA corpora have been ho- mogenized for the task and a new challenge set is introduced. A first series of experiments analyzes the impact of different training se- tups, while a second series seeks to understand what is still difficult for these models. The re- sults from automatic metrics and human evalu- ation show that simple questions and frequent templates of SPARQL queries are usually well processed whereas complex questions and con- versational dimensions (coreferences and el- lipses) are still difficult to handle. The experi- mental material is publicly available1. 1 Introduction Knowledge-based approaches have recently be- come popular in the field of question answering (QA) and dialogue, raising the task of semantic parsing that seeks to map a user’s input questions to a formal representation that can be queried in a Knowledge Graph (KG). Alternatively, techniques have been proposed to verbalize small KGs, for in- stance to summarize information to a user. Still, the task which consists in verbalizing formal queries has been less studied. Yet, interesting applications could be derived from SPARQL-to-text question generation: for instance, the generation of tutoring systems where users can exercise on a topic, or the simulation of users for QA or dialogue systems. This is why this paper studies SPARQL-to-text question generation, with a particular considera- tion attached to the generation of questions in a conversational context. 1https://github.com/Orange-OpenSource/ sparql-to-text The objective of the paper is to study what can be achieved so far on SPARQL-to-text question generation using datasets and pretrained models available in the literature. In this regard, the contri- butions are the following: 1. The release of 5 knowledge-based QA cor- pora (including 2 conversational ones) that have been homogenized and prepared for the SPARQL-to-text task: 4 of them are derived from existing corpora, and the last one is a new challenge set with unseen query types and do- mains. 2. The comparison of different fine-tuning ap- proaches for BART and T5, using different in- put features and training data. As a results, we show that feeding the model with the expected answer and conversational contexts helps. We also show that these information can be effi- ciently replaced by a paragraph when available. 3. An in-depth analysis of the models’ perfor- mance with respect to varied query types. This highlights the limits of the current transformer-based …

Content truncated to 500 words