Is Shortest Always Best? The Role of Brevity in Logic-to-Text Generation

Proceedings of the The 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 180–192 July 13-14, 2023 ©2023 Association for Computational Linguistics Is Shortest Always Best? The Role of Brevity in Logic-to-Text Generation Eduardo Calòα Jordi Levyω Albert Gattα Kees van Deemterα αUtrecht University ωIIIA, CSIC {e.calo,a.gatt,c.j.vandeemter}@uu.nl levy@iiia.csic.es Abstract Some applications of artificial intelligence make it desirable that logical formulae be con- verted computationally to comprehensible nat- ural language sentences. As there are many logical equivalents to a given formula, finding the most suitable equivalent to be used as input for such a “logic-to-text" generation system is a difficult challenge. In this paper, we focus on the role of brevity: Are the shortest formulae the most suitable? We focus on propositional logic (PL), framing formula minimization (i.e., the problem of finding the shortest equivalent of a given formula) as a Quantified Boolean Formulae (QBFs) satisfiability problem. We ex- periment with several generators and selection strategies to prune the resulting candidates. We conduct exhaustive automatic and human eval- uations of the comprehensibility and fluency of the generated texts. The results suggest that while, in many cases, minimization has a posi- tive impact on the quality of the sentences gen- erated, formula minimization may ultimately not be the best strategy. https://gitlab.nl4xai.eu/eduardo. calo/brevity-PL 1 Introduction Logical formulae (LFs) are essential for scholars in many scientific fields, such as artificial intelli- gence and linguistics (e.g., formal semantics). For instance, some explainable artificial intelligence (XAI) methods (e.g., Guidotti et al., 2018) use LFs to provide interpretable and faithful explanations to black-box models. However, one of the drawbacks of these XAI methods is that their output formulae might be complex, hindering their understandabil- ity. Grasping the meaning of formulae is also hard for students of logic, especially when they are ex- posed to formalisms they are not yet accustomed to (Rector et al., 2004). Natural language generation (NLG) methods can be employed to simplify and translate LFs into understandable text in natural lan- guages (NLs), effectively providing explanations for them. Recently, NLG has made remarkable progress. In particular, in the context of data-to-text gener- ation, good results have been achieved in differ- ent domains and datasets, such as sport (e.g., the ROTOWIRE dataset (Wiseman et al., 2017; Thom- son et al., 2020)), restaurant (e.g., the E2E Chal- lenge (Smiley et al., 2018; Dušek et al., 2020)), or WebNLG (Gardent et al., 2017). However, the texts produced in these contexts are often relatively poor in logical and rhetorical structure. Moreover, neural language models still fail to encode the se- mantics of logical formulae (Traylor et al., 2021b) and acquire analytical and deductive logical rea- soning capabilities (Ryb et al., 2022). In particular, they struggle with logical connectives, where they fail to differentiate between conjunction and dis- junction (Traylor et al., 2021a). Logic-to-text gen- eration thus addresses an area of natural language processing where further progress is much needed. One way in which the task of generating NL from complex LFs could be facilitated is by …

Content truncated to 500 words