Generating Textual Explanations for Machine Learning Models Performance: A Table-to-Text Task

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3542–3551 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3542 Generating Textual Explanations for Machine Learning Models Performance: A Table-to-Text Task Isaac Ampomah1, James Burton1, Amir Enshaei2,3, Noura Al Moubayed1 1Durham University, 2Newcastle University, 3Caspian Learning Ltd {isaac.k.ampomah, james.burton, noura.al-moubayed}@durham.ac.uk, amir.enshaei@newcastle.ac.uk Abstract Numerical tables are widely employed to communicate or report the classification performance of machine learning (ML) models with respect to a set of evaluation metrics. For non-experts, domain knowledge is required to fully understand and interpret the information presented by numerical tables. This paper proposes a new natural language generation (NLG) task where neural models are trained to generate textual explanations, analytically describing the classification performance of ML models based on the metrics’ scores reported in the tables. Presenting the generated texts along with the numerical tables will allow for a better understanding of the classification performance of ML models. We constructed a dataset comprising numerical tables paired with their corresponding textual explanations written by experts to facilitate this NLG task. Experiments on the dataset are conducted by fine-tuning pre-trained language models (T5 and BART) to generate analytical textual explanations conditioned on the information in the tables. Furthermore, we propose a neural module, Metrics Processing Unit (MPU), to improve the performance of the baselines in terms of correctly verbalising the information in the corresponding table. Evaluation and analysis conducted indicate, that exploring pre-trained models for data-to-text generation leads to better generalisation performance and can produce high-quality textual explanations. Keywords: Table-to-text generation, Structured data, Model performance summarisation 1. Introduction Structured data-to-text generation is a natural language generation (NLG) task aiming to generate text from a structured source of data, most commonly either graphs or tables (Wen et al., 2015; Liu et al., 2018; Strauss and Kipp, 2008). Earlier work on data-to-text genera- tion predominantly used rule-based methods (Goldberg et al., 1994; Reiter and Dale, 1997; Strauss and Kipp, 2008). These methods generate natural language text by employing linguistic rules and heuristics to select and populate pre-defined templates. However, a typical NLG system requires different sets of rules to perform content determination, text planning, sentence planning and surface realisation modules (Goldberg et al., 1994; van der Lee et al., 2017). This makes traditional NLG models difficult to maintain and less generalised. Recently, leveraging deep neural methods for NLG has been shown to outperform existing rule-based methods (Wen et al., 2015; Liu et al., 2018; Puduppully et al., 2019; Parikh et al., 2020; Suadaa et al., 2021). These models are usually trained end-to-end without the need for pre-defined linguistic rules. In broader terms, trans- fer learning has been shown to produce close to state- of-the-art performance for downstream NLP tasks with a limited amount of the dataset by utilising large pre- trained language models (Devlin et al., 2019; Radford et al., 2019). Furthermore, (Peng et al., 2020; Chen et al., 2020c) argue that pre-trained language models (GPT-2 (Radford et al., 2019) and T5 …

Content truncated to 500 words