Fine-tuning BERT with Focus Words for Explanation Regeneration

Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics (*SEM), pages 125–130 Barcelona, Spain (Online), December 12–13, 2020 125 Fine-tuning BERT with Focus Words for Explanation Regeneration Isaiah Onando Mulang’1,Jennifer D’Souza2, S¨oren Auer2 1University of Bonn, Bonn, Germany {mulang}@iai.uni-bonn.de 2TIB Leibniz Information Centre for Science and Technology, Hannover, Germany {jennifer.dsouza | auer}@tib.eu Abstract Explanation generation introduced as the WorldTree corpus (Jansen et al., 2018) is an emerging NLP task involving multi-hop in- ference for explaining the correct answer in multiple-choice QA. It is a challenging task ev- idenced by low state-of-the-art performances (below 60% in F-score) demonstrated on the task. Of the state-of-the-art approaches, fine- tuned transformer-based (Vaswani et al., 2017) BERT models have shown great promise to- ward continued system performance improve- ments compared with approaches relying on surface-level cues alone that demonstrate per- formance saturation. In this work, we take a novel direction by addressing a particular linguistic characteristic of the data—we intro- duce a novel and lightweight focus feature in the transformer-based model and examine task improvements. Our evaluations reveal a signif- icantly positive impact of this lightweight fo- cus feature achieving highest scores, second only to a significantly computationally inten- sive system. 1 Introduction Multi-hop Inference for Explanation Regener- ation (MIER) is an emerging task in NLP that concerns aggregating facts to justify the correct answer choice in multiple-choice question answer- ing settings. The WorldTree corpus (Jansen et al., 2018) that introduced this as a community shared task (Jansen and Ustalov, 2019), was dedicated to finding systems that generate explanations for an- swers to elementary science questions based on the MIER paradigm. The core task essentially entails two main steps: identification of relevant explanation facts from a given knowledge base, followed by ranking the This work is licensed under a Creative Commons Attri- bution 4.0 International License. License details: http: //creativecommons.org/licenses/by/4.0/. a�tree�is�a�kind�of�a�plant Which�of�the�following�helps�the�leaves�break�down�after� Answer�:�decomposers they�have�fallen��off�the�tree�?�� a�leaf�is�a�part�of�a�tree a�plant�is�a�kind�of�a�organism if�a�leaf�falls�off�of�a�tree�then�that�leaf�is�dead decomposition�is�when�a�decomposer�breaks�down�dead organisms Figure 1: A elementary science question, its correct an- swer, and the ordered set of justification facts for the answer in the WorldTree corpus (Jansen et al., 2018) depicted as a subgraph of lexical matches. selected facts as a logically coherent paragraph. Figure 1 shows an example data instance from the WorldTree corpus (Jansen et al., 2018) that defines this task. It is basically a question, its correct an- swer, and a set of ordered facts that justify the correct answer choice. Depicted in the figure, as a subgraph, is a crucial characteristic feature of the data: that there are lexical overlaps between the question, the correct answer, and the explana- tion facts. In this respect, however, there are two notable caveats: 1) distractors—the lexical over- laps can also exist with irrelevant facts to the QA. E.g., given the KB fact: a decomposer is usually a bacterium or fungus, it has a lexical match to the answer, but it is not relevant to the explanation. Similarly, at least 13 other such matching irrelevant facts can be found in the WorldTree corpus (2018) knowledge base. …

Content truncated to 500 words