Rationalization through Concepts

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 761–775 August 1–6, 2021. ©2021 Association for Computational Linguistics 761 Rationalization through Concepts Diego Antognini and Boi Faltings École Polytechnique Fédérale de Lausanne, Switzerland firstname.lastname@epfl.ch Abstract Automated predictions require explanations to be interpretable by humans. One type of ex- planation is a rationale, i.e., a selection of in- put features such as relevant text snippets from which the model computes the outcome. How- ever, a single overall selection does not pro- vide a complete explanation, e.g., weighing several aspects for decisions. To this end, we present a novel self-interpretable model called ConRAT. Inspired by how human explanations for high-level decisions are often based on key concepts, ConRAT extracts a set of text snip- pets as concepts and infers which ones are de- scribed in the document. Then, it explains the outcome with a linear aggregation of concepts. Two regularizers drive ConRAT to build in- terpretable concepts. In addition, we propose two techniques to boost the rationale and pre- dictive performance further. Experiments on both single- and multi-aspect sentiment classi- fication tasks show that ConRAT is the first to generate concepts that align with human ratio- nalization while using only the overall label. Further, it outperforms state-of-the-art meth- ods trained on each aspect label independently. 1 Introduction Neural models have become the standard for many tasks, owing to their large performance gains. However, their adoption in decision-critical fields is more limited because of their lack of inter- pretability, particularly with textual data. One of the simplest means of explaining predic- tions of complex models is by selecting relevant input features. Attention mechanisms (Bahdanau et al., 2015) model the selection using a condi- tional importance distribution over the inputs, but the resulting explanations are noisy (Jain and Wal- lace, 2019; Pruthi et al., 2020). Multi-head at- tention (Vaswani et al., 2017) extends attention mechanisms to attend information from different Document Overall Aspect Label: Positive ConRAT The pour is a hazy straw color with an initially fluffy white head that […]. An amazingly funky and tart aroma is present immediately. Lots of […]. The tartness is bright and green, lots of lemons and apples. The body is somewhat light and crisp, with a great level of effervescence and slight dryness. This is an absolutely fantastic beer. I would drink this […] Concept 1: Mouthfeel Concept 2: Aroma Concept 3: Appearance Concept 4: Taste Concept 5: Overall Positive Prediction Explanation The pour is a hazy straw color with an initially fluffy white head that […]. An amazingly funky and tart aroma is present immediately. Lots of […]. The tartness is bright and green, lots of lemons and apples. The body is somewhat light and crisp, with a great level of effervescence and slight dryness. This is an absolutely fantastic beer. I would drink this […] Figure 1: An illustration of ConRAT. Given a beer re- view, ConRAT identifies five excerpts that relate to par- ticular concepts of beers (i.e., the explanation), de- picted in …

Content truncated to 500 words