Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5609–5626 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5609 Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport Kyle Swanson* ASAPP, Inc. New York, USA kswanson@asapp.com Lili Yu* ASAPP, Inc. New York, USA liliyu@asapp.com Tao Lei ASAPP, Inc. New York, USA tao@asapp.com Abstract Selecting input features of top relevance has become a popular method for building self- explaining models. In this work, we ex- tend this selective rationalization approach to text matching, where the goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the down- stream prediction. Our approach employs op- timal transport (OT) to find a minimal cost alignment between the inputs. However, di- rectly applying OT often produces dense and therefore uninterpretable alignments. To over- come this limitation, we introduce novel con- strained variants of the OT problem that re- sult in highly sparse alignments with control- lable sparsity. Our model is end-to-end differ- entiable using the Sinkhorn algorithm for OT and can be trained without any alignment an- notations. We evaluate our model on the Stack- Exchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very sparse ratio- nale selections with high fidelity while preserv- ing prediction accuracy compared to strong at- tention baseline models.† 1 Introduction The growing complexity of deep neural networks has given rise to the desire for self-explaining mod- els (Li et al., 2016; Ribeiro et al., 2016; Zhang et al., 2016; Ross et al., 2017; Sundararajan et al., 2017; Alvarez-Melis and Jaakkola, 2018b; Chen et al., 2018a). In text classification, for instance, one popular method is to design models that can perform classification using only a rationale, which is a subset of the text selected from the model in- put that fully explains the model’s prediction (Lei et al., 2016; Bastings et al., 2019; Chang et al., 2019). This selective rationalization method, often *Denotes equal contribution. †Our code is publicly available at https://github. com/asappresearch/rationale-alignment. Can I find duplicate songs with different names? I have so many duplicate songs but they have different names. Is there an application I can use to find and delete the duplicates? How to find (and delete) duplicate files? I have a largish music collection and there are some duplicates in there. Is there any way to find duplicate files. At a minimum by doing a hash and seeing if two files have the same hash. … I’m happy using the command line if that is the easiest way. Figure 1: An illustration of a text matching rationale for detecting similar forum posts. trained to choose a small yet sufficient number of text spans, makes it easy to interpret the model’s prediction by examining the selected text. In contrast to classification, very little progress has been made toward rationalization for text matching models. The task of text matching encom- passes a wide range of downstream applications, such as similar document recommendation (dos Santos …

Content truncated to 500 words