Unbalanced Optimal Transport for Unbalanced Word Alignment

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3966–3986 July 9-14, 2023 ©2023 Association for Computational Linguistics Unbalanced Optimal Transport for Unbalanced Word Alignment Yuki Arase Osaka University, Japan arase@ist.osaka-u.ac.jp Han Bao Kyoto University, Japan bao@i.kyoto-u.ac.jp Sho Yokoi Tohoku University, Japan RIKEN, Japan yokoi@tohoku.ac.jp Abstract Monolingual word alignment is crucial to model semantic interactions between sentences. In particular, null alignment, a phenomenon in which words have no corresponding counter- parts, is pervasive and critical in handling se- mantically divergent sentences. Identification of null alignment is useful on its own to reason about the semantic similarity of sentences by indicating there exists information inequality. To achieve unbalanced word alignment that values both alignment and null alignment, this study shows that the family of optimal trans- port (OT), i.e., balanced, partial, and unbal- anced OT, are natural and powerful approaches even without tailor-made techniques. Our ex- tensive experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment, remarkably on challenging datasets with high null alignment frequencies. 1 Introduction Monolingual word alignment, which identifies se- mantically corresponding words in a sentence pair, has been actively studied as a crucial technique for modelling semantic relationships between sen- tences, such as for paraphrase identification, textual entailment recognition, and question answering (MacCartney et al., 2008; Das and Smith, 2009; Wang and Manning, 2010; Heilman and Smith, 2010; Yao et al., 2013a; Feldman and El-Yaniv, 2019). Its ability to declare redundant informa- tion in sentences is also useful for summarisation and sentence fusion (Thadani and McKeown, 2013; Brook Weiss et al., 2021). In addition, the align- ment information is valuable for interpretability of model predictions (Agirre et al., 2015; Li and Sriku- mar, 2016) and for realising interactive document exploration as well (Shapira et al., 2017; Hirsch et al., 2021). Many-to-many one-to-many Null alignment Figure 1: Unbalanced monolingual word alignment ma- trix; frequent null alignment (enclosed in red boxes) and mapping beyond one-to-one (enclosed in orange dashed boxes) are primary challenges. Figure 1 illustrates the challenges of monolin- gual word alignment. The first challenge is the null alignment, where words may not have cor- responding counterparts, which causes alignment asymmetricity (MacCartney et al., 2008). Null alignment is prevalent in semantically divergent sentences; indeed, the null alignment ratio reaches 63.8% in entailment sentence pairs used in our ex- periments (shown in the third row of Table 2). Its identification explicitly declares semantic gaps be- tween two sentences and helps reason about their semantic (dis)similarity (Li and Srikumar, 2016). The second challenge is that alignment beyond one- to-one mapping needs to be addressed. These chal- lenges constitute an unbalanced word alignment problem, where both word-by-word and null align- ment should be fully identified. This study reveals that a family of optimal trans- port (OT) are suitable tools for unbalanced word alignment. Among the OT problems, balanced OT (BOT) (Monge, 1781; Kantorovich, 1942)1 should be the most prominent …

Content truncated to 500 words