Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1563–1580, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 1563 Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank Eleftheria Briakou and Marine Carpuat Department of Computer Science University of Maryland College Park, MD 20742, USA ebriakou@cs.umd.edu, marine@cs.umd.edu Abstract Detecting fine-grained differences in con- tent conveyed in different languages matters for cross-lingual NLP and multilingual cor- pora analysis, but it is a challenging ma- chine learning problem since annotation is expensive and hard to scale. This work im- proves the prediction and annotation of fine- grained semantic divergences. We introduce a training strategy for multilingual BERT mod- els by learning to rank synthetic divergent examples of varying granularity. We evalu- ate our models on the Rationalized English- French Semantic Divergences, a new dataset released with this work, consisting of English- French sentence-pairs annotated with seman- tic divergence classes and token-level ratio- nales. Learning to rank helps detect fine- grained sentence-level divergences more ac- curately than a strong sentence-level similar- ity model, while token-level predictions have the potential of further distinguishing between coarse and fine-grained divergences. 1 Introduction Comparing and contrasting the meaning of text conveyed in different languages is a fundamental NLP task. It can be used to curate clean paral- lel corpora for downstream tasks such as machine translation (Koehn et al., 2018), cross-lingual trans- fer learning, or semantic modeling (Ganitkevitch et al., 2013; Conneau and Lample, 2019), and it is also useful to directly analyze multilingual corpora. For instance, detecting the commonalities and di- vergences between sentences drawn from English and French Wikipedia articles about the same topic would help analyze language bias (Bao et al., 2012; Massa and Scrinzi, 2012), or mitigate differences in coverage and usage across languages (Yeung et al., 2011; Wulczyn et al., 2016; Lemmerich et al., 2019). This requires not only detecting coarse con- tent mismatches, but also fine-grained differences in sentences that overlap in content. Consider the following English and French sentences, sampled from the WikiMatrix parallel corpus. While they share important content, highlighted words convey meaning missing from the other language: EN Alexander Muir’s “The Maple Leaf For- ever” served for many years as an unofficial Canadian national anthem. FR Alexander Muir compose The Maple Leaf Forever (en) qui est un chant patriotique pro canadien anglais. GLOSS Alexander Muir composes The Maple Leaf Forever which is an English Canadian patriotic song. We show that explicitly considering diverse types of semantic divergences in bilingual text ben- efits both the annotation and prediction of cross- lingual semantic divergences. We create and re- lease the Rationalized English-French Semantic Divergences corpus (REFRESD), based on a novel divergence annotation protocol that exploits ratio- nales to improve annotator agreement. We intro- duce Divergent mBERT, a BERT-based model that detects fine-grained semantic divergences without supervision by learning to rank synthetic diver- gences of varying granularity. Experiments on RE- FRESD show that our model distinguishes seman- tically equivalent from divergent examples much better than …

Content truncated to 500 words