Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5511–5520, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5511 Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning Wanyun Cui§ Guangyu Zheng‡ Wei Wang‡ cui.wanyun@sufe.edu.cn, simonzheng96@gmail.com,weiwang1@fudan.edu.cn §Shanghai University of Finance and Economics ‡Shanghai Key Laboratory of Data Science, Fudan University Abstract We propose to solve the natural language in- ference problem without any supervision from the inference labels via task-agnostic multi- modal pretraining. Although recent studies of multimodal self-supervised learning also rep- resent the linguistic and visual context, their encoders for different modalities are coupled. Thus they cannot incorporate visual informa- tion when encoding plain text alone. In this paper, we propose Multimodal Aligned Contrastive Decoupled learning (MACD) net- work. MACD forces the decoupled text en- coder to represent the visual information vi- a contrastive learning. Therefore, it embed- s visual knowledge even for plain text infer- ence. We conducted comprehensive experi- ments over plain text inference datasets (i.e. SNLI and STS-B). The unsupervised MACD even outperforms the fully-supervised BiLST- M and BiLSTM+ELMO on STS-B. 1 Introduction Humans are not supervised by the natural language inference (NLI). Supervision is necessary for appli- cations in human-defined domains. For example, humans need the supervision of what is a noun before they do POS tagging, or what is a tiger in Wordnet before they classify an image of tiger in ImageNet. However, for NLI, people are able to entail that a⃝ A man plays a piano contra- dicts b⃝ A man plays the clarinet for his family without any supervision from the NLI labels. In this paper, we define such inference as a more general process of establishing associ- ations and inferences between texts, rather than strictly classifying whether two sentences entail or contradict each other. Inspired by this, we raise the core problem in this paper: Given a pair of natural language sentences, can machines entail their re- lationship without any supervision from inference labels? In his highly acclaimed paper, neuroscientist Moshe Bar claims that “predictions rely on the existing scripts in memory, which are the result of real as well as of previously imagined experi- ences” (Bar, 2009). The exemplar theory argues that humans use similarity to recognize different objects and make decisions (Tversky and Kahne- man, 1973; Homa et al., 1981). Analogy helps humans understand a novel object by linking it to a similar representation existing in memory (Bar, 2007). Such linking is facilitated by the object itself and its context (Bar, 2004). Con- text information has been widely applied in self- supervision learning (SSL) (Devlin et al., 2018; de Sa, 1994; He et al., 2020). Adapting context to NLI is even more straightforward. A simple idea of constant conjunction is that A causes B if they are constantly conjoined. Although constant con- junction contradicts “correlation is not causation”, modern neuroscience has confirmed that humans use it for reasoning in their mental world (Levy and Steward, 1983). For example, they found an increase in synaptic efficacy …

Content truncated to 500 words