A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4747–4761 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4747 A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching Jihun Choi, Taeuk Kim, Sang-goo Lee Department of Computer Science and Engineering Seoul National University {jhchoi,taeuk,sglee}@europa.snu.ac.kr Abstract We present a latent variable model for pre- dicting the relationship between a pair of text sequences. Unlike previous auto-encoding– based approaches that consider each sequence separately, our proposed framework utilizes both sequences within a single model by gen- erating a sequence that has a given relation- ship with a source sequence. We further extend the cross-sentence generating frame- work to facilitate semi-supervised training. We also define novel semantic constraints that lead the decoder network to generate seman- tically plausible and diverse sequences. We demonstrate the effectiveness of the proposed model from quantitative and qualitative exper- iments, while achieving state-of-the-art results on semi-supervised natural language inference and paraphrase identification. 1 Introduction Text sequence matching is a task whose objec- tive is to predict the degree of match between two or more text sequences. For example, in natu- ral language inference, a system has to infer the relationship between a premise and a hypothesis sentence, and in paraphrase identification a sys- tem should find out whether a sentence is a para- phrase of the other. Since various natural language processing problems, including answer sentence selection, text retrieval, and machine comprehen- sion, involve text sequence matching components, building a high-performance text matching model plays a key role in enhancing quality of systems for these problems (Tan et al., 2016; Rajpurkar et al., 2016; Wang and Jiang, 2017; Tymoshenko and Moschitti, 2018). With the emergence of large-scale corpora, end- to-end deep learning models are achieving remark- able results on text sequence matching; these in- clude architectures that are linguistically moti- vated (Bowman et al., 2016a; Chen et al., 2017a; Kim et al., 2019), that introduce external knowl- edge (Chen et al., 2018), and that use atten- tion mechanisms (Parikh et al., 2016; Shen et al., 2018b). The recent deep neural network–based work on text matching could roughly be catego- rized into two subclasses: i) methods that exploit inter-sentence features and ii) methods based on sentence encoders. In this work, we focus on the latter where sentences1 are separately encoded us- ing a shared encoder and then fed to a classifier network, due to its efficiency and general applica- bility across tasks. Meanwhile, despite the success of deep neu- ral networks in natural language processing, the fact that they require abundant training data might be problematic, as constructing labeled data is a time-consuming and labor-intensive process. To mitigate the data scarcity problem, several semi- supervised learning paradigms, that take advan- tage of unlabeled data when only some of the data examples are labeled (Chapelle et al., 2010), are proposed. These unlabeled data are much easier to collect, thus utilizing them could be a good option; for example in text …

Content truncated to 500 words