Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 310–320 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1029 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 310–320 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1029 Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction Chunting Zhou, Graham Neubig Language Technologies Institute Carnegie Mellon University ctzhou,gneubig@cs.cmu.edu Abstract Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata speci- fied by a set of labels. In this paper we propose multi-space variational encoder- decoders, a new model for labeled se- quence transduction with semi-supervised learning. The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. Experiments show that our model provides not only a powerful supervised framework but also can effectively take advantage of the un- labeled data. On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of- art results by a large margin for the major- ity of languages.1 1 Introduction This paper proposes a model for labeled sequence transduction tasks, tasks where we are given an input sequence and a set of labels, from which we are expected to generate an output sequence that reflects the content of the input sequence and desiderata specified by the labels. Several exam- ples of these tasks exist in prior work: using labels to moderate politeness in machine translation re- sults (Sennrich et al., 2016), modifying the output language of a machine translation system (John- son et al., 2016), or controlling the length of a summary in summarization (Kikuchi et al., 2016). In particular, however, we are motivated by the task of morphological reinflection (Cotterell et al., 1An implementation of our model are avail- able at https://github.com/violet-zct/ MSVED-morph-reinflection. playing played POS=Verb, Tense=Past Model plays Supervised Learning Semi-Supervised Learning Figure 1: Standard supervised labeled sequence transduction, and our proposed semi-supervised method. 2016), which we will use as an example in our de- scription and test bed for our models. In morphologically rich languages, different af- fixes (i.e. prefixes, infixes, suffixes) can be com- bined with the lemma to reflect various syntac- tic and semantic features of a word. The ability to accurately analyze and generate morphologi- cal forms is crucial to creating applications such as machine translation (Chahuneau et al., 2013; Toutanova et al., 2008) or information retrieval (Darwish and Oard, 2007) in these languages. As shown in 1, re-inflection of an inflected form given the target linguistic labels is a challenging sub- task of handling morphology as a whole, in which we take as input an inflected form (in the exam- ple, “playing”) and labels representing the desired form (“pos=Verb, tense=Past”) and must gen- erate the desired form (“played”). Approaches to this task include those utilizing hand-crafted linguistic rules and heuristics (Taji et al., 2016), as well as learning-based approaches using alignment and extracted transduction rules …

Content truncated to 500 words