Cross-language Learning with Adversarial Neural Networks

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 226–237, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Cross-language Learning with Adversarial Neural Networks: Application to Community Question Answering Shafiq Joty, Preslav Nakov, Llu´ıs M`arquez and Israa Jaradat ALT Research Group Qatar Computing Research Institute, HBKU {sjoty, pnakov, lmarquez, ijaradat}@hbku.edu.qa Abstract We address the problem of cross-language adaptation for question-question similarity reranking in community question answer- ing, with the objective to port a system trained on one input language to another input language given labeled training data for the first language and only unlabeled data for the second language. In particular, we propose to use adversarial training of neural networks to learn high-level features that are discriminative for the main learn- ing task, and at the same time are invariant across the input languages. The evalua- tion results show sizable improvements for our cross-language adversarial neural net- work (CLANN) model over a strong non- adversarial system. 1 Introduction Developing natural language processing (NLP) sys- tems that can work indistinctly with different input languages is a challenging task; yet, such a setup is useful for many real-world applications. One ex- pensive solution is to annotate data for each input language and then to train a separate system for each one. Another option, which can be also costly, is to translate the input, e.g., using machine transla- tion (MT), and then to work monolingually in the target language (Hartrumpf et al., 2008; Lin and Kuo, 2010; Ture and Boschee, 2016). However, the machine-translated text can be of low quality, might lose some input signal, e.g., it can alter sentiment (Mohammad et al., 2016), or may not be really needed (Bouma et al., 2008; Pouran Ben Veyseh, 2016). Using a unified cross-language representa- tion of the input is a third, less costly option, which allows any combination of input languages during both training and testing. In this paper, we take this last approach, i.e., com- bining languages during both training and testing, and we study the problem of question-question sim- ilarity reranking in community Question Answer- ing (cQA), when the input question can be either in English or in Arabic, and the questions it is com- pared to are always in English. We start with a simple language-independent representation based on cross-language word embeddings, which we in- put into a feed-forward multilayer neural network to classify pairs of questions, (English, English) or (Arabic, English), regarding their similarity. Furthermore, we explore the question of whether adversarial training can be used to improve the per- formance of the network when we have some unla- beled examples in the target language. In particular, we adapt the Domain Adversarial Neural Network model from (Ganin et al., 2016), which was orig- inally used for domain adaptation, to our cross- language setting. To the best of our knowledge, this is novel for cross-language question-question similarity reranking, as well as for natural language processing (NLP) in general; moreover, we are not …

Content truncated to 500 words