Inducing Multilingual Text Analysis Tools Using Bidirectional Recurrent Neural Networks

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 450–460, Osaka, Japan, December 11-17 2016. Inducing Multilingual Text Analysis Tools Using Bidirectional Recurrent Neural Networks Othman Zennaki CEA, LIST, LVIC Gif-sur-Yvette, France othman.zennaki@cea.fr Nasredine Semmar CEA, LIST, LVIC Gif-sur-Yvette, France nasredine.semmar@cea.fr Laurent Besacier LIG, Univ. Grenoble-Alpes Grenoble, France laurent.besacier@imag.fr Abstract This work focuses on the rapid development of linguistic annotation tools for resource-poor languages. We experiment several cross-lingual annotation projection methods using Recurrent Neural Networks (RNN) models. The distinctive feature of our approach is that our multilin- gual word representation requires only a parallel corpus between source and target languages. More precisely, our method has the following characteristics: (a) it does not use word alignment information, (b) it does not assume any knowledge about foreign languages, which makes it ap- plicable to a wide range of resource-poor languages, (c) it provides truly multilingual taggers. We investigate both uni- and bi-directional RNN models and propose a method to include ex- ternal information (for instance low level information from Part-Of-Speech tags) in the RNN to train higher level taggers (for instance, super sense taggers). We demonstrate the validity and genericity of our model by using parallel corpora (obtained by manual or automatic translation). Our experiments are conducted to induce cross-lingual POS and super sense taggers. 1 Introduction In order to minimize the need for annotated resources (produced through manual annotation, or by man- ual check of automatic annotation), several research works were interested in building Natural Language Processing (NLP) tools based on unsupervised or semi-supervised approaches (Collins and Singer, 1999; Klein, 2005; Goldberg, 2010). For example, NLP tools based on cross-language projection of linguistic annotations achieved good performances in the early 2000s (Yarowsky et al., 2001). The key idea of annotation projection can be summarized as follows: through word alignment in parallel text corpora, the annotations are transferred from the source (resource-rich) language to the target (under-resourced) language, and the resulting annotations are used for supervised training in the target language. However, automatic word alignment errors (Fraser and Marcu, 2007) limit the performance of these approaches. Our work is built upon these previous contributions and observations. We explore the possibility of using Recurrent Neural Networks (RNN) to build multilingual NLP tools for resource-poor languages analysis. The major difference with previous works is that we do not explicitly use word alignment information. Our only assumption is that parallel sentences (source-target) are available and that the source part is annotated. In other words, we try to infer annotations in the target language from sentence- based alignments only. While most NLP researches on RNN have focused on monolingual tasks1 and sequence labeling (Collobert et al., 2011; Graves, 2012), this paper, however, considers the problem of learning multilingual NLP tools using RNN. Contributions In this paper, we investigate the effectiveness of RNN architectures — Simple RNN (SRNN) and Bidirectional RNN (BRNN) — for multilingual sequence labeling tasks without using any word alignment information. Two NLP tasks are considered: Part-Of-Speech (POS) tagging …

Content truncated to 500 words