Many Languages, One Parser

Many Languages, One Parser Waleed Ammar♦ George Mulcaire♥ Miguel Ballesteros♠♦ Chris Dyer♦ Noah A. Smith♥ ♦School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA ♥Computer Science & Engineering, University of Washington, Seattle, WA, USA ♠NLP Group, Pompeu Fabra University, Barcelona, Spain wammar@cs.cmu.edu, gmulc@uw.edu, miguel.ballesteros@upf.edu cdyer@cs.cmu.edu, nasmith@cs.washington.edu Abstract We train one multilingual model for depen- dency parsing and use it to parse sentences in several languages. The parsing model uses (i) multilingual word clusters and em- beddings; (ii) token-level language informa- tion; and (iii) language-specific features (fine- grained POS tags). This input representation enables the parser not only to parse effec- tively in multiple languages, but also to gener- alize across languages based on linguistic uni- versals and typological similarities, making it more effective to learn from limited annota- tions. Our parser’s performance compares fa- vorably to strong baselines in a range of data scenarios, including when the target language has a large treebank, a small treebank, or no treebank for training. 1 Introduction Developing tools for processing many languages has long been an important goal in NLP (Rösner, 1988; Heid and Raab, 1989),1 but it was only when statistical methods became standard that massively multilingual NLP became economical. The main- stream approach for multilingual NLP is to design language-specific models. For each language of in- terest, the resources necessary for training the model are obtained (or created), and separate parameters are fit for each language separately. This approach is simple and grants the flexibility of customizing 1As of 2007, the total number of native speakers of the hundred most popular languages only accounts for 85% of the world’s population (Wikipedia, 2016). the model and features to the needs of each lan- guage, but it is suboptimal for theoretical and prac- tical reasons. Theoretically, the study of linguistic typology tells us that many languages share mor- phological, phonological, and syntactic phenomena (Bender, 2011); therefore, the mainstream approach misses an opportunity to exploit relevant supervi- sion from typologically related languages. Practi- cally, it is inconvenient to deploy or distribute NLP tools that are customized for many different lan- guages because, for each language of interest, we need to configure, train, tune, monitor, and occasion- ally update the model. Furthermore, code-switching or code-mixing (mixing more than one language in the same discourse), which is pervasive in some gen- res, in particular social media, presents a challenge for monolingually-trained NLP models (Barman et al., 2014).2 In parsing, the availability of homogeneous syn- tactic dependency annotations in many languages (McDonald et al., 2013; Nivre et al., 2015b; Agi´c et al., 2015; Nivre et al., 2015a) has created an opportunity to develop a parser that is capable of parsing sentences in multiple languages, address- ing these theoretical and practical concerns.3 multilingual parser can potentially replace an arrayA of language-specific monolingually-trained parsers 2While our parser can be used to parse input with code- switching, we have not evaluated this capability due to the lack of appropriate data. 3Although multilingual dependency treebanks have been available for a …

Content truncated to 500 words