Learning Multilingual Topics from Incomparable Corpora

Proceedings of the 27th International Conference on Computational Linguistics, pages 2595–2609 Santa Fe, New Mexico, USA, August 20-26, 2018. 2595 Learning Multilingual Topics from Incomparable Corpora Shudong Hao Computer Science University of Colorado Boulder, CO, USA shudong@colorado.edu Michael J. Paul Information Science University of Colorado Boulder, CO, USA mpaul@colorado.edu Abstract Multilingual topic models enable crosslingual tasks by extracting consistent topics from multi- lingual corpora. Most models require parallel or comparable training corpora, which limits their ability to generalize. In this paper, we first demystify the knowledge transfer mechanism behind multilingual topic models by defining an alternative but equivalent formulation. Based on this analysis, we then relax the assumption of training data required by most existing models, creating a model that only requires a dictionary for training. Experiments show that our new method ef- fectively learns coherent multilingual topics from partially and fully incomparable corpora with limited amounts of dictionary resources. 1 Introduction Multilingual topic models provide an overview of document structures in multilingual corpora, by learn- ing language-specific versions of each topic (Figure 1). Their simplicity, efficiency and interpretability make models from this family popular for various crosslingual tasks, e.g., feature extraction (Liu et al., 2015), cultural difference discovery (Shutova et al., 2017; Guti´errez et al., 2016), translation detec- tion (Krstovski et al., 2016; Krstovski and Smith, 2016), and others (Barrett et al., 2016; Agi´c et al., 2016; Hintz and Biemann, 2016). Typical probabilistic multilingual topic models are based on Latent Dirichlet Allocation (LDA, Blei et al. (2003)), adding supervision on connections between languages. Most models achieve this by making strong assumptions on the training data—they either require a parallel corpus that has sentence-aligned documents in different languages (e.g., EuroParl, Koehn (2005)), or a comparable corpus that has docu- ments of similar content (e.g., Wikipedia articles paired across languages). These training requirements limit the usage of such models: an adequately large parallel corpus is difficult to obtain, particularly for low-resource languages. For example, only 300 languages are available on Wikipedia,1 and only 250 languages have more than 1,000 articles. Another common choice for parallel corpus in multilingual research, the Bible, is available in 2,530 languages (Agi´c et al., 2015).2 However, studies show that its archaic themes and small corpus size (1,189 chapters) can limit performance (Hao et al., 2018; Moritz and B¨uchler, 2017). Therefore, the requirement of parallel/comparable corpora for multilingual topic models limits their usage in many situations. Another line of research focuses on using multilingual dictionaries as supervision (Ma and Nasukawa, 2017; Guti´errez et al., 2016; Liu et al., 2015; Jagarlamudi and Daum´e III, 2010; Boyd-Graber and Blei, 2009). In contrast to parallel corpora, dictionaries are widely available and often easy to obtain. PANLEX, a free online dictionary database, for example, covers 5,700 languages and more than one billion dictionary entries (Kamholz et al., 2014; Baldwin et al., 2010).3 Thus, a multilingual topic model built on a dictionary rather than a parallel corpus is potentially applicable to more languages. This work is licensed under a Creative Commons Attribution …

Content truncated to 500 words