A Multilingual Topic Model for Learning Weighted Topic Links Across Corpora with Low Comparability

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1243–1248, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1243 A Multilingual Topic Model for Learning Weighted Topic Links Across Corpora with Low Comparability Weiwei Yang∗ Computer Science University of Maryland wwyang@cs.umd.edu Jordan Boyd-Graber† Computer Science, iSchool, Language Science, UMIACS University of Maryland jbg@umiacs.umd.edu Philip Resnik Linguistics and UMIACS University of Maryland resnik@umd.edu Abstract Multilingual topic models (MTMs) learn top- ics on documents in multiple languages. Past models align topics across languages by im- plicitly assuming the documents in different languages are highly comparable, often a false assumption. We introduce a new model that does not rely on this assumption, particularly useful in important low-resource language sce- narios. Our MTM learns weighted topic links and connects cross-lingual topics only when the dominant words defining them are sim- ilar, outperforming LDA and previous MTMs in classification tasks using documents’ topic posteriors as features. It also learns coherent topics on documents with low comparability. 1 Introduction Topic models explain document collections at a high level (Boyd-Graber et al., 2017). Multi- lingual topic models (MTMs) uncover latent top- ics across languages and reveal commonalities and differences across languages and cultures (Ni et al., 2009; Shi et al., 2016; Guti´errez et al., 2016). Existing models extend latent Dirichlet al- location (Blei et al., 2003, LDA) and learn aligned topics across languages (Mimno et al., 2009). Prior models work well because they implicitly assume—even if not part of the model—parallel or highly comparable data with well-aligned topics. However, this assumption does not always com- port with reality. Even documents from the same place and time can discuss very different things across languages: in multicultural London, Hindi tweets focus on a Bollywood actor’s BBC appear- ance, French blogs fret about Brexit, and English articles focus on Tottenham’s lineup. Generally, corpora have a range of “nonparallelness” (Fung, 2000). In less comparable settings, while some ∗ Now at Facebook † Now at Google AI Z¨urich EN-1 EN-2 EN-3 EN-4 ZH-1 ZH-2 ZH-3 ZH-4 universities schools students research science economics dollars million invest income technology information computers smart system sports match referee tournament champion °_(technology) �–(science) ÑÇj(computer) ª-(smart) +�(system) #€(music) CÁ(album) |{(singer) Œñ(works) DaJ(concert) �Ø(sports) �‹(match) ñT(referee) V7‹(tournament) ÐË(champion) ÿ~(economics) ¾s(dollars) ®7(million) Åt(invest) f•(income) Figure 1: Topic pairs with many word translation pairs have high link weights, e.g., (EN-1, ZH-3) and (EN-2, ZH-4); topic pairs with partial overlap receive lower weights, e.g., (EN-4, ZH-1); a topic is unlinked if there is no corresponding topic in the other language (ZH-2). topics are shared, languages’ emphasis may di- verge and some topics may lack analogs. We therefore introduce a new multilingual topic model that assumes each language has its own topic sets and jointly learns all topics, but does not force one-to-one alignment across languages. In- stead, our MTM learns weighted topic links across languages and only assigns a high link weight to a topic …

Content truncated to 500 words