Coordinated Topic Modeling

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9831 - 9843 December 7-11, 2022 ©2022 Association for Computational Linguistics Coordinated Topic Modeling Pritom Saha Akash Jie Huang Kevin Chen-Chuan Chang University of Illinois at Urbana-Champaign, USA {pakash2, jeffhj, kcchang}@illinois.edu Abstract We propose a new problem called coordinated topic modeling that imitates human behavior while describing a text corpus. It considers a set of well-defined topics like the axes of a se- mantic space with a reference representation. It then uses the axes to model a corpus for easily understandable representation. This new task helps represent a corpus more interpretably by reusing existing knowledge and benefits the cor- pora comparison task. We design ECTM, an embedding-based coordinated topic model that effectively uses the reference representation to capture the target corpus-specific aspects while maintaining each topic’s global semantics. In ECTM, we introduce the topic- and document- level supervision with a self-training mecha- nism to solve the problem. Finally, extensive experiments on multiple domains show the su- periority of our model over other baselines.1 1 Introduction We often ask questions about well-defined topics when we read articles (e.g., news/research articles). E.g., in news domain, such a question can be like “what is in entertainment today? (it’s about os- car 2022)” or in academic domain, it can be like “what is trending in machine learning in 2016? (it’s about deep learning)”. This is a practical human trait while understanding information in a text cor- pus. Rather than finding arbitrary topics, people often want to explore the text based on some well- defined topics. E.g., in news domain, such topics are business, politics, sports, etc. A well-defined topic is not merely a name (i.e., surface name); it generally also has a representa- tion (e.g., word distribution) that can be obtained from a large text corpus, which we may call the reference representation. E.g., in such representa- tion of the topic sports, words like play, game, and 1Code and data are available at https://github.com/ pritomsaha/Coordinated-Topic-Modeling Figure 1: Coordinated interpretation of Topic model player are mainly dominated. Similarly, for poli- tics, these are vote, election, party, etc. However, a topic’s representation generally deviates from its reference to other corpora. E.g., while describing a news corpus specific to the USA, it is likely to use words such as nfl, baseball or football to represent sports, while words like cricket, ipl or run are more relevant for India. The above scenario can be explained by the con- cept of the coordinate system in geometry. Specifi- cally, we can consider a set of well-defined topics as the axes or basis of semantic space with their reference representation βr. E.g., in Figure 1, two topics C1:sports and C2:politics represents two axes (i.e., reference axes) defined by their reference rep- resentations βr 1 and βr 2 respectively. Defining a target corpus by these two topics is analogous to finding a representation in the space defined by the reference axes. Figure 1 shows such target representation βUSA and …

Content truncated to 500 words