Can Topic Modelling benefit from Word Sense Information?

Can Topic Modelling benefit from Word Sense Information? Adriana Ferrugento1, Hugo Gonc¸alo Oliveira1, Ana Oliveira Alves1,2, Filipe Rodrigues1 1CISUC, Department of Informatics Engineering University of Coimbra, Portugal 2IPC, Polytechnic Institute of Coimbra, Portugal aferr@student.dei.uc.pt, {hroliv,ana,fmpr}@dei.uc.pt Abstract This paper proposes a new topic model that exploits word sense information in order to discover less redundant and more informative topics. Word sense information is obtained from WordNet and the discovered topics are groups of synsets, instead of mere surface words. A key feature is that all the known senses of a word are considered, with their probabilities. Alternative configurations of the model are described and compared to each other and to LDA, the most popular topic model. However, the obtained results suggest that there are no benefits of enriching LDA with word sense information. Keywords: topic model, word senses, WordNet, semantics, SemLDA 1. Introduction Topic models uncover the main subjects addressed in the documents of a corpus by inferring their themes or topics, which in turn correspond to probability distributions over words. Topic models are useful for improving traditional browsing and indexing, summarisation of large quantities of text, and text classification, among others. Classic topic modelling algorithms, such as the popular LDA (Blei et al., 2003), rely on the co-occurrences of surface words to cap- ture their semantic proximity. One limitation of existing topic models is that they do not consider semantic knowledge on the words, including their possible senses. Namely, they consider a surface word to be identical in different contexts and leverage on its co- occurrences with other words to differentiate topics. This may, for instance, result in topics with synonyms, both re- dundant and less informative. Our goal was to develop SemLDA, a LDA-based topic model that would be sensitive to word senses. To min- imise the aforementioned limitations, SemLDA considers not only the context where each word occurs, but also infor- mation on its possible senses, obtained, for example, from the lexical-semantic knowledge-base WordNet (Fellbaum, 1998). Similarly to other semantic topic models, instead of sets of surface words, the topics produced by SemLDA are based on word senses, inside their WordNet synsets. The main difference is that SemLDA considers all the possible senses of the words in a document, together with their prob- abilities. Moreover, it only requires a small intuitive change to the classic and popular LDA algorithm. SemLDA was originally introduced in Ferrugento et al. (2015). This paper describes and performs an in-depth analysis of alternative configurations, involving different approaches for computing the probabilities of a word in a synset, which is the key step towards introducing word sense information in topic models and towards making SemLDA work in practice. Implemented configurations are compared among each other and with the classic LDA, first based on word association measures and then on a clas- sification task. Despite resulting in more informative topic models, improvements over the classic LDA are not as clear as we would expect. In fact, the results obtained show that most versions of SemLDA …

Content truncated to 500 words