Comparing Apples to Apple: The Effects of Stemmers on Topic Models

Comparing Apples to Apple: The Effects of Stemmers on Topic Models Alexandra Schofield Cornell University Ithaca, NY 14853 xanda@cs.cornell.edu David Mimno Cornell University Ithaca, NY 14853 mimno@cornell.edu Abstract Rule-based stemmers such as the Porter stem- mer are frequently used to preprocess English corpora for topic modeling. In this work, we train and evaluate topic models on a variety of corpora using several different stemming algo- rithms. We examine several different quantita- tive measures of the resulting models, includ- ing likelihood, coherence, model stability, and entropy. Despite their frequent use in topic modeling, we find that stemmers produce no meaningful improvement in likelihood and co- herence and in fact can degrade topic stability. 1 Introduction Stemming is a popular way to reduce the size of a vocabulary in natural language tasks by conflating words with related meanings. Specifically, stem- ming aims to convert words with the same “stem” or root (e.g “creative” and “creator”) to a single word type (“create”). Though originally developed in the context of information retrieval (IR) systems, stem- mers are now commonly used as a preprocessing step in unsupervised machine learning tasks. It this work we consider one such application, topic model- ing. Although stemmers are commonly used in topic models (Liu et al., 2010; Lo et al., 2015; Nan et al., 2015; Kamath S et al., 2015; Su, 2015; Jacobi et al., 2016), we find no empirical benefits for the practice. One could conjecture several reasons to stem for semantic models. First, conflating semanti- cally related words into one word type could im- prove model fit by intelligently reducing the space of possible models. Given that reducing the fea- ture space randomly is already known to be poten- tially beneficial (Ganchev and Dredze, 2008), do- ing so in a semantically-inspired way might be even better. Second, stemmers could reduce the effect of small morphological differences on the stability of a learned model. Reducing the words “happy”, “happily”, and “happier” to one token may result in fewer possible models with divergent “happy” top- ics. Third, stemmers approximate intuitive word equivalence classes, so language models based on stemmed corpora inherit that semantic similarity, which may improve interpretability as perceived by human evaluators. However, stemmers have the potential to be con- fusing, unreliable, and possibly even harmful in lan- guage models. First, many stemmers produce terms that are not recognizable English words and may be difficult to map back to a valid original word, such as “stai” as the Porter stem of “stay”. Sec- ond, although stemming aids document retrieval for many languages, English is a notorious exception (Harman, 1991). In English, the complexity of compound affixes with meaning can lead to over- stemming, such as “recondition,” a word sharing a stem but not a root meaning with “recondite.” These complexities can also lead to the incorrect conflation of words with the same root but divergent meaning such as “absolutely” and “absolution”. Third, and most troubling, there are cases in which morpholog- ical variants of the same stem carry …

Content truncated to 500 words