Using LDA to detect semantically incoherent documents

CoNLL 2008: Proceedings of the 12th Conference on Computational Natural Language Learning, pages 41–48 Manchester, August 2008 Using LDA to detect semantically incoherent documents Hemant Misra and Olivier Capp´e LTCI/CNRS and TELECOM ParisTech {misra,cappe}@enst.fr Franc¸ois Yvon Univ Paris-Sud 11 and LMISI-CNRS yvon@limsi.fr Abstract Detecting the semantic coherence of a doc- ument is a challenging task and has sev- eral applications such as in text segmenta- tion and categorization. This paper is an attempt to distinguish between a ‘semanti- cally coherent’ true document and a ‘ran- domly generated’ false document through topic detection in the framework of latent Dirichlet analysis. Based on the premise that a true document contains only a few topics and a false document is made up of many topics, it is asserted that the entropy of the topic distribution will be lower for a true document than that for a false docu- ment. This hypothesis is tested on several false document sets generated by various methods and is found to be useful for fake content detection applications. 1 Introduction The “Internet revolution” has dramatically in- creased the monetary value of higher ranking on the web search engines index, fostering the ex- pansion of techniques, collectively known as “Web Spam”, that fraudulently help to do so. Internet is indeed “polluted” with fake Web sites whose only purpose is to deceive the search engines by arti- ficially pushing up the popularity of commercial sites, or sites promoting illegal content 1. These fake sites are often forged using very crude content generation techniques, ranging from web scrap- ping (blending of chunks of actual contents) to simple-minded text generation techniques based c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. 1The annual AirWeb challenge http://airweb. cse.lehigh.edu gives a state-of-the art on current Web Spam detection techniques. on random sampling of words (“word salads”), or randomly replacing words in actual documents (“word stuffing”) 2. Among these, the latter two are easy to detect using simple statistical models of natural texts, but the former is more challeng- ing, it being made up of actual sentences: recog- nizing these texts as forged requires either to resort to plagiarism detection techniques, or to automati- cally identify their lack of semantic consistency. Detecting the consistency of texts or of text chunks has many applications in Natural Language Processing. So far, it has been used mainly in the context of automatic text segmentation, where a change in vocabulary is often the mark of topic change (Hearst, 1997), and, to a lesser extent, in discourse studies (see, e.g., (Foltz et al., 1998)). It could also serve to devise automatic metrics for text summarization or machine translation tasks. This paper is an attempt to address the issue of differentiating between ‘true’ and ‘false’ doc- uments on the basis of their consistency through topic modeling approach. We have used La- tent Dirichlet allocation (LDA) (Blei et al., 2002) model as our main topic modeling tool. One of the aims of LDA and …

Content truncated to 500 words