TAP-DLND 1.0 : A Corpus for Document Level Novelty Detection

TAP-DLND 1.0 : A Corpus for Document Level Novelty Detection Tirthankar Ghosal, Amitra Salam, Swati Tiwari, Asif Ekbal, Pushpak Bhattacharyya Indian Institute of Technology Patna Bihta, Bihar-801106, India (tirthankar.pcs16, salam.recs, stlex.re, asif, pb)@iitp.ac.in Abstract Detecting novelty of an entire document is an Artificial Intelligence (AI) frontier problem. This has immense importance in widespread Natural Language Processing (NLP) applications ranging from extractive text document summarization to tracking development of news events to predicting impact of scholarly articles. Although a very relevant problem in the present context of exponential data duplication, we are unaware of any document level dataset that correctly addresses the evaluation of automatic novelty detection techniques in a classification framework. To bridge this relative gap, here in this work, we present a resource for benchmarking the techniques for document level novelty detection. We create the resource via topic-specific crawling of news documents across several domains in a periodic manner. We release the annotated corpus with necessary statistics and show its use with a developed system for the problem in concern. Keywords: Document level novelty detection, Classification, Web crawling, Corpus 1. Introduction Novelty detection implies finding elements that have not appeared before, or new, or original with respect to rele- vant references. The explosive growth of documents across the web has resulted in the accumulation of redundant ones, thereby consuming space as well as precious time of read- ers seeking new information. This necessitates finding means for discarding redundant document(s) and retaining ones containing novel information. The level of informa- tion duplication is not just limited to the lexical surface form of texts but has encroached the barriers of semantics and pragmatics too. Paraphrasing, semantic level plagia- rism etc. are instances of such practices. Intelligent text reuse, synonym replacement and careful alignment may lead to a surface form which is very different from the originating source yet convey the same meaning. Present state-of-the-art text matching techniques are unable to pro- cess such redundancy. The quest of new information is an eternal human need and urges attention in this very age of exploratory data redundancy. One major objective of this work is to provide a benchmark setup for experiments to filter out superfluous information across the web. With this work we introduce a simplistic dataset to the research com- munity to inculcate efficient methods to detect document level novelty or on the contrary document level redundancy. We create the resource by crawling news articles of dif- ferent categories by various agencies and coin it as TAP- DLND 1.01 (after the initial names of the principal investi- gators Tirthankar-Asif-Pushpak) which also stands for Ex- plore Document Level Novelty Detection (DLND). In this work we view the problem of novelty detection as a two- class classification problem with the judgment that whether an incoming document bears sufficiently new information to be labeled as novel with respect to a set of source docu- ments. The source document set could be seen as the mem- ory of the reader which stores known information. We ex- …

Content truncated to 500 words