Untangling Text Data Mining

Untangling Text Data Mining Marti A. Hearst School of Information Management & Systems University of California, Berkeley 102 South Hall Berkeley, CA 94720-4600 h ttp ://www. sims. berkeley, edu/-hearst Abstract The possibilities for data mining from large text collections are virtually untapped. Text ex- presses a vast, rich range of information, but en- codes this information in a form that is difficult to decipher automatically. Perhaps for this rea- son, there has been little work in text data min- ing to date, and most people who have talked about it have either conflated it with informa- tion access or have not made use of text directly to discover heretofore unknown information. In this paper I will first define data mining, information access, and corpus-based computa- tional linguistics, and then discuss the relation- ship of these to text data mining. The intent behind these contrasts is to draw attention to exciting new kinds of problems for computa- tional linguists. I describe examples of what I consider to be reM text data mining efforts and briefly outline recent ideas about how to pursue exploratory data analysis over text. 1 Introduction The nascent field of text data mining (TDM) has the peculiar distinction of having a name and a fair amount of hype but as yet almost no practitioners. I suspect this has happened because people assume TDM is a natural ex- tension of the slightly less nascent field of data mining (DM), also known as knowledge dis- covery in databases (Fayyad and Uthurusamy, 1999), and information archeology (Brachman et al., 1993). Additionally, there are some disagreements about what actually constitutes data mining. It turns out that "mining" is not a very good metaphor for what people in the field actually do. Mining implies extracting precious nuggets of ore from otherwise worthless rock. If data mining really followed this metaphor, it would mean that people were discovering new factoids within their inventory databases. How- ever, in practice this is not really the case. Instead, data mining applications tend to be (semi)automated discovery of trends and pat- terns across very large datasets, usually for the purposes of decision making (Fayyad and Uthu- rusamy, 1999; Fayyad, 1997). Part of what I wish to argue here is that in the case of text, it can be interesting to take the mining-for- nuggets metaphor seriously. The various contrasts discussed below are summarized in Table 1. 2 TDM vs. Information Access It is important to differentiate between text data mining and information access (or infor- mation retrieval, as it is more widely known). The goal of information access is to help users find documents that satisfy their information needs (Baeza-Yates and Ribeiro-Neto, 1999). The standard procedure is akin to looking for needles in a needlestack - the problem isn't so much that the desired information is not known, but rather that the desired information coex- ists with many other valid pieces of information. Just because a user is currently interested in NAFTA and not Furbies does not mean …

Content truncated to 500 words