Extracting Domain-Specific Words - A Statistical Approach

An Unsupervised Approach to Domain-Specific Term Extraction Su Nam Kim♠, Timothy Baldwin♥ ♠ ♥ CSSE ♥ NICTA VRL University of Melbourne VIC 3010 Australia sunamkim@gmail.com, tb@ldwin.net Min-Yen Kan♣ ♣ School of Computing National University of Singapore kanmy@comp.nus.edu.sg Abstract Domain-specific terms provide vital se- mantic information for many natural lan- guage processing (NLP) tasks and appli- cations, but remain a largely untapped re- source in the field. In this paper, we pro- pose an unsupervised method to extract domain-specific terms from the Reuters document collection using term frequency and inverse document frequency. 1 Introduction Automatic domain-specific term extraction is a categorization/classification task where terms are categorized into a set of predefined domains. It has been employed in tasks such as keyphrase extraction (Frank et al., 1999; Witten et al., 1999), word sense disambiguation (Magnini et al., 2002), and query expansion and cross-lingual text categorization (Rigutini et al., 2005). Even though the approach shows promise, relatively lit- tle research has been carried out to study its ef- fects in detail (Drouin, 2004; Milne et al., 2006; Rigutini et al., 2006; Kida et al., 2007; Park et al., 2008). Most of the research to date on domain- specific term extraction has employed supervised machine learning, within the fields of term cat- egorization and text mining. However, to date, the only research to approach the task in an un- supervised manner is that of Park et al. (2008). Unsupervised methods have the obvious advan- tage that they circumvent the need for laborious manual classification of training instances, and are thus readily applicable to arbitrary sets of do- mains, tasks and languages. In this paper, we present a novel unsupervised method for automatically extracting domain- specific terms, targeted specifically at building domain-specific lexicons for natural language processing (NLP) purposes. One of the main properties utilized in this work is domain speci- ficity. Our notion of domain specificity is based on statistical analysis of word usage, and adopts the simple notions of term frequency (TF) and in- verse document frequency (IDF) over domains to capture their domain specificity. 2 Unsupervised Domain-Specific Term Extraction In this section, we elaborate on our proposed method, as well as the benchmark method of Park et al. (2008). 2.1 Proposed Method (D1) Our proposed unsupervised method is based on TF-IDF. The basic underlying idea is that domain-specific terms occur in a particular do- main with markedly higher frequency than they do in other domains, similar to term frequency patterns captured by TF-IDF. Hence, we compute TF-IDF from TFij, the term frequency of term i from documents in do- main j, and IDFi, the inverse domain frequency. The calculation of TFij is via: TFij = ∑nij k nkj (1) where nij is the number of occurrences of term i in the documents associated with domain j. IDFi is calculated via: IDFi = log( |D| |{d : ti ∈ d}|) (2) where ti is the term, and D is the set of all do- mains. The final TF-IDFij value of a given term …

Content truncated to 500 words