An Improved Corpus Comparison Approach to Domain Specific Term Recognition

An Improved Corpus Comparison Approach to Domain Specific Term Recognition* Xiaoyue Liu, and Chunyu Kit Department of Chinese, Translation and Linguistics City University of Hong Kong Tat Chee Ave., Kowloon, Hong Kong {xyliu0, ctckit}@cityu.edu.hk Abstract. Domain specific terms are words carrying special conceptual meanings in a sub- ject field. Automatic term recognition plays an important role in many natural language processing and knowledge engineering applications such as information retrieval and knowledge mining. This paper explores a novel approach to automatic term extraction based on the basic ideas of corpus comparison and emerging pattern with significant elabo- ration. It measures the termhood of a term candidate in terms of its peculiarity to a given domain via comparison to several background domains. Our experiments confirm its out- performance against other approaches, achieving an average precision of 83% on the top 10% candidates in terms of their termhood. Keywords: Automatic term recognition, corpus comparison, emerging pattern 1. Introduction Automatic term recognition (ATR) plays an important role in many natural language processing applications, e.g., information retrieval (IR) (Chowdhury, 1999; Zhou and Nie, 2005), informa- tion extraction (Yangarber et al., 2000), domain specific lexicon construction (Hull, 2001), and topic extraction (Lin, 2004). Its technological advancements can facilitate all these applications for performance enhancement. Despite the large volume of literature on ATR, further significant success in the field still re- lies heavily on a sound resolution of two basic issues, namely, the unithood and termhood of a term candidate, as identified in Kageura and Umino (1996). The former quantifies the unity of a candidate (especially, a multi-word candidate), indicating how likely a candidate is to be an atomic linguistic unit. It works more like a filter to exclude non-atomic (thus unqualified) term candidates but has little authority to determine which atomic language unit is a true term. The latter measures how likely a qualified candidate is to be a true term in a subject field. It plays a decisive role in licensing a term. Regardless of the previous progress in term extraction, termhood measurement remains the most critical problem to be solved. Novel technologies and methodologies are needed in order to bring up new insights into our understanding of this problem. This paper is intended to pre- * The research described in this paper was supported by the Research Grants Council of HKSAR, China, through the CERG grant 9040861 (CityU 1318/03H) and by the City University of Hong Kong through the Strategic Research Grants 7002037 and 7001879. Correspondence concerning this research should be addressed to Dr. Chunyu Kit. Copyright 2008 by Xiaoyue Liu, and Chunyu Kit 253 22nd Pacific Asia Conference on Language, Information and Computation, pages 253–261 sent a novel statistical approach to domain specific term extraction from a collection of thematic documents following the basic ideas of corpus comparison and emerging pattern. It measures the termhood of a term candidate in a subject domain in terms of its peculiarity to this subject domain via comparison to several background domains. In order to avoid unexpected …

Content truncated to 500 words