Employing Topic Models for Pattern-based Semantic Class Discovery

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 459–467, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Employing Topic Models for Pattern-based Semantic Class Discovery Huibin Zhang1* Mingjie Zhu2* Shuming Shi3 Ji-Rong Wen3 1Nankai University 2University of Science and Technology of China 3Microsoft Research Asia {v-huibzh, v-mingjz, shumings, jrwen}@microsoft.com Abstract A semantic class is a collection of items (words or phrases) which have semantically peer or sibling relationship. This paper studies the employment of topic models to automati- cally construct semantic classes, taking as the source data a collection of raw semantic classes (RASCs), which were extracted by ap- plying predefined patterns to web pages. The primary requirement (and challenge) here is dealing with multi-membership: An item may belong to multiple semantic classes; and we need to discover as many as possible the dif- ferent semantic classes the item belongs to. To adopt topic models, we treat RASCs as “doc- uments”, items as “words”, and the final se- mantic classes as “topics”. Appropriate preprocessing and postprocessing are per- formed to improve results quality, to reduce computation cost, and to tackle the fixed-k constraint of a typical topic model. Experi- ments conducted on 40 million web pages show that our approach could yield better re- sults than alternative approaches. 1 Introduction Semantic class construction (Lin and Pantel, 2001; Pantel and Lin, 2002; Pasca, 2004; Shinza- to and Torisawa, 2005; Ohshima et al., 2006) tries to discover the peer or sibling relationship among terms or phrases by organizing them into semantic classes. For example, {red, white, black…} is a semantic class consisting of color instances. A popular way for semantic class dis- covery is pattern-based approach, where prede- fined patterns (Table 1) are applied to a  This work was performed when the authors were interns at Microsoft Research Asia collection of web pages or an online web search engine to produce some raw semantic classes (abbreviated as RASCs, Table 2). RASCs cannot be treated as the ultimate semantic classes, be- cause they are typically noisy and incomplete, as shown in Table 2. In addition, the information of one real semantic class may be distributed in lots of RASCs (R2 and R3 in Table 2). Type Pattern SENT NP {, NP}*{,} (and|or) {other} NP TAG <UL> <LI>item</LI> … <LI>item</LI> </UL> TAG <SELECT> <OPTION>item…<OPTION>item </SELECT> * SENT: Sentence structure patterns; TAG: HTML Tag patterns Table 1. Sample patterns R1: {gold, silver, copper, coal, iron, uranium} R2: {red, yellow, color, gold, silver, copper} R3: {red, green, blue, yellow} R4: {HTML, Text, PDF, MS Word, Any file type} R5: {Today, Tomorrow, Wednesday, Thursday, Friday, Saturday, Sunday} R6: {Bush, Iraq, Photos, USA, War} Table 2. Sample raw semantic classes (RASCs) This paper aims to discover high-quality se- mantic classes from a large collection of noisy RASCs. The primary requirement (and chal- lenge) here is to deal with multi-membership, i.e., one item may belong to multiple different seman- tic classes. For example, the term “Lincoln” can simultaneously …

Content truncated to 500 words