Domain-Specific New Words Detection in Chinese

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 44–53, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Domain-Specific New Words Detection in Chinese Ao Chen1, Maosong Sun1 1Department of Computer Science and Technology, State Key Lab on Intelligent Technology and Systems, National Lab for Information Science and Technology, Tsinghua University, China chenao3220@gmail.com, sms@mail.tsinghua.edu.cn Abstract With the explosive growth of Internet, more and more domain-specific environ- ments appear, such as forums, blogs, MOOCs and etc. Domain-specific words appear in these areas and always play a critical role in the domain-specific NLP tasks. This paper aims at extracting Chi- nese domain-specific new words automat- ically. The extraction of domain-specific new words has two parts including both new words in this domain and the espe- cially important words. In this work, we propose a joint statistical model to perform these two works simultaneously. Com- pared to traditional new words detection models, our model doesn't need handcraft features which are labor intensive. Exper- imental results demonstrate that our joint model achieves a better performance com- pared with the state-of-the-art methods. 1 Introduction Accompanying with the development of Inter- net, many new specific domains appear, such as forums, blogs, Massive Open Online Courses (MOOCs) and etc. There are always a group of important words in these domains, which are known as domain-specific words. Domain- specific words include two types as shown in Ta- ble 1. The first ones are rare and unambiguous words which will seldom appear in other domains such as “栈顶”(stack top) and “二叉树”(binary tree). These words may cause word segmentation problems. For example, if we do not recognize “栈顶”(stack top) as a word, the segmentation “栈 顶 运算符 是 乘号”(the operator at stack top is multiplication sign) will be like “栈 顶运 算符 是 乘号”. In this case, “栈顶” means “stack top” Domain words Translation Type 栈顶 stack top 1 二叉树 binary tree 1 复杂度 complexity 2 遍历 iterate 2 Table 1: Examples of domain-specific word in data structure domain and “运算符” means “operator”, but in the seg- mentation result, “顶运” is segmented into a word in mistake and will bring lots of problems to the further applications. The other type is common and ambiguous words which have specific new meanings in this domain, such as “复杂度”(complexity) and “遍 历”(iterate). These words often play important roles in domain-specific tasks. For example，in MOOCs which are typical domain-specific envi- ronments, there is an Automated Navigation Sug- gestion(ANS)(Zhang et al., 2017) task which sug- gests a time point for users when they want to review the front contents of the video. With the help of the recognition of this type of words, we can easily give higher weights to those domain- specific contents. After extracting these two type of words, we can also use them for creating ontologies, term lists, and in the Semantic Web Area for find- ing novel entities(F¨arber et al., 2016). Besides, in MOOCs area it will also benefit Certification Prediction(CP)(Coleman et al., 2015) …

Content truncated to 500 words