Cross-Lingual Text Filtering Based on Text Concepts and kNN

Cross-Lingual Text Filtering Based On Text Concepts And kNN Li Shaozi I. School of Computer, National University of Defence Technology Changsha, P. R. China 2. Department of Computer Science, Xiamen University Xiamen, P. R. China szlig@xmu.edu.cn Su Weifeng Department of Computer Science, Hong kong University of Science and Technology Kowloon, Hong Kong weifeng@ust.hk Li Tangqiu Department of Computer Science, Xiamen University Xiamen, P. R. China tqli@xmu.edu.cn Chen Huowang School of Computer, National University of Defence Technology Changsha, R R. China chenhuowang@nud t.edu.cn Abstract This paper presents the model that can be used to filter the texts which the user is interested in from a large scale of source texts in Chinese or in English. Each text which the user is interested in can be represented as a vector in the vector space of classifiable sememes. The text to be sifted is represented as a vector too. The relevance of the text to the user can be measured by using the cosine angle between the text and its k nearest neighbor in the vector space. Experiments have been done and their results show that this scheme yields good results . 1 Introduction With the rapid growth of the Internet and other networked information, there is an increasing need for reliable automatic texts filtering. As the Internet is boundless, Cross-Lingual Information Filtering(CLIF) system is also eager to access information which may be important to the user. Information filtering includes three subtasks: collecting information from information sources, selecting information which may interest the user and presenting the information to the user. Figure 1 depicts this subdivision.[5] collection� V filtering� V represencaLion Figure 1 Information seeking task diagram In this paper, we investigate how a cross lingual texts filtering model could be structured to acquire a user profile which enables it to distinguish between relevant and irrelevant documents in texts form on the Internet. This user profile is then used to accomplish the task of filtering documents from information sources automatically in English or in Chinese. The task of texts filtering entailed: building a model of the features in the text predicting the relevance of the text to the user's interest. This asks for text representation which is discussed in section 3 and the user profile which is described in section 4. Techniques to compare between the text representation and the user profile to decide whether or not to keep the text and to notify the user are discussed in section 5. The architecture of the model is described in section 2. In section 6, report about some experiments is displayed to evaluate the value of this system. Conclusion about this model is made at the end of this paper. All techniques discussed below are suit to texts 166 both in Chinese and in English. Some differences when handling information in Chinese and English will be discussed separately. 2 Multilingual texts filtering model Figure 2 is the architecture of multilingual texts filtering model. First of all , the user provides sample texts as his …

Content truncated to 500 words