Improved Text Categorisation for Wikipedia Named Entities

Improved Text Categorisation for Wikipedia Named Entities Sam Tardif and James R. Curran and Tara Murphy School of Information Technologies University of Sydney NSW 2006, Australia {star4245,james,tm}@it.usyd.edu.au Abstract The accuracy of named entity recognition sys- tems relies heavily upon the volume and qual- ity of available training data. Improving the process of automatically producing such train- ing data is an important task, as manual ac- quisition is both time consuming and expen- sive. We explore the use of a variety of machine learning algorithms for categorising Wikipedia articles, an initial step in produc- ing the named entity training data. We were able to achieve a categorisation accuracy of 95% F-score over six coarse categories, an improvement of up to 5% F-score over pre- vious methods. 1 Introduction Named Entity Recognition (NER) is the task of iden- tifying proper nouns, such as location, organisation and personal names, in text. It emerged as a dis- tinct type of information extraction during the sixth Message Understanding Conference (MUC) evalua- tion in 1995, and was further defined and explored in the CONLL NER evaluations of 2002 and 2003. A set of four broad categories became the stan- dard scheme for marking named entities (NEs) in text: person (PER), organisation (ORG), location (LOC), and miscellaneous (MISC). This scheme re- mains the most common, despite the development of more complex hierarchical category schemes (e.g. Brunstein (2002); Sekine et al. (2002)). Domain- specific category schemes have also been devel- oped in many areas, such as astroinformatics (Mur- phy et al., 2006), bioinformatics (Kim et al., 2003) and the travel industry (Vijayakrishna and Sobha, 2008). We also extend the broad scheme with a DAB category for Wikipedia “disambiguation” pages — pages used to group articles with identical titles. NER systems that categorise NEs under these schemes require a large amount of highly accurate training data to perform well at the task. Expert an- notation is time consuming and expensive, so there is an imperative to generate this data automatically. Wikipedia is emerging as a significant resource due to its immense size and rich structural information, such as its link structure. Nothman et al. (2009) introduced a novel ap- proach to exploiting Wikipedia’s internal structure to produce training data for NER systems. Their process involved an initial step of categorising all Wikipedia articles using a simple heuristic-based bootstrapping algorithm. Potential NEs were then identified as the words in an article’s text that served as links to other Wikipedia articles. To label a NE they then used the category assigned to the article that it linked to. We have explored the use of Na¨ıve Bayes (NB) and support vector machines (SVMs) as replace- ments for the text categorisation approach taken by Nothman. This involved the conversion of heuristics used by Nothman into features as well as the incor- poration of a number of new features. We demon- strate the superiority of our approach, providing a comparison of the individual text categorisation step to both Nothman’s system and other previous re- …

Content truncated to 500 words