DutchSemCor: in quest of the ideal sense-tagged corpus

Proceedings of Recent Advances in Natural Language Processing, pages 710–718, Hissar, Bulgaria, 7-13 September 2013. DutchSemCor: in Quest of the Ideal Sense Tagged Corpus Piek Vossen, Rub´en Izquierdo and Attila G¨or¨og VU University Amsterdam piek.vossen/ruben.izquierdobevia/a.gorog@vu.nl Abstract The most-frequent-sense and the pre- dominant domain sense play an impor- tant role in the debate on word-sense- disambiguation. This discussion is, how- ever, biased by the way sense-tagged cor- pora are built. In this paper, we ar- gue that current sense-tagged corpora ne- glect rare senses and contexts and, as a result, do not represent a good cor- pus for training and testing word-sense- disambiguation. We defined three qual- ity criteria for sense-tagged corpora and a methodology to satisfy these criteria with minimal effort. Following this method, we built a Dutch sense-tagged corpus that tried to meet these criteria. The cor- pus was evaluated by deriving word-sense- disambiguation systems and testing these on different subsets of the corpus in dif- ferent ways. The performance of our sys- tems and the quality of the derived data are equal to state-of-the-art English systems and corpora. Finally, we used the sys- tems to create a Dutch corpus of over 47 million sense-tagged tokens spread over a large variety of genres, domains and us- ages of Dutch. The results of the project can be downloaded freely from the project website. 1 Credits The DutchSemCor project was an NWO Human- ities medium investment subsidy project with a subsidy period of September 2009 - August 2012. We would like to thank NWO for making it possi- ble to carry out the project. 2 Introduction Word Sense Disambiguation (WSD) research in the last decade demonstrated a number of impor- tant insights (Agirre and Edmonds, 2006): 1. eval- uation results are strongly dependent on the cor- pus and the lexicons used, 2. the most-frequent- sense derived from SemCor (Miller et al., 1993) is a strong baseline that is not easy to beat in evalu- ations like SensEval or SemEval and 3. predomi- nant senses in specific domains give the best WSD results by far (McCarthy et al., 2007). From these observations, one may conclude that we need to collect large sets of (sense-tagged) domain- and probably genre-specific corpora to determine pre- dominant senses. Obtaining sufficient data with- out ignoring rare or low-frequent senses, however, requires an enormous effort. Manually tagged data is still very sparse and evaluation results vary from task to task, hence we still do not know where we stand in the area of WSD. This raises the question: how should the ideal sense-tagged corpus for WSD look like, to enable detection of any sense in any type of corpus? Ex- isting sense-tagged corpora have different design properties that make them good corpora in some aspects but not in others. In this paper, we will de- fine quality criteria for sense-tagged corpora and will describe a novel method for building a large- scale sense-tagged Dutch corpus that meets these criteria with as little manual effort as possible. We argue …

Content truncated to 500 words