The Effect of Ambiguity on the Automated Acquisition of WSD Examples

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 353–356, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics The Effect of Ambiguity on the Automated Acquisition of WSD Examples Mark Stevenson and Yikun Guo Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello, Sheffield, S1 4DP United Kingdom m.stevenson@dcs.shef.ac.uk and g.yikun@dcs.shef.ac.uk Abstract Several methods for automatically gen- erating labeled examples that can be used as training data for WSD systems have been proposed, including a semi- supervised approach based on relevance feedback (Stevenson et al., 2008a). This approach was shown to generate examples that improved the performance of a WSD system for a set of ambiguous terms from the biomedical domain. However, we find that this approach does not perform as well on other data sets. The levels of ambigu- ity in these data sets are analysed and we suggest this is the reason for this negative result. 1 Introduction Several studies, for example (Mihalcea et al., 2004; Pradhan et al., 2007), have shown that su- pervised approaches to Word Sense Disambigua- tion (WSD) outperform unsupervised ones. But these rely on labeled training data which is diffi- cult to create and not always available (e.g. (Wee- ber et al., 2001)). Various techniques for creating labeled training data automatically have been sug- gested in the literature. Stevenson et al. (2008a) describe a semi-supervised approach that used rel- evance feedback (Rocchio, 1971) to analyse ex- isting labeled examples and use the information produced to generate further ones. The approach was tested on the biomedical domain and the addi- tional examples found to improve performance of a WSD system. However, biomedical documents represent a restricted domain. In this paper the same approach is tested against two data sets that are not limited to a single domain. 2 Application to a Range of Data Sets In this paper the relevance feedback approach de- scribed by Stevenson et al. (2008a) is evaluated us- ing three data sets: the NLM-WSD corpus (Wee- ber et al., 2001) which Stevenson et al. (2008a) used for their experiments, the Senseval-3 lexical sample task (Mihalcea et al., 2004) and the coarse- grained version of the SemEval English lexical sample task (Pradhan et al., 2007). 2.1 Generating Examples To generate examples for a particular sense of an ambiguous term all of the examples where the term is used in that sense are considered to be “relevant documents” while the examples in which any other sense of the term is used are considered to be “irrelevant documents”. Relevance feed- back (Rocchio, 1971) is used to generate a set of query terms designed to identify relevant docu- ments, and therefore instances of the sense. The top five query terms are used to retrieve docu- ments and these are used as labeled examples of the sense. Further details of this process are de- scribed by Stevenson et al. (2008a). This process requires a collection of documents that can be queried to generate the …

Content truncated to 500 words