Sparse Bayesian Classification of Predicate Arguments

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 177–180, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Sparse Bayesian Classification of Predicate Arguments Richard Johansson and Pierre Nugues LUCAS, Department of Computer Science, Lund University Box 118 SE-221 00 Lund, Sweden {richard, pierre}@cs.lth.se Abstract We present an application of Sparse Bayesian Learning to the task of semantic role labeling, and we demonstrate that this method produces smaller classifiers than the popular Support Vector approach. We describe the classification strategy and the features used by the classifier. In par- ticular, the contribution of six parse tree path features is investigated. 1 Introduction Generalized linear classifiers, in particular Support Vector Machines (SVMs), have recently been suc- cessfully applied to the task of semantic role iden- tification and classification (Pradhan et al., 2005), inter alia. Although the SVM approach has a number of properties that make it attractive (above all, excel- lent software packages exist), it also has drawbacks. First, the resulting classifier is slow since it makes heavy use of kernel function evaluations. This is especially the case in the presence of noise (since each misclassified example has to be stored as a bound support vector). The number of support vec- tors typically grows with the number of training ex- amples. Although there exist optimization methods that speed up the computations, the main drawback of the SVM approach is still the classification speed. Another point is that it is necessary to tune the parameters (typically C and γ). This makes it nec- essary to train repeatedly using cross-validation to find the best combination of parameter values. Also, the output of the decision function of the SVM is not probabilistic. There are methods to map the decision function onto a probability output using the sigmoid function, but they are considered some- what ad-hoc (see (Tipping, 2001) for a discussion). In this paper, we apply a recent learning paradigm, namely Sparse Bayesian learning, or more specifically the Relevance Vector learning method, to the problem of role classification. Its principal advantages compared to the SVM ap- proach are: • It typically utilizes fewer examples compared to the SVM, which makes the classifier faster. • It uses no C parameter, which reduces the need for cross-validation. • The decision function is adapted for probabilis- tic output. • Arbitrary basis functions can be used. Its significant drawback is that the training pro- cedure relies heavily on dense linear algebra, and is thus difficult to scale up to large training sets and may be prone to numerical difficulties. For a description of the task and the data, see (Car- reras and Màrquez, 2005). 2 Sparse Bayesian Learning and the Relevance Vector Machine The Sparse Bayesian method is described in detail in (Tipping, 2001). Like other generalized linear learn- ing methods, the resulting binary classifier has the form signf(x) = sign m � i=1 αifi(x) + b 177 where the fi are basis functions. Training the model then consists of finding a suitable α …

Content truncated to 500 words