Mixing Weak Learners in Semantic Parsin

Mixing Weak Learners in Semantic Parsing Rodney D. Nielsen Dept of Computer Science University of Colorado, UCB-430 Boulder, CO 80309-0430 USA Rodney.Nielsen@Colorado.edu Sameer Pradhan Center for Spoken Language Research University of Colorado Boulder, CO 80303 USA Sameer.Pradhan@Colorado.edu Abstract We apply a novel variant of Random Forests (Breiman, 2001) to the shallow semantic parsing problem and show extremely promising results. The final system has a semantic role classification accuracy of 88.3% using PropBank gold-standard parses. These results are better than all others published except those of the Support Vector Ma- chine (SVM) approach implemented by Pradhan et al. (2003) and Random Forests have numerous advantages over SVMs including simplicity, faster training and classification, easier multi-class classi- fication, and easier problem-specific customization. We also present new features which result in a 1.1% gain in classification accuracy and describe a tech- nique that results in a 97% reduction in the feature space with no significant degradation in accuracy. 1 Introduction Shallow semantic parsing is the process of finding sentence constituents that play a semantic role rela- tive to a target predicate and then labeling those con- stituents according to their respective roles. Speci- fying an event’s agent, patient, location, time of oc- currence, etc, can be useful for NLP tasks such as information extraction (c.f., Surdeanu et al., 2003), dialog understanding, question answering, text sum- marization, and machine translation. Example 1 de- picts a semantic parse. (1) [Agent She] [P bought] [P atient the vase] [Locative in Egypt] We expand on previous semantic parsing work (Gildea and Jurafsky, 2002; Pradhan et al., 2003; Surdeanu et al., 2003) by presenting a novel algo- rithm worthy of further exploration, describing a technique to drastically reduce feature space size, and presenting statistically significant new features. The accuracy of the final system is 88.3% on the classification task using the PropBank (Kingsbury et al., 2002) corpus. This is just 0.6% off the best accuracy reported in the literature. The classification algorithm used here is a vari- ant of Random Forests (RFs) (Breiman, 2001). This was motivated by Breiman’s empirical stud- ies of numerous datasets showing that RFs often have lower generalize error than AdaBoost (Fre- und and Schapire, 1997), are less sensitive to noise in the training data, and learn well from weak in- puts, while taking much less time to train. RFs are also simpler to understand and implement than SVMs, leading to, among other things, easier in- terpretation of feature importance and interactions (c.f., Breiman, 2004), easier multi-class classifica- tion (requiring only a single training session versus one for each class), and easier problem-specific cus- tomization (e.g., by introducing prior knowledge). The algorithm described here is considerably differ- ent from those in (Breiman, 2001). It was signifi- cantly revised to better handle high dimensional cat- egorical inputs and as a result provides much better accuracy on the shallow semantic parsing problem. The experiments reported here focus on the clas- sification task – given a parsed constituent known to play a semantic role relative to a …

Content truncated to 500 words