ULISSE: an Unsupervised Algorithm for Detecting Reliable Dependency Parses

Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 115–124, Portland, Oregon, USA, 23–24 June 2011. c⃝2011 Association for Computational Linguistics ULISSE: an Unsupervised Algorithm for Detecting Reliable Dependency Parses Felice Dell’Orletta, Giulia Venturi and Simonetta Montemagni Istituto di Linguistica Computazionale “Antonio Zampolli” (ILC–CNR) via G. Moruzzi, 1 – Pisa (Italy) {felice.dellorletta,giulia.venturi,simonetta.montemagni}@ilc.cnr.it Abstract In this paper we present ULISSE, an unsu- pervised linguistically–driven algorithm to se- lect reliable parses from the output of a de- pendency parser. Different experiments were devised to show that the algorithm is robust enough to deal with the output of different parsers and with different languages, as well as to be used across different domains. In all cases, ULISSE appears to outperform the baseline algorithms. 1 Introduction While the accuracy of state–of–the–art parsers is in- creasing more and more, this is still not enough for their output to be used in practical NLP–based ap- plications. In fact, when applied to real–world texts (e.g. the web or domain–specific corpora such as bio–medical literature, legal texts, etc.) their accu- racy decreases significantly. This is a real problem since it is broadly acknowledged that applications such as Information Extraction, Question Answer- ing, Machine Translation, and so on can benefit sig- nificantly from exploiting the output of a syntactic parser. To overcome this problem, over the last few years a growing interest has been shown in assessing the reliability of automatically produced parses: the selection of high quality parses represents nowadays a key and challenging issue. The number of stud- ies devoted to detecting reliable parses from the out- put of a syntactic parser is spreading. They mainly differ with respect to the kind of selection algo- rithm they exploit. Depending on whether training data, machine learning classifiers or external parsers are exploited, existing algorithms can be classified into i) supervised–based, ii) ensemble–based and iii) unsupervised–based methods. The first is the case of the construction of a ma- chine learning classifier to predict the reliability of parses on the basis of different feature types. Yates et al. (2006) exploited semantic features derived from the web to create a statistical model to de- tect unreliable parses produced by a constituency parser. Kawahara and Uchimoto (2008) relied on features derived from the output of a supervised de- pendency parser (e.g. dependency lengths, num- ber of unknown words, number of coordinated con- junctions, etc.), whereas Ravi et al. (2008) exploited an external constituency parser to extract text–based features (e.g. sentence length, unknown words, etc.) as well as syntactic features to develop a super- vised predictor of the target parser accuracy. The approaches proposed by Reichart and Rappoport (2007a) and Sagae and Tsujii (2007) can be classi- fied as ensemble–based methods. Both select high quality parses by computing the level of agreement among different parser outputs: wheras the former uses several versions of a constituency parser, each trained on a different sample from the training data, the latter uses the parses produced by different de- pendency parsing algorithms trained on the …

Content truncated to 500 words