Descriptive and Empirical Approaches to Capturing Underlying Dependencies among Parsing Errors

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1162–1171, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Descriptive and Empirical Approaches to Capturing Underlying Dependencies among Parsing Errors Tadayoshi Hara1 Yusuke Miyao1 1Department of Computer Science, University of Tokyo Hongo 7-3-1, Bunkyo-ku, Tokyo, 113-0033, JAPAN 2School of Computer Science, University of Manchester 3NaCTeM (National Center for Text Mining) {harasan,yusuke,tsujii}@is.s.u-tokyo.ac.jp Jun’ichi Tsujii1,2,3 Abstract In this paper, we provide descriptive and empirical approaches to effectively ex- tracting underlying dependencies among parsing errors. In the descriptive ap- proach, we define some combinations of error patterns and extract them from given errors. In the empirical approach, on the other hand, we re-parse a sentence with a target error corrected and observe er- rors corrected together. Experiments on an HPSG parser show that each of these approaches can clarify the dependencies among individual errors from each point of view. Moreover, the comparison be- tween the results of the two approaches shows that combining these approaches can achieve a more detailed error analysis. 1 Introduction For any kind of technology, analyzing causes of errors given by a system is a very helpful process for improving its performance. In recent sophisti- cated parsing technologies, the process has taken on more and more important roles since critical ideas for parsing performance have already been introduced and the researches are now focusing on exploring the rest of the pieces for making addi- tional improvements. In most cases for parsers’ error analysis, re- searchers associate output errors with failures in handling certain linguistic phenomena and attempt to avoid them by adding or modifying correspond- ing settings of their parsers. However, such an analysis cannot been done so smoothly since pars- ing errors sometimes depend on each other and the underlying dependencies behind superficial phe- nomena cannot be captured easily. In this paper, we propose descriptive and em- pirical approaches to effective extraction of de- pendencies among parsing errors and engage in a deeper error analysis with them. In our descriptive approach, we define various combinations of error patterns as organized error phenomena on the ba- sis of linguistic knowledge, and then extract such combinations from given errors. In our empirical approach, on the other and, we re-parse a sentence under the condition where a target error is cor- rected, and errors which are additionally corrected are regarded as dependent errors. By capturing de- pendencies among parsing errors through system- atic approaches, we can effectively collect errors which are related to the same linguistic properties. In the experiments, we applied both of our ap- proaches to an HPSG parser Enju (Miyao and Tsu- jii, 2005; Ninomiya et al., 2006), and then evalu- ated the obtained error classes. After examining the individual approaches, we explored the com- bination of them. 2 Parser and its evaluation A parser is a system which interprets structures of given sentences from some grammatical or in some cases semantical viewpoints, and interpreted structures are utilized as essential information for various natural language …

Content truncated to 500 words