A Language Independent Shallow-Parser Compiler

A language−independent shallow−parser Compiler Alexandra Kinyon CIS Dpt. . University of Pennsylvania kinyon@linc.cis.upenn.edu http://www.cis.upenn.edu/˜kinyon Abstract We present a rule−based shallow− parser compiler, which allows to generate a robust shallow−parser for any language, even in the absence of training data, by resorting to a very limited number of rules which aim at identifying constituent boundaries. We contrast our approach to other approaches used for shallow−parsing (i.e. finite−state and probabilistic methods). We present an evaluation of our tool for English (Penn Treebank) and for French (newspaper corpus "LeMonde") for several tasks (NP−chunking & "deeper" parsing) . 1 Introduction Full syntactic parsers of unrestricted text are costly to develop, costly to run and often yield errors, because of lack of robustness of wide− coverage grammars and problems of attachment. This has led, as early as 1958 (Joshi & Hopely 97), to the development of shallow−parsers, which aim at identifying as quickly and accurately as possible, main constituents (and possibly syntactic functions) in an input, without dealing with the most difficult problems encountered with "full−parsing". Hence, shallow−parsers are very practical tools. There are two main techniques used to develop shallow−parsers: 1− Probabilistic techniques (e.g. Magerman 94, Ratnaparkhi 97, Daelmans & al. 99) 2− Finite−state techniques (e.g. Grefenstette 96) Probabilistic techniques require large amounts of syntactically−annotated training data1, which makes them very unsuitable for languages for 1 We are leaving aside unsupervised learning techniques here, since to our knowledge they have not proved a successful for developing practical shallow−parsers. which no such data is available (i.e. most languages except English) and also, they are not domain−independent nor "style−independent" (e.g. they do not allow to successfully shallow− parse speech, if no annotated data is available for that “style”). Finally, a shallow−parser developed using these techniques will have to mirror the information contained in the training data. For instance, if one trains such a tool on data were only non recursive NP chunks are marked2, then one will not be able to obtain richer information such as chunks of other categories, embeddings, syntactic functions... On the other hand, finite−state techniques rely on the development of a large set of rules (often based on regular expressions) to capture all the ways a constituent can expend. So for example for detecting English NPs, one could write the following rules : NP → Det adj* noun adj* NP → Det adj (for noun ellipsis) NP → ProperNoun etc .... But this is time consuming and difficult since one needs to foresee all possible rewriting cases, and if some rule is forgotten, or if too many POS errors are left, robustness and/or accuracy will suffer. Then these regular expressions have to be manipulated i.e. transformed into automata, which will be determinized and minimized (both being costly operations). And even though determinization and minimization must be done only once (in theory) for a given set of rules, it is still costly to port such tools to a new set of rules (e.g. for a new language, a new domain) or to change …

Content truncated to 500 words