Shallow Language Processing Architecture for Bulgarian

Shallow language processing architecture for Bulgarian Hristo Tanev ITC-Irst, Centro per la Ricerca Scientifica e Tecnologica Povo,Trento, Italy 38050 tanev@itc.it Ruslan Mitkov School of Humanities, Languages and Social Studies Wolverhampton WV1 1SB UK R.Mitkov@wlv.ac.uk Abstract This paper describes LINGUA - an architec- ture for text processing in Bulgarian. First, the pre-processing modules for tokenisation, sen- tence splitting, paragraph segmentation, part- of-speech tagging, clause chunking and noun phrase extraction are outlined. Next, the pa- per proceeds to describe in more detail the anaphora resolution module. Evaluation results are reported for each processing task. 1 Introduction The state of the art of today’s full parsing and knowledge-based automatic analysis still falls short of providing a reliable processing frame- work for robust, real-world applications such as automatic abstracting or information ex- traction. The problem is especially acute for languages which do not benefit from a wide range of processing programs such as Bulgar- ian. There have been various projects which address different aspects of the automatic anal- ysis in Bulgarian such as morphological analy- sis (Krushkov, 1997), (Simov et al., 1992), mor- phological disambiguation (Simov et al., 1992) and parsing (Avgustinova et al., 1989), but no previous work has pursued the development of a knowledge-poor, robust processing environ- ment with a high level of component integrity. This paper reports the development and im- plementation of a robust architecture for lan- guage processing in Bulgarian referred to as LINGUA, which includes modules for POS tag- ging, sentence splitting, clause segmentation, parsing and anaphora resolution. Our text pro- cessing framework builds on the basis of con- siderably shallower linguistic analysis of the in- put, thus trading off depth of interpretation for breadth of coverage and workable, robust solu- tion. LINGUA uses knowledge poor, heuristi- cally based algorithms for language analysis, in this way getting round the lack of resources for Bulgarian. 2 LINGUA - an architecture for language processing in Bulgarian LINGUA is a text processing framework for Bulgarian which automatically performs tokeni- sation, sentence splitting, part-of-speech tag- ging, parsing, clause segmentation, section- heading identification and resolution for third person personal pronouns (Figure 1). All mod- ules of LINGUA are original and purpose- built, except for the module for morphological analysis which uses Krushkov’s morphological analyser BULMORPH (Krushkov, 1997). The anaphora resolver is an adaptation for Bulgar- ian of Mitkovs knowledge-poor pronoun resolu- tion approach (Mitkov, 1998). LINGUA was used in a number of projects covering automatic text abridging, word seman- tic extraction (Totkov and Tanev, 1999) and term extraction. The following sections outline the basic language processing functions, pro- vided by the language engine. 2.1 Text segmentation: tokenisation, sentence splitting and paragraph identification The first stage of every text processing task is the segmentation of text in terms of tokens, sen- tences and paragraphs. LINGUA performs text segmentation by op- erating within an input window of 30 tokens, ap- plying rule-based algorithm for token synthesis, sentence splitting and paragraph identification. 2.1.1 Tokenisation and token stapling Tokens identified from the input text serve as input …

Content truncated to 500 words