LoLo: A System based on Terminology for Multilingual Extraction

Proceedings of the Workshop on Information Extraction Beyond The Document, pages 56–65, Sydney, July 2006. c⃝2006 Association for Computational Linguistics LoLo: A System based on Terminology for Multilingual Extraction Yousif Almas Department of Computing University of Surrey Guildford, Surrey, GU2 7XH, UK y.almas@surrey.ac.uk Khurshid Ahmad Department of Computer Science Trinity College, Dublin-2. IRELAND kahmad@cs.tcd.ie Abstract An unsupervised learning method, based on corpus linguistics and special lan- guage terminology, is described that can extract time-varying information from text streams. The method is shown to be ‘language-independent’ in that its use leads to sets of regular-expressions that can be used to extract the information in typologically distinct languages like Eng- lish and Arabic. The method uses the in- formation related to the distribution of N- grams, for automatically extracting ‘meaning bearing’ patterns of usage in a training corpus. The analysis of an Eng- lish news wire corpus (1,720,142 tokens) and Arabic news wire corpus (1,720,154 tokens) show encouraging results. 1 Introduction One of the recent trends in (adaptive) IE has been motivated by the empirical argument that annotated corpora, either annotated automatically or annotated manually, can provide sufficient information for creating the knowledge base of an IE system (McLernon and Kushmerick, 2006). Another equally important trend is to use manually selected seed patterns to initiate learn- ing: In turn, active-training methods use seed patterns to learn new related patterns from un- annotated corpora. Many of the adaptive IE sys- tems rely on the existing part-of-speech (POS) taggers (Debnath and Giles, 2005) and/or syntac- tic parsers (Stevenson and Greenwood, 2005) for analysing and annotating text corpora. The use of corpora in IE, especially adaptive IE, should, in principle, alleviate the need for manually creat- ing the rules for information extraction. The successful use of POS/syntactic taggers is dependent on the availability of the knowledge of (natural) language used by the authors of documents in a given corpus. There is a wealth of POS taggers and parsers available for English language, as it has been the most widely used language in computational linguistics. However, this is not the case for strategically important languages like Arabic and Chinese; to start with, in Chinese one does not have the luxury of sepa- rating word-tokens by a white space and in Ara- bic complex rules are required to identify mor- phemes compared to English. The development of segmentation programs in these languages has certainly helped (Gao et al., 2005; Habash and Rambow, 2005). More work is needed in under- standing these languages such that the knowl- edge thus derived can be used to power taggers and parsers. Typically, IE systems are used to analyse news wire corpora, telephone conversations, and more recently in bio-informatics. The first two systems deal with language of everyday commu- nications –the general language- whereas bio- informatics deals with a specialist domain and has its own ‘special language’. English special languages, for example languages of law, com- merce, finance, science & technology, each have a limited vocabulary and idiosyncratic syntactic structures when compared …

Content truncated to 500 words