Extracting Textual Associations in Part-of-Speech Tagged Corpora

Extracting Textual Associations in Part-of-Speech Tagged Corpora Gaël Dias123, Sylvie Guilloré2 & José Gabriel Pereira Lopes3 1 Universidade da Beira Interior Departamento de Matemática-Informática ddg@noe.ubi.pt 2 Université d’Orléans Laboratoire d’Informatique Fondamentale d’Orléans sylvie.guillore@lifo.univ-orleans.fr 3 Universidade Nova de Lisboa Departamento de Informática gpl@di.fct.unl.pt Abstract Identifying textual associations from text corpora is a useful pre-processing step for many applications in Natural Language Processing. In this paper, we will present an innovative system that extracts relevant sequences of characters, words and part-of- speech tags from corpora. We will show that the combination of a new association measure (Mutual Expectation) with a new acquisition process (LocalMaxs) proposes an integrated solution to the problems of enticement techniques and global thresholds highlighted by previous researches. Introduction Identifying textual associations from corpora is a useful pre-processing step for many applications in Natural Language Processing. In the context ������������������������������������������������ multiword lexical units (sequences of words that co-occur more often than expected by chance) are often opaque in the comprehension phase and cause hesitations in the production process. For instance, "Bill of Rights", "swimming pool", "as well as", "in order to", "to comply with" and "to put forward" are multiword lexical units. Consequently, their identification within the process of text normalisation is a crucial issue for the specific tasks of machine translation, information extraction and information retrieval. Textual associations are not restricted to word associations. Indeed, Argamon-Engelson et al. (1999) assess that the identification of local patterns of syntactical sequences is essential for various application areas including word sense disambiguation, bilingual alignment and text summarisation. The principle approach for the detection of syntactical patterns is the task of shallow parsing which consists, according to Abney's definition (1991), in a chunker that offers potential part-of- speech tag sequences to an attacher that solves attachment ambiguities and selects the final chunks. It is clear that systems that may identify meaningful part-of-speech tags associations such as "AT JJ NN"1, "JJ NP CC JJ NP", "NP $ JJ NN", "NP CO NP CO NP CC NP" and "HV RB BEN" would greatly benefit the shallow parsing task. Finally, in the context of character associations, the decomposition of words into morphemes has proved to lead to improved results in different areas including text indexing, bilingual alignment and information extraction. Indeed, Grabar and Zweigenbaum (1999) confirm that a great deal of words in European Languages share common Greek and Latin morphemes that allow the generalisation of concepts. As a consequence, it is convenient to define morphological segments as meaningful sequences of characters that should be automatically extracted from corpora. For instance, the following words belong to the same morphological family as they share the same stem: ������������, ����������, ������� and �����. In order to identify and extract meaningful sequences of words, part-of-speech tags and characters, we present a statistically- based architecture called SENTA (Software for the Extraction of N-ary Textual Associations) that retrieves, from naturally occurring text, relevant contiguous and non-contiguous textual associations. For that purpose, we combine a new association measure called the Mutual …

Content truncated to 500 words