Lexical Parallelism in Text Structure Determination and Content Analysis

COLING 82, J. Horeclo) (ed.) North-Holland Publishing Company © Academia, 1982 LEXICAL PARALLELISM IN TEXT STRUCTURE DETERMINATION AND CONTENT ANALYSIS Yoshiyuki Sakamoto Electrotechnical Laboratory Tsukuba, Japan Tetsuya Okamoto University of Electrocommunications Tokyo, Japan ABSTRACT In this paper the problem is discussed about the text structure determination and content analysis by lexical parall~lism, or the repetition of lexical items. Intersentential relations are determined through the identical, partly identical or lexico-semantic repetition in Japanese scientific texts. Lexical parallelism ratio and lexical parallelism indicator distance are obtained on computer and by hand. And the application of the characteristics to automatic content analysis is dicsussed. I. INTRODUCTION Lexical parallelism, that is, the repetition of lexical items, is an important device for indicating the sentence connections in a text(discouse). The recurrent lexical items, or lexical equivalents need not have the same syntactic function or parts of speech in the two sentences in which they occur. They may be identical in form and in meaning, or they may be related by lexico-semantic relationship, such as synonymy, hyponymy, antonymy. In a special case they may be partly identical both in form and in meaning, as in ~ (ultrasonic wave), ~(sound wave) and ~ (sound). Another device for indicating the sentence connections is a syntactic device, such as substitutes, logical connecters, time and place relaters and structural parallelism [I]. For example, in Japanese substitutes--- ~/ C~ (this), ~_.__~__~ (here), ~ (we/our), ~_____~ (it), time relaters--- ~ (next), ~ (above mentioned), and logical connecters--- ~ (and), ~ (or), m--~ (secondly) belong to this device. Sevbo studied lexlcal parallelism in normalized text, where substitutes were replaced by their lexical equivalents and complex sentences were decomposed into successive simple sentences(clauses). She traced the repetition patterns of lexical items in Subject/Predicate oppossitio n. She assumes here that the syntactic subject or its dependent, direct or indirect, corresponds to "Subject(old information) of elementary thought" and the syntactic predicate or its dependent to "Predicate(new information) of elementary thought"J2]. In Japanese, sentence components occur in any positions before predicate and old information or topic is placed, as a rule, at/near the beginning of a sentence[3]. In the following discussion we analyze the repetition of lexical items in an unnormalized text without regard to their syntactic functions, parts of speech and topic/comment distinctions, assuming that the lexical equivalents at/near the beginning of the sentences function as the keywords in indicating the sentence connections and the contents of a text. Nouns do not inflect and most verbs and adjectives have the unchanging stems and inflectional suffixes in Japanese. The important concepts and technical terms (noun, verb or 339 340 Y. SAKAMOTO and T. OKAMOTO adjective stems) are written in Kanji (Chinese ideographs) or Katakana(square Japanese syllabary). Katakana is used to transcribe foreign technical terms. Hiragana(Japanese cursive syllabary), on the other hand, is used to write post-positional particles and suffixes, denoting case, topic, mood, tense aspect etc. In view of these facts we define lexical items as a word or phrase in Kanji and Katakana. We have studied …

Content truncated to 500 words