Paradigmatic Modifiability Statistics for the Extraction of Complex Multi-Word Terms

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 843–850, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Paradigmatic Modifiability Statistics for the Extraction of Complex Multi-Word Terms Joachim Wermter Jena University Language & Information Engineering JULIE Lab Friedrich-Schiller-Universit¨at Jena F¨urstengraben 30 D-07743 Jena, Germany joachim.wermter@uni-jena.de Udo Hahn Jena University Language & Information Engineering JULIE Lab Friedrich-Schiller-Universit¨at Jena F¨urstengraben 30 D-07743 Jena, Germany udo.hahn@uni-jena.de Abstract We here propose a new method which sets apart domain-specific terminology from common non-specific noun phrases. is based on the observation that termino-It logical multi-word groups reveal a con- siderably lesser degree of distributional variation than non-specific noun phrases. We define a measure for the observable amount of paradigmatic modifiability of terms and, subsequently, test it on bigram, trigram and quadgram noun phrases ex- tracted from a 104-million-word biomedi- cal text corpus. Using a community-wide curated biomedical terminology system as an evaluation gold standard, we show that our algorithm significantly outperforms a variety of standard term identification measures. We also provide empirical ev- idence that our methodolgy is essentially domain- and corpus-size-independent. 1 Introduction As we witness the ever-increasing proliferation of volumes of medical and biological documents, the available dictionaries and terminological systems cannot keep up with this pace of growth and, hence, become more and more incomplete. What’s worse, the constant stream of new terms is increasingly get- ting unmanageable because human curators are in the loop. The costly, often error-prone and time- consuming nature of manually identifying new ter- minology from the most recent literature calls for advanced procedures which can automatically assist database curators in the task of assembling, updat- ing and maintaining domain-specific controlled vo- cabularies. Whereas the recognition of single-word terms usually does not pose any particular chal- lenges, the vast majority of biomedical or any other domain-specific terms typically consists of multi- word units.1 Unfortunately these are much more difficult to recognize and extract than their singleton counterparts. Moreover, although the need to assem- ble and extend technical and scientific terminologies is currently most pressing in the biomedical domain, virtually any (sub-)field of human research/expertise in which we deal with terminologically structured knowledge calls for high-performance terminology identification and extraction methods. We want to target exactly this challenge. 2 Related Work The automatic extraction of complex multi-word terms from domain-specific corpora is already an active field of research (cf., e.g., for the biomedi- cal domain Rindflesch et al. (1999), Collier et al. (2002), Bodenreider et al. (2002), or Nenadi´c et al. (2003)). Typically, in all of these approaches term candidates are collected from texts by vari- ous forms of linguistic filtering (part-of-speech tag- ging, phrase chunking, etc.), through which candi- dates obeying various linguistic patterns are iden- tified (e.g., noun-noun, adjective-noun-noun com- binations). These candidates are then submitted to frequency- or statistically-based evidence measures 1Nakagawa and Mori (2002) claim that more than 85% of domain-specific terms are multi-word units. 843 (such as the C-value (Frantzi et al., 2000)), which compute scores …

Content truncated to 500 words