Hybrid Text Mining for Finding Abbreviations and their Definitions
43242Authors
nan
Abstract
We� present� a� hybrid� text� mining method� for� finding� abbreviations� and their�definitions�in�free�format�texts.�To deal� with� the� problem,� this� method employs� pattern-based� abbreviation rules�in�addition�to�text�markers�and�cue words.�The�pattern-based�rules�describe how� abbreviations� are� formed� from definitions.� Rules� can� be� generated automatically� and/or� manually� and� can be� augmented� when� the� system processes� new� documents.� The proposed�method�has�the�advantages�of high� accuracy,� high� flexibility,� wide coverage,�and�fast�recognition. �
Keywords
Full Content
Hybrid�text�mining�for�finding�abbreviations�and�their�definitions Youngja�Park,���Roy�J.�Byrd IBM�Thomas�J.�Watson�Research�Center P.O.�Box�704,�Yorktown�Heights New�York�10598,�USA {pyoungja,�roybyrd}@us.ibm.com �Abstract We� present� a� hybrid� text� mining method� for� finding� abbreviations� and their�definitions�in�free�format�texts.�To deal� with� the� problem,� this� method employs� pattern-based� abbreviation rules�in�addition�to�text�markers�and�cue words.�The�pattern-based�rules�describe how� abbreviations� are� formed� from definitions.� Rules� can� be� generated automatically� and/or� manually� and� can be� augmented� when� the� system processes� new� documents.� The proposed�method�has�the�advantages�of high� accuracy,� high� flexibility,� wide coverage,�and�fast�recognition. �Introduction Many� organizations� have� a� large� number� of on-line�documents�--�such�as�manuals,�technical reports,� transcriptions� of� customer� service� calls or�telephone�conferences,�and�electronic�mail�-- which� contain� information� of� great� potential value.� In� order� to� utilize� the� knowledge� these data� contain,� we� need� to� be� able� to� create common� glossaries� of� domain-specific� names and�terms.�While�we�were�working�on�automatic glossary� extraction,� we� noticed� that� technical documents� contain� a� lot� of� abbreviated� terms, which� carry� important� knowledge� about� the domains.� We� concluded� that� the� correct recognition�of�abbreviations�and�their�definitions is� very� important� for� understanding� the documents� and� for� extracting� information� from them�[1,�6,�9,�11]. An� abbreviation� is� usually� formed� by� a simple�method:�taking�zero�or�more�letters�from each� word� of� its� definition.� However,� the tendency� to� make� unique,� interesting abbreviations�is�growing.��So,�it�is�easy�to�find new� kinds� of� abbreviations� which� cannot� be processed� by� hard-coded� heuristics-based algorithms� [1,� 6,� 7,� 13,� 14],� since� they� are formed� in� ways� not� anticipated� when� the algorithms�were�devised. We�propose�a�hybrid�text�mining�approach�to deal�with�these�problems.�We�use�three�kinds�of knowledge:� � pattern-based� abbreviation� rules, text� markers,� and� linguistic� cue� words.� An abbreviation� rule� consists� of� an� abbreviation pattern,�a�definition�pattern�and�a�formation�rule. The� formation� rule� describes� how� an abbreviation�is�formed�from�a�definition.�There may� exist� multiple� formation� rules� for� a� given pair� of� abbreviation� and� definition� patterns. Abbreviation�rules�are�described�in�Section�3. used�Text��markers�are�special�symbols�frequently to� imply� the� abbreviation-definition relationship� in� texts.� � They� include� characters such� as� ‘(…)’,� ‘[…]’,� and� ‘=’.� Cue� words� are particular�words�occurring�in�the� local�contexts of� abbreviations� and� the� definitions,� which strongly� imply� the� abbreviation� relationship. They� include� words� such� as� “or”,� “short”, “acronym”�and�“stand”.���Text�markers�and�cue words�are�discussed�in�section�2.4. �����This� system� has� 5� components� abbreviation� recognizer,� definition� finder,� rule-- applier,� abbreviation� matcher� and� best� match selector� --� as� shown� in� Figure� 1.� The abbreviation� recognizer� seeks� candidate abbreviations� in� a� text� and� generates� their patterns� (Section� 1).� When� an� abbreviation candidate� is� found,� the� system� determines� the contexts� within� which� to� look� for� a� definition. When�it�finds�a�candidate�definition,�it�generates a�pattern�for�it�also�(Section�2). �����Having� generated� the� abbreviation� pattern and� the� definition� pattern,� the� system� first searches� the� rulebase� for� a� rule� which� would generate� the� abbreviation� from� the� definition. The�rules�for�the�given�candidates�are�applied�in the� order� of� rule� priorities� (Section� 4.1).� If� the rulebase�is�empty�or�if�no�existing�rule�matches the� candidate� abbreviation� with� the� candidate definition,� the� system� runs� the� abbreviation matcher�and� generates�a�new�abbreviation�rule. The� abbreviation� matcher� consists� of� 5� layered matching� algorithms� (Section� 4.2).� If� the matcher� succeeds,� new� rules� may� be� added� to the�rulebase,�allowing� it�to� grow� as� the� system processes�new�documents. �1.�Abbreviation�Recognition 1.1�Candidate�Abbreviations An� abbreviation� is� defined� as� a� shortened form�of�a�written�word�or�phrase�used�in�place�of the�full�form�[2].�Acronyms�are�a�special�case�of abbreviations� which� are� devoted� to� multi-word full�forms.��In�this�work,�we�consider�a�string�of alphabetic,� numeric� and� special� characters� as�a candidate� abbreviation� if� it� satisfies� the following�three�conditions: ���(1)�Its�first�character�is�alphabetic�or�numeric ���(2)�Its�length�is�between�2�and�10�characters ���(3)�It�contains�at�least�one�capital�letter and�if�the�string�meets�the�following�restrictions: (1)� It� is� not� a� known� (dictionary)� word containing� an� initial� capital� letter� and appearing�as�the�first�word�in�a�sentence. (2)� It�is�not�a�member�of�a�predefined�set�of person�and�location�names. (3)� It�is�not�a�member�of�user-defined�list�of stopwords. The� first� restriction� keeps� many� common words� from� being� treated� as� possible abbreviations.� Many� proper� names� have� the same� characteristics� above� and� may� be recognized� as� abbreviations.� …
Content truncated to 500 words
Labeled Text
We� present� a� hybrid� text� mining method� for� finding� abbreviations� and their�definitions�in�free�format�texts.�To deal� with� the� problem,� this� method employs� pattern-based� abbreviation rules�in�addition�to�text�markers�and�cue words.�The�pattern-based�rules�describe how� abbreviations� are� formed� from definitions.� Rules� can� be� generated automatically� and/or� manually� and� can be� augmented� when� the� system processes� new� documents.� The proposed�method�has�the�advantages�of high� accuracy,� high� flexibility,� wide coverage,�and�fast�recognition. �
Publication Details
- Paper ID:
- 43242
- Year:
- 2001
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan