Named Entity Extraction using Information Distance

International Joint Conference on Natural Language Processing, pages 1264–1270, Nagoya, Japan, 14-18 October 2013. Named Entity Extraction Using Information Distance∗ Sangameshwar Patil†, Sachin Pawar‡, Girish K. Palshikar Tata Research Development and Design Centre, TCS 54B, Hadapsar Industrial Estate, Pune 411013, India {sangameshwar.patil,sachin7.p,gk.palshikar}@tcs.com Abstract Named entities (NE) are important infor- mation carrying units within documents. Named Entity extraction (NEX) task con- sists of automatic construction of a list of phrases belonging to each NE of in- terest. NEX is important for domains which lack a corpus with tagged NEs. We present an enhanced version and im- proved results of our unsupervised (boot- strapping) NEX technique (Patil et al., 2013) and establish its domain indepen- dence using experimental results on cor- pora from two different domains: agricul- ture and mechanical engineering (IC en- gine1 parts). We use a new variant of Mul- tiword Expression Distance (MED) (Bu et al., 2010) to quantify proximity of a can- didate phrase with a given NE type. MED itself is an approximation of the informa- tion distance (Bennett et al., 1998). Effi- cacy of our method is shown using exper- imental comparison with pointwise mu- tual information (PMI), BASILISK and KNOWITALL. Our method discovered 8 new plant diseases which are not found in Wikipedia. To the best of our knowl- edge, this is the first use of NEX tech- niques for agriculture and mechanical en- gineering (engine parts) domains. 1 Introduction Agriculture is an activity of fundamental impor- tance in all societies. For example, agriculture plays a vital role in the economy of India: gen- erating second largest farm output in the world, ∗Preliminary version of this paper was presented as a poster at NLDB 2013. †Doctoral research scholar at Dept. of CSE, IIT Madras ‡Doctoral research scholar at Dept. of CSE, IIT Bombay 1Internal Combustion engine providing 52% of rural employment (≈ 250 mil- lion people) and contributing ≈ 15% to the GDP 2. Hence timely and widespread dissemination of agriculture-related information is important. Such information - extracted, collated and summarized from a variety of document sources such as news, reports, web-sites and scientific literature - can be used to improve various aspects of services pro- vided to large population dependent on agricul- ture. Problem of information extraction for do- mains such as agriculture is particularly challeng- ing due to non-availability of any tagged corpus. Several domain-specific named entities (NE) occur in the documents (such as news) related to the agriculture domain- CROP: names of the crops including varieties; DISEASE: names of the crop diseases and disease causing agents such as pathogen (bacteria, viruses, fungi), insects etc.; CHEMICAL TREATMENT: names of pesticides, insecticides, fungicides etc. used in the treatment of a crop disease. Consider an example of NEs tagged for agricultural information extraction: We usually spray [soybeans]CROP with a [strobilurin fungicide]CHEM T REAT MENT because of the potential for [soybean rust]DISEASE and other diseases. As there are few, if any, tagged corpora of agriculture-related documents, we consider an un- supervised approach for extracting these NEs. …

Content truncated to 500 words