A New Approach for Idiom Identification Using Meanings and the Web

Proceedings of Recent Advances in Natural Language Processing, pages 681–687, Hissar, Bulgaria, Sep 7–9 2015. A New Approach for Idiom Identification Using Meanings and the Web∗ Rakesh Verma Computer Science Dept. University of Houston Houston, TX, 77204, USA rverma@uh.edu Vasanthi Vuppuluri Computer Science Dept. University of Houston Houston, TX, 77204, USA vvuppuluri@uh.edu Abstract There is a great deal of knowledge avail- able on the Web, which represents a great opportunity for automatic, intelligent text processing and understanding, but the ma- jor problems are finding the legitimate sources of information and the fact that search engines provide page statistics not occurrences. This paper presents a new, domain independent, general-purpose id- iom identification approach. Our approach combines the knowledge of the Web with the knowledge extracted from dictionaries. This method can overcome the limitations of current techniques that rely on linguis- tic knowledge or statistics. It can recog- nize idioms even when the complete sen- tence is not present, and without the need for domain knowledge. It is currently de- signed to work with text in English but can be extended to other languages. 1 Introduction Automatically extracting phrases from the doc- uments, be they structured, un-structured or semistructured has always been an important yet challenging task. The overall goal is to create a easily machine-readable text to process the sen- tences. In this paper we focus on identifying id- ioms from text. An idiom is a phrase made up of a sequence of two or more words that has prop- erties that are not predictable from the properties of the individual words or their normal mode of combination. Recognition of idioms is a challeng- ing problem with wide applications. Some exam- ples of idioms are ‘yellow journalism,’ ‘kick the bucket,’ and ‘quick fix’. For example, the mean- ing of ‘yellow journalism’ cannot be derived from the meanings of ‘yellow’ and ‘journalism.’ ∗Research supported in part by NSF grants CNS 1319212, DUE 1241772 and DGE1433817 Idioms play an important role in Natural Lan- guage Processing (NLP). They exist in almost all languages and are hard to extract as there is no al- gorithm that can precisely outline the structure of an idiom. Idioms are important for natural lan- guage generation, parsing, and significantly influ- ence machine translation and semantic tagging. Idioms could be also useful in document index- ing, information retrieval, and in text summariza- tion or question-answering approaches that rely on extracting key words or phrases from the docu- ment to be summarized, e.g., (Barrera and Verma, 2011; Barrera and Verma, 2012; Barrera et al., 2011). Efficiently extracting idioms significantly improves many areas of NLP. But most of the idiom extraction techniques are biased in a way that they focus on a specific domain or make use of statistical techniques alone, which results in poor performance. The technique in this paper makes use of knowledge from the Web combined with knowledge from dictionaries in deciding if a phrase is a idiom rather than solely depending on frequency measures or following rules …

Content truncated to 500 words