Bilingual Synonym Identification with Spelling Variations

Bilingual Synonym Identification with Spelling Variations Takashi Tsunakawa∗ Jun’ichi Tsujii∗†‡ ∗Department of Computer Science, Graduate School of Information Science and Technology, University of Tokyo 7-3-1, Hongo, Bunkyo-ku, Tokyo, 113-0033 Japan †School of Computer Science, University of Manchester Oxford Road, Manchester, M13 9PL, UK ‡National Centre for Text Mining 131 Princess Street, Manchester, M1 7DN, UK {tuna, tsujii}@is.s.u-tokyo.ac.jp Abstract This paper proposes a method for identify- ing synonymous relations in a bilingual lex- icon, which is a set of translation-equivalent term pairs. We train a classifier for identi- fying those synonymous relations by using spelling variations as main clues. We com- pared two approaches: the direct identifi- cation of bilingual synonym pairs, and the merger of two monolingual synonyms. We showed that our approach achieves a high pair-wise precision and recall, and outper- forms the baseline method. 1 Introduction Automatically collecting synonyms from language resources is an ongoing task for natural language processing (NLP). Most NLP systems have diffi- culties in dealing with synonyms, which are differ- ent representations that have the same meaning in a language. Information retrieval (IR) could lever- age synonyms to improve the coverage of search re- sults (Qiu and Frei, 1993). For example, when we input the query ‘transportation in India’ into an IR system, the system can expand the query to its syn- onyms; e.g. ‘transport’ and ‘railway’, to find more documents. This paper proposes a method for the automatic identification of bilingual synonyms in a bilingual lexicon, with spelling variation clues. A bilingual synonym set is a set of translation-equivalent term pairs sharing the same meaning. Although a number of studies have aimed at identifying synonyms, this is the first study that simultaneously finds synonyms in two languages, to our best knowledge. Let us consider the case where a user enters the Japanese query ‘k¯oj¯o’ (工場, industrial plant) into a cross-lingual IR system to find English documents. After translating the query into the English trans- lation equivalent, ‘plant,’ the cross-lingual IR sys- tem may expand the query to its English synonyms, e.g. ‘factory,’ and ‘workshop,’ and retrieve docu- ments that include the expanded terms. However, the term ‘plant’ is ambiguous; the system may also expand the query to ‘vegetable,’ and the system is prevented by the term which is different from our intention. In contrast, the system can easily reject the latter expansion, ‘vegetable,’ if we are aware of bilingual synonyms, which indicate synonymous re- lations over bilingual lexicons: (k¯oj¯o, plant) ∼ (k¯oj¯o, factory) and (shokubutsu1, plant) ∼ (shokubutsu, vegetable)2 (See Figure 1). The expression of the translation equivalent, (k¯oj¯o, plant), helps a cross- lingual IR system to retrieve documents that include the term ‘plant,’ used in the meaning for k¯oj¯o, or in- dustrial plants. We present a supervised machine learning ap- proach for identifying bilingual synonyms. Design- ing features for bilingual synonyms such as spelling variations and bilingual associations, we train a clas- sifier with a manually annotated bilingual lexicon with synonymous information. In order to evaluate the performance of our method, we …

Content truncated to 500 words