Translation and Extension of Concepts Across Languages

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 175–183, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Translation and Extension of Concepts Across Languages Dmitry Davidov ICNC The Hebrew University of Jerusalem dmitry@alice.nc.huji.ac.il Ari Rappoport Institute of Computer Science The Hebrew University of Jerusalem arir@cs.huji.ac.il Abstract We present a method which, given a few words defining a concept in some lan- guage, retrieves, disambiguates and ex- tends corresponding terms that define a similar concept in another specified lan- guage. This can be very useful for cross-lingual information retrieval and the preparation of multi-lingual lexical re- sources. We automatically obtain term translations from multilingual dictionaries and disambiguate them using web counts. We then retrieve web snippets with co- occurring translations, and discover ad- ditional concept terms from these snip- pets. Our term discovery is based on co- appearance of similar words in symmetric patterns. We evaluate our method on a set of language pairs involving 45 languages, including combinations of very dissimilar ones such as Russian, Chinese, and He- brew for various concepts. We assess the quality of the retrieved sets using both hu- man judgments and automatically compar- ing the obtained categories to correspond- ing English WordNet synsets. 1 Introduction Numerous NLP tasks utilize lexical databases that incorporate concepts (or word categories): sets of terms that share a significant aspect of their meanings (e.g., terms denoting types of food, tool names, etc). These sets are useful by themselves for improvement of thesauri and dictionaries, and they are also utilized in various applications in- cluding textual entailment and question answer- ing. Manual development of lexical databases is labor intensive, error prone, and susceptible to arbitrary human decisions. While databases like WordNet (WN) are invaluable for NLP, for some applications any offline resource would not be ex- tensive enough. Frequently, an application re- quires data on some very specific topic or on very recent news-related events. In these cases even huge and ever-growing resources like Wikipedia may provide insufficient coverage. Hence appli- cations turn to Web-based on-demand queries to obtain the desired data. The majority of web pages are written in En- glish and a few other salient languages, hence most of the web-based information retrieval stud- ies are done on these languages. However, due to the substantial growth of the multilingual web1, queries can be performed and the required infor- mation can be found in less common languages, while the query language frequently does not match the language of available information. Thus, if we are looking for information about some lexical category where terms are given in a relatively uncommon language such as Hebrew, it is likely to find more detailed information and more category instances in a salient language such as English. To obtain such information, we need to discover a word list that represents the desired category in English. This list can be used, for in- stance, in subsequent focused search in order to obtain pages relevant for the …

Content truncated to 500 words