Bilingual Lexicon Construction from Comparable Corpora via Dependency Mapping

Proceedings of COLING 2012: Technical Papers, pages 2275–2290, COLING 2012, Mumbai, December 2012. Bilingual Lexicon Construction from Comparable Corpora via Dependency Mapping QIAN Long Hua WANG Hong Ling ZHOU Guo Dong ZHU Qiao Ming* NLP Lab, School of Computer Science and Technology Soochow University, Suzhou China 215006 {qianlonghua,hlwang,gdzhou,qmzhu}@suda.edu.cn ABSTRACT Bilingual lexicon construction (BLC) from comparable corpora is based on the idea that bilingual similar words tend to occur in similar contexts, usually of words. This, however, introduces noise and leads to low performance. This paper proposes a bilingual dependency mapping model for BLC which encodes a word’s context as a combination of its dependent words and their relationships. This combination can provide more reliable clues than mere context words for bilingual translation words. We further demonstrate that this kind of bilingual dependency mappings can be successfully generated and maximally exploited without human intervention. The experiments on BLC from English to Chinese show that, by mapping context words and their dependency relationships simultaneously when calculating the similarity between bilingual words, our approach significantly outperforms a state-of-the-art one by ~14 units in accuracy for frequently occurring noun pairs and similarly, though in a less degree, for nouns and verbs in a wide frequency range. This justifies the effectiveness of our dependency mapping model for BLC. TITLE AND ABSTRACT IN ANOTHER LANGUAGE, CHINESE 应用依存映射从可比较语料库中抽取双语词表 从可比较语料库中抽取双语词表的基本思想是，双语相似的词语出现在相同的语词上下文 中。不过，这种方法引入了噪声，从而导致了低的抽取性能。本文提出了一种用于双语词 表抽取的双语依存映射模型，在该模型中一个词语的上下文结合了依存词语及其依存关 系。这种结合方法为双语词表构建提供了比单一的词语上下文更为可靠的信息。我们还进 一步展示了在没有人工干预的情况下可以产生和利用这种双语依存关系。从英文到中文的 双语词表构建实验表明，通过在计算双语词语相似度时同时映射词语及其依存关系，同目 前性能最好的系统相比，我们的方法显著提高了精度。对于经常出现的名词，精度提高了 14个百分点；对于较大频率范围内的名词和动词，性能也提高了，尽管程度较小。这说明 了依存映射模型对双语词表构建的有效性。 KEYWORDS: Bilingual Lexicon Construction, Comparable Corpora, Dependency Mapping KEYWORDS IN CHINESE: 双语词表构建, 可比较语料库, 依存映射 * Corresponding author 2275 1 Introduction Bilingual lexicons play an important role in many natural language processing tasks, such as machine translation (MT) (Och and Ney, 2003; Gong et al., 2011) and cross-language information retrieval (CLIR) (Grefenstette, 1998). Traditionally, bilingual lexicons are built manually with tremendous efforts. With the availability of large-scale parallel corpora, researchers turn to automatic construction from parallel corpora and achieve certain success (Wu and Xia, 1994). However, large-scale parallel corpora do not always exist for most language pairs. Therefore, researchers turn their attention to either pivot languages or non-parallel but comparable corpora. Using pivot languages in BLC was pioneered by Tanaka and Umemura (1994). Thereafter, various studies have been done to take advantage of multiple paths (Mann and Yarowsky, 2001) and even multiple pivot languages (Mausam et al., 2009) between the source and target languages. Since such automatically constructed lexicons usually contain noisy and polysemous entries, corpus-based occurrence information has been widely used to help rank the candidate target words (Schafer and Yarowsky, 2002; Kaji et al., 2008; Shezaf and Rappoport, 2010). Alternatively, extracting bilingual lexicons from comparable corpora assumes that words with similar meanings in different languages tend to occur in similar contexts, even in non-parallel corpora. Rapp (1999) and Fung (2001) proposed a bilingual context vector mapping strategy to explore word co-occurrence information. Both studies rely on a large, one-to-one mapping seed lexicon between the source and target languages. Koehn and Knight (2002) investigated various clues such as cognates, similar context, preservation of word similarity and word frequency. Garera et …

Content truncated to 500 words