MIPA: Mutual Information Based Paraphrase Acquisition via Bilingual Pivoting

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 80–89, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP MIPA: Mutual Information Based Paraphrase Acquisition via Bilingual Pivoting Tomoyuki Kajiwara∗ Mamoru Komachi† Daichi Mochihashi‡ ∗Tokyo Metropolitan University, Tokyo, Japan, kajiwara-tomoyuki@ed.tmu.ac.jp †Tokyo Metropolitan University, Tokyo, Japan, komachi@tmu.ac.jp ‡The Institute of Statistical Mathematics, Tokyo, Japan, daichi@ism.ac.jp Abstract We present a pointwise mutual informa- tion (PMI) based approach for formaliz- ing paraphrasability and propose a variant of PMI, called mutual information based paraphrase acquisition (MIPA), for para- phrase acquisition. Our paraphrase acqui- sition method first acquires lexical para- phrase pairs by bilingual pivoting and then reranks them by PMI and distributional similarity. The complementary nature of information from bilingual corpora and from monolingual corpora renders the pro- posed method robust. Experimental re- sults show that the proposed method sub- stantially outperforms bilingual pivoting and distributional similarity themselves in terms of metrics such as mean reciprocal rank, mean average precision, coverage, and Spearman’s correlation. 1 Introduction Paraphrases are useful for flexible language understanding in many NLP applications. For example, the usefulness of the paraphrase database PPDB (Ganitkevitch et al., 2013; Pavlick et al., 2015), a publicly available large- scale resource for lexical paraphrasing, has been reported for tasks such as learning word embeddings (Yu and Dredze, 2014) and semantic textual similarity (Sultan et al., 2015). In PPDB, paraphrase pairs are acquired via word alignment on a bilingual corpus by a process called bilingual pivoting (Bannard and Callison-Burch, 2005). Figure 1 shows an example of English language paraphrase acquisition using the German language as a pivot. Although bilingual pivoting is widely used for paraphrase acquisition, it always includes noise Figure 1: Paraphrase acquisition via bilingual piv- oting (Ganitkevitch et al., 2013). due to unrelated word pairs caused by word align- ment errors on the bilingual corpus. Distribu- tional similarity, another well-known method for paraphrase acquisition, is free of alignment er- rors, but includes noise due to antonym pairs that share the same contexts on the monolingual cor- pus (Mohammad et al., 2013). In this study, we formalize the paraphrasability of paraphrase pairs acquired via bilingual pivoting using pointwise mutual information (PMI) and re- duce the noise by reranking the pairs using dis- tributional similarity. The proposed method ex- tends Local PMI (Evert, 2005), which is a vari- ant of weighted PMI that aims to avoid low- frequency bias in PMI, for paraphrase acquisi- tion. Since bilingual pivoting and distributional similarity have different advantages and disad- vantages, we combine them to construct a com- plementary paraphrase acquisition method, called mutual information based paraphrase acquisition (MIPA). Experimental results show that MIPA outperforms bilingual pivoting and distributional similarity themselves in terms of metrics such as mean reciprocal rank (MRR), mean average pre- cision (MAP), coverage, and Spearman’s correla- tion. The contributions of our study are as follows. 80 • Bilingual pivoting-based lexical paraphrase acquisition is generalized using PMI. • Lexical paraphrases are acquired robustly us- ing both bilingual and monolingual corpora. • We …

Content truncated to 500 words