Bilingual Segmented Topic Model

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1266–1276, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Bilingual Segmented Topic Model Akihiro Tamura and Eiichiro Sumita National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, JAPAN {akihiro.tamura, eiichiro.sumita}@nict.go.jp Abstract This study proposes the bilingual seg- mented topic model (BiSTM), which hi- erarchically models documents by treat- ing each document as a set of segments, e.g., sections. While previous bilingual topic models, such as bilingual latent Dirichlet allocation (BiLDA) (Mimno et al., 2009; Ni et al., 2009), consider only cross-lingual alignments between entire documents, the proposed model consid- ers cross-lingual alignments between seg- ments in addition to document-level align- ments and assigns the same topic distri- bution to aligned segments. This study also presents a method for simultane- ously inferring latent topics and segmen- tation boundaries, incorporating unsuper- vised topic segmentation (Du et al., 2013) into BiSTM. Experimental results show that the proposed model significantly out- performs BiLDA in terms of perplexity and demonstrates improved performance in translation pair extraction (up to +0.083 extraction accuracy). 1 Introduction Probabilistic topic models, such as probabilis- tic latent semantic analysis (PLSA) (Hofmann, 1999) and latent Dirichlet allocation (LDA) (Blei et al., 2003), are generative models for documents that have been used as unsupervised frameworks to discover latent topics in document collections without prior knowledge. These topic models were originally applied to monolingual data; how- ever, various recent studies have proposed the use of probabilistic topic models in multilingual set- ��������� ������ ��������� ���������� ��������������� �������������������� ��������������������������� ��������������������������� ������� ��� ������������������� ������ �� ������������������ ���������� ������� �������� ����������������������������������� ������������������������������������� ���� ����������������� ��������� ��������������������� ���������� ������ ������������������ ���� (football) ������: soccer���� ���� �����1�� �11��2 ������������ ����� ���� ������� � ����ciatiall ; �� ������� ������ ������� ������ 1 �� 2 �� (name) 3 �� (game) 4 �� (history) 5 ������� � � Figure 1: Wikipedia Article Example tings1, where latent topics are shared across multi- ple languages. These models have improved sev- eral multilingual tasks, such as translation pair ex- traction and cross-lingual text classification (see the survey paper by Vuli´c et al. (2015) for details). Most multilingual topic models, including bilin- gual LDA (BiLDA) (Mimno et al., 2009; Ni et al., 2009), model a document-aligned comparable corpus, such as a collection of Wikipedia articles, where aligned documents are topically similar but are not direct translations2. In particular, these models assume that the documents in each tuple share the same topic distribution and that each cross-lingual topic has a language-specific word distribution. Existing multilingual topic models consider only document-level alignments. However, most documents are hierarchically structured, i.e., a document comprises segments (e.g., sections and paragraphs) that can be aligned across languages. Figure 1 shows a Wikipedia article example, which contains a set of sections. Sections 1, 2, and 3 in the English article correspond topically to sections 4, 2, and 3 in the Japanese counterpart, re- 1In this work, we deal with a bilingual setting, but our …

Content truncated to 500 words