A Regularization-based Framework for Bilingual Grammar Induction

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1423–1428, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1423 A Regularization-based Framework for Bilingual Grammar Induction∗ Yong Jiang⋄, Wenjuan Han†, Kewei Tu† †School of Information Science and Technology, ShanghaiTech University, Shanghai, China ⋄Alibaba Group yongjiang.jy@alibaba-inc.com {hanwj,tukw}@shanghaitech.edu.cn Abstract Grammar induction aims to discover syntac- tic structures from unannotated sentences. In this paper, we propose a framework in which the learning process of the grammar model of one language is influenced by knowledge from the model of another language. Unlike previ- ous work on multilingual grammar induction, our approach does not rely on any external re- sources, such as parallel corpora, word align- ments or linguistic phylogenetic trees. We pro- pose three regularization methods that encour- age similarity between model parameters, de- pendency edge scores, and parse trees respec- tively. We deploy our methods on a state-of- the-art unsupervised discriminative parser and evaluate it on both transfer grammar induction and bilingual grammar induction. Empirical results on multiple languages show that our methods outperform strong baselines. 1 Introduction Syntactic parsing is an important task in natural language processing. Supervised parsing requires manual labeling of gold parse trees, which is a very labor-intensive task. On the other hand, unsu- pervised parsing (a.k.a. grammar induction) does not require labeled data and can make use of large amounts of unlabeled data that are freely available. However, grammar induction is very challenging and its accuracy is still far below that of supervised parsing. To compensate the lack of supervision in grammar induction, some previous work consid- ers multilingual grammar induction, i.e., simulta- neously learning grammars of multiple languages (Snyder et al., 2009; Berg-Kirkpatrick and Klein, 2010; Liu et al., 2013). Existing multilingual ap- proaches require external resources such as paral- lel corpora, word alignments, and linguistic phy- logenetic trees. ∗ Yong Jiang contributed to this work when at Shang- haiTech University. Kewei Tu is the corresponding author. Language German English Spanish French Indonesian Code DE EN ES FR ID C-MST 60.2 62.3 68.8 72.3 69.7 D-Tran 59.9 – 65.3 67.8 45.7 ∆ -0.3 – -3.5 -4.5 -24.0 Language Italian Japanese Korean Portuguese Swedish Code IT JA KO PTBR SV C-MST 64.3 57.5 59.0 68.3 66.2 D-Tran 63.1 54.6 50.0 66.2 67.8 ∆ -1.2 -2.9 -9.0 -2.1 +1.6 Table 1: Directed dependency accuracy (DDA) on the universal treebanks with universal POS tags, on sen- tences of length ≤ 10. C-MST denotes the original Convex-MST model. D-Tran denotes direct transfer. ∆ refers to the difference between C-MST and D-Tran. In this paper, we aim at bilingual grammar in- duction without external resource. We are moti- vated by our observation that learning the unsu- pervised Convex-MST model (Grave and Elhadad, 2015) on the English corpus and then directly ap- plying it to parse other languages produces sur- prisingly good results (Table 1). From the table, we can see that even with this simplistic …

Content truncated to 500 words