An Imitation Learning Approach to Unsupervised Parsing

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3485–3492 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3485 An Imitation Learning Approach to Unsupervised Parsing Bowen Li† Lili Mou‡ Frank Keller† †Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh, UK ‡University of Waterloo, Canada bowen.li@ed.ac.uk, doublepower.mou@gmail.com keller@inf.ed.ac.uk Abstract Recently, there has been an increasing inter- est in unsupervised parsers that optimize se- mantically oriented objectives, typically us- ing reinforcement learning. Unfortunately, the learned trees often do not match actual syn- tax trees well. Shen et al. (2018) propose a structured attention mechanism for language modeling (PRPN), which induces better syn- tactic structures but relies on ad hoc heuristics. Also, their model lacks interpretability as it is not grounded in parsing actions. In our work, we propose an imitation learning approach to unsupervised parsing, where we transfer the syntactic knowledge induced by the PRPN to a Tree-LSTM model with discrete parsing ac- tions. Its policy is then refined by Gumbel- Softmax training towards a semantically ori- ented objective. We evaluate our approach on the All Natural Language Inference dataset and show that it achieves a new state of the art in terms of parsing F-score, outperforming our base models, including the PRPN.1 1 Introduction From a linguistic perspective, a natural language sentence can be thought of as a set of nested con- stituents in the form of a tree structure (Partee et al., 2012). When a parser is trained on la- beled treebanks, the predicted constituency trees are useful for various natural language processing (NLP) tasks, including relation extraction (Verga et al., 2016), text simplification (Narayan and Gar- dent, 2014), and machine translation (Aharoni and Goldberg, 2017). However, expensive expert an- notations are usually required to create treebanks. Unsupervised parsing (also known as grammar induction or latent tree learning) aims to learn syntactic structures without access to a treebank 1Our code can be found at https://github.com/libowen2121/ Imitation-Learning-for-Unsup-Parsing during training, with potential uses in low resource or out-of-domain scenarios. In early approaches, unsupervised parsers were trained by optimizing the marginal likelihood of sentences (Klein and Manning, 2014). More recent deep learning ap- proaches (Yogatama et al., 2017; Maillard et al., 2017; Choi et al., 2018) obtain latent tree struc- tures by reinforcement learning (RL). Typically, this involves a secondary task, e.g., a language modeling objective or a semantic task. How- ever, Williams et al. (2018a) have pointed out that these methods do not yield linguistically plausible structures, and have low self-agreement when ran- domly initialized multiple times. Recently, Shen et al. (2018) proposed the parsing-reading-predict network (PRPN), which performs language modeling with structured at- tention. The model uses heuristics to induce tree structures from attention scores, and in a repli- cation was found to be the first latent tree model to produce syntactically plausible structures (Htut et al., 2018). Structured attention in the PRPN is formalized as differentiable continuous vari- ables, making the model easy to train. But …

Content truncated to 500 words