Rule Augmented Unsupervised Constituency Parsing

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4923–4932 August 1–6, 2021. ©2021 Association for Computational Linguistics 4923 Rule Augmented Unsupervised Constituency Parsing Atul Sahay 1∗, Anshul Nasery 1∗, Ayush Maheshwari 1, Ganesh Ramakrishnan 1 , Rishabh Iyer 2 1Department of CSE, IIT Bombay, India 2 The University of Texas at Dallas {atulsahay, anshuln, ayusham, ganesh}@cse.iitb.ac.in rishabh.iyer@utdallas.edu Abstract Recently, unsupervised parsing of syntactic trees has gained considerable attention. prototypical approach to such unsupervisedA parsing employs reinforcement learning and auto-encoders. However, no mechanism en- sures that the learnt model leverages the well- understood language grammar. We propose an approach that utilizes very generic linguistic knowledge of the language present in the form of syntactic rules, thus inducing better syntac- tic structures. We introduce a novel formula- tion that takes advantage of the syntactic gram- mar rules and is independent of the base sys- tem. We achieve new state-of-the-art results on two benchmarks datasets, MNLI and WSJ.1 1 Introduction Syntactic parse trees have demonstrated their im- portance in several downstream NLP applications such as machine translation (Eriguchi et al., 2017; Zaremoodi and Haffari, 2018), natural language inference (NLI) (Choi et al., 2018), relation ex- traction (Gamallo et al., 2012) and text classifica- tion (Tai et al., 2015). Based on linguistic theories that have promoted the usefulness of tree-based representation of natural language text, tree-based models such as Tree-LSTM have been proposed to learn sentence representations (Socher et al., 2011). Inspired by the Tree-LSTM based mod- els, many approaches were proposed do not require parse tree supervision (Yogatama et al., 2017; Choi et al., 2018; Maillard et al., 2019; Drozdov et al., 2019). However, (Williams et al., 2018; Sahay et al., 2021) have shown that these methods can- not learn meaningful semantics (not even simple grammar), though they perform well on NLI tasks. Recently, there has been surge in approaches using weak supervision in the form of rules for various ∗Equal contribution 1The source code of the paper is available at https://github.com/anshuln/Diora with rules. tasks such as sequence classification (Safranchik et al., 2020), text classification (Chatterjee et al., 2020; Maheshwari et al., 2020), etc. These ap- proaches have demonstrated the importance of ex- ternal knowledge in both unsupervised and super- vised setup. To the best of our knowledge, previous works on syntactic parse tree has not utilized such external information. In this paper, we propose an approach that leverages linguistic (and potentially domain agnostic) knowledge in the form of explicit syntactic grammar rules while building upon a state of the art, deep and unsupervised inside-outside recursive autoencoder (DIORA; (Drozdov et al., 2019)). DIORA is an unsupervised model that uses inside-outside dynamic programming to compose latent representations from all possible binary trees. We extend DIORA and propose a framework that harness grammar rules to learn constituent parse trees. We use context free grammar (CFG) produc- tions for English language (like NP →VP NP, PP →IN NP, etc) as rules. Note that the construction of such a rule set is a one time …

Content truncated to 500 words