Improved Latent Tree Induction with Distant Supervision via Span Constraints

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4818–4831 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4818 Improved Latent Tree Induction with Distant Supervision via Span Constraints Zhiyang Xu⇤, Andrew Drozdov⇤, Jay Yoon Lee, Tim O’Gorman, Subendhu Rongali, Dylan Finkbeiner, Shilpa Suresh, Mohit Iyyer, and Andrew McCallum College of Information and Computer Sciences University of Massachusetts Amherst {zhiyangxu, adrozdov, jaylee, togorman, srongali, dfinkbeiner, ssuresh, miyyer, mccallum}@cs.umass.edu Abstract For over thirty years, researchers have devel- oped and analyzed methods for latent tree in- duction as an approach for unsupervised syn- tactic parsing. Nonetheless, modern systems still do not perform well enough compared to their supervised counterparts to have any prac- tical use as structural annotation of text. In this work, we present a technique that uses distant supervision in the form of span con- straints (i.e. phrase bracketing) to improve per- formance in unsupervised constituency pars- ing. Using a relatively small number of span constraints we can substantially improve the output from DIORA, an already competitive unsupervised parsing system. Compared with full parse tree annotation, span constraints can be acquired with minimal effort, such as with a lexicon derived from Wikipedia, to find ex- act text matches. Our experiments show span constraints based on entities improves con- stituency parsing on English WSJ Penn Tree- bank by more than 5 F1. Furthermore, our method extends to any domain where span con- straints are easily attainable, and as a case study we demonstrate its effectiveness by pars- ing biomedical text from the CRAFT dataset. 1 Introduction Syntactic parse trees are helpful for various down- stream tasks such as speech recognition (Moore et al., 1995), machine translation (Akoury et al., 2019), paraphrase generation (Iyyer et al., 2018), semantic parsing (Xu et al., 2020), and informa- tion extraction (Naradowsky, 2014). While super- vised syntactic parsers are state-of-the-art models for creating these parse trees, their performance does not transfer well across domains. Moreover, new syntactic annotations are prohibitively expen- sive; the original Penn Treebank required eight years of annotation (Taylor et al., 2003), and ex- panding PTB annotation to a new domain can be ⇤ Equal contribution. the driver heading to evermore falls Span Constraint the driver heading to evermore falls ✓ Distant Supervision ❌ Unsupervised Learning Figure 1: An example sentence and parsing to illustrate distant supervision via span constraints. Top: The un- supervised parser predicts a parse tree, but due to nat- ural ambiguity in the text the prediction crosses with a known constraint. Bottom: By incorporating the span constraint, the prediction improves and, as a result, re- covers the ground truth parse tree. In our experiments, we both inject span constraints directly into parse tree decoding and separately use the constraints only for dis- tant supervision at training time. We find the latter ap- proach is typically more effective. a large endeavor. For example, the 20k sentences of biomedical treebanking in the CRAFT corpus required 80 annotator hours per week for 2.5 years, include 6 months for …

Content truncated to 500 words