Filler-gaps that neural networks fail to generalize

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 26–40 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 26 Neural Proof Nets Konstantinos Kogkalidis and Michael Moortgat and Richard Moot Utrecht Institute of Linguistics OTS, Utrecht University LIRMM, Université de Montpellier, CNRS {k.kogkalidis,m.j.moortgat}@uu.nl, richard.moot@lirmm.fr Abstract Linear logic and the linear λ-calculus have a long standing tradition in the study of natu- ral language form and meaning. Among the proof calculi of linear logic, proof nets are of particular interest, offering an attractive geo- metric representation of derivations that is un- burdened by the bureaucratic complications of conventional prooftheoretic formats. Build- ing on recent advances in set-theoretic learn- ing, we propose a neural variant of proof nets based on Sinkhorn networks, which allows us to translate parsing as the problem of extract- ing syntactic primitives and permuting them into alignment. Our methodology induces a batch-efficient, end-to-end differentiable archi- tecture that actualizes a formally grounded yet highly efficient neuro-symbolic parser. We test our approach on Æthel, a dataset of type- logical derivations for written Dutch, where it manages to correctly transcribe raw text sen- tences into proofs and terms of the linear λ- calculus with an accuracy of as high as 70%. 1 Introduction There is a broad consensus among grammar for- malisms that the composition of form and meaning in natural language is a resource-sensitive process, with the words making up a phrase contributing exactly once to the resulting whole. The sentence “the Mad Hatter offered” is ill-formed because of a lack of grammatical material, “offer” being a di- transitive verb; “the Cheshire Cat grinned Alice a cup of tea” on the other hand is ill-formed because of an excess of material, which the intransitive verb “grin” cannot accommodate. Given the resource-sensitive nature of language, it comes as no surprise that Linear Logic (Gi- rard, 1987), in particular its intuitionistic version ILL, plays a central role in current logic-based grammar formalisms. Abstract Categorial Gram- mars and Lambda Grammars (de Groote, 2001; Muskens, 2001) use ILL “as-is” to characterize an abstract level of grammatical structure from which surface form and semantic interpretation are obtained by means of compositional transla- tions. Modern typelogical grammars in the tra- dition of the Lambek Calculus (Lambek, 1958), e.g. Multimodal TLG (Moortgat, 1996), Displace- ment Calculus (Morrill, 2014), Hybrid TLG (Kub- ota and Levine, 2020), refine the type language to account for syntactic aspects of word order and con- stituency; ILL here is the target logic for semantic interpretation, reached by a homomorphism relat- ing types and derivations of the syntactic calculus to their semantic counterparts. A common feature of the aforementioned for- malisms is their adoption of the parsing-as- deduction method: determining whether a phrase is syntactically well-formed is seen as the outcome of a process of logical deduction. This logical deduction automatically gives rise to a program for meaning composition, thanks to the remark- able correspondence between logical proof and computation known as the Curry-Howard isomor- phism (Sørensen and Urzyczyn, 2006), …

Content truncated to 500 words