The Curious Case of Control

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11065 - 11076 December 7-11, 2022 ©2022 Association for Computational Linguistics The Curious Case of Control Elias Stengel-Eskin Johns Hopkins University elias@jhu.edu Benjamin Van Durme Johns Hopkins University vandurme@jhu.edu Abstract Children acquiring English make systematic er- rors on subject control sentences even after they have reached near-adult competence (Chom- sky, 1969), possibly due to heuristics based on semantic roles (Maratsos, 1974). Given the advanced fluency of large generative language models, we ask whether model outputs are con- sistent with these heuristics, and to what degree different models are consistent with each other. We find that models can be categorized by be- havior into three separate groups, with broad differences between the groups. The outputs of models in the largest group are consistent with positional heuristics that succeed on subject control but fail on object control. This result is surprising, given that object control is orders of magnitude more frequent in the text data used to train such models. We examine to what de- gree the models are sensitive to prompting with agent-patient information, finding that raising the salience of agent and patient relations re- sults in significant changes in the outputs of most models. Based on this observation, we leverage an existing dataset of semantic proto- role annotations (White et al., 2020) to explore the connections between control and labeling event participants with properties typically as- sociated with agents and patients.1 1 Introduction Normally-developing children learning English struggle with subject control clauses long after they have successfully acquired the components to understand them (Chomsky, 1969; Cromer, 1970; Maratsos, 1974; Sherman and Lust, 1993). A sen- tence with a subject control clause has a matrix (main) clause containing a main verb, an agent, and a patient, and an embedded infinitival clause. For example, in Cole promised Joe to call, the agent is Cole, the patient is Joe, and the embedded clause is 1Code and prompts available at https://github. com/esteng/curious-case-of-control. to call. Crucially, the embedded verb here does not have an overt subject, but rather implicitly refers to a subject in the matrix clause (in this case, Cole). In a subject control clause, this latent subject of the embedded infinitival clause (usually written as PRO) is coindexed with the subject (Cole) rather than the object (Joe) of the matrix (main) clause, i.e. Cole (the agent) is doing the calling. This is typically written: [Cole]NPi promised [Joe]NPj PROi to call (1) where subscripts indicate the noun phrase (NP) “Cole” is the subject of “to call”. (1) can be con- trasted with the more common case of object con- trol; for example, if the matrix verb “promised” is swapped with an object control verb like “told”, then the coreferrent of PRO changes: [Cole]NPi told [Joe]NPj PROj to call (2) Chomsky (1969)2 finds that children ages 5 to 10 regularly misinterpret subject control (1) for object control (2) while correctly interpreting ob- ject control clauses, and proposes that children are following the …

Content truncated to 500 words