Finding Concept-specific Biases in Form–Meaning Associations

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380–396 June 6–11, 2021. ©2021 Association for Computational Linguistics 380 Aspect-Controlled Neural Argument Generation Benjamin Schiller and Johannes Daxenberger and Iryna Gurevych Ubiquitous Knowledge Processing Lab Department of Computer Science, Technical University of Darmstadt http://www.ukp.tu-darmstadt.de/ Abstract We rely on arguments in our daily lives to de- liver our opinions and base them on evidence, making them more convincing in turn. How- ever, finding and formulating arguments can be challenging. In this work, we present the Arg-CTRL—a language model for argument generation that can be controlled to gener- ate sentence-level arguments for a given topic, stance, and aspect. We define argument as- pect detection as a necessary method to allow this fine-granular control and crowdsource a dataset with 5,032 arguments annotated with aspects. Our evaluation shows that the Arg- CTRL is able to generate high-quality, aspect- specific arguments, applicable to automatic counter-argument generation. We publish the model weights and all datasets and code to train the Arg-CTRL.1 1 Introduction Language models (Bengio et al., 2003) allow to generate text through learned distributions of a lan- guage and have been applied to a variety of areas like machine translation (Bahdanau et al., 2015), summarization (Paulus et al., 2018), or dialogue systems (Wen et al., 2017). A rather new field for these models is the task of producing text with ar- gumentative content (Wang and Ling, 2016). We believe this technology can support humans in the challenging task of finding and formulating argu- ments. A politician might use this to prepare for a debate with a political opponent or for a press conference. It may be used to support students in writing argumentative essays or to enrich one-sided discussions with counter-arguments. In contrast to retrieval methods, generation allows to combine and stylistically adapt text (e.g. arguments) based on a given input (usually the beginning of a sen- tence). Current argument generation models, how- ever, produce lengthy texts and allow the user little 1https://github.com/UKPLab/ controlled-argument-generation control over the aspect the argument should address (Hua et al., 2019; Hua and Wang, 2018). We show that argument generation can be enhanced by al- lowing for a fine-grained control and limiting the argument to a single but concise sentence. Controllable language models like the CTRL (Keskar et al., 2019) allow to condition the model at training time to certain control codes. At infer- ence, these can be used to direct the model’s output with regard to content or style. We build upon this architecture to control argument generation based solely on a given topic, stance, and argument as- pect. For instance, to enforce focus on the aspect of cancer for the topic of nuclear energy, we in- put a control code “Nuclear Energy CON cancer” that creates a contra argument discussing this as- pect, for instance: “Studies show that people living next to nuclear power plants have a higher risk of developing cancer.”. To obtain control codes …

Content truncated to 500 words