Improving the Production Efficiency and Well-formedness of Automatically-Generated Multiple-Choice Cloze Vocabulary Questions
13292Authors
nan
Abstract
AbstractMultiple-choice cloze (fill-in-the-blank) questions are widely used in knowledge testing and are commonly used for testing vocabulary knowledge. Word Quiz Constructor (WQC) is a Java application that is designed to produce such test items automatically from the Academic Word List (Coxhead, 2000) and using various online and offline resources. The present work evaluates recently added features of WQC to see whether they improve the production quality and well-formedness of vocabulary quiz items over previously implemented features in WQC. Results of a production test and a well-formedness survey using Amazon Mechanical Turk show that newly-introduced features (Linsear Write readability formula and Google Books NGrams frequency list) significantly improve the production quality of items over previous features (Automated Readability Index and frequency list derived from the British Academic Written English corpus). Items are produced faster and stem sentences are shorter in length without any degradation in their well-formedness. Approximately 90% of such items are judged well-formed, surpassing the rate of manually-produced items.
Keywords
Full Content
Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 696–701 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 696 Relation between Degree of Empathy for Narrative Speech and Type of Responsive Utterance in Attentive Listening Koichiro Ito†1, Masaki Murata†2, Tomohiro Ohno†3, Shigeki Matsubara†4 †1Graduate School of Informatics, Nagoya University †2National Institute of Technology, Toyota College †3Graduate School of Advanced Science and Technology, Tokyo Denki University †4Information & Communications, Nagoya University †1,4Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan †2Eisei-cho 2-1, Toyota, 471-8525, Japan †3Senjuasahi-cho 5, Adachi-ku, Tokyo,120-8551, Japan ito.koichiro@a.mbox.nagoya-u.ac.jp, murata@toyota-ct.ac.jp, ohno@mail.dendai.ac.jp, matubara@nagoya-u.jp Abstract Nowadays, spoken dialogue agents such as communication robots and smart speakers listen to narratives of humans. In order for such an agent to be recognized as a listener of narratives and convey the attitude of attentive listening, it is necessary to generate responsive utterances. Moreover, responsive utterances can express empathy to narratives and showing an appropriate degree of empathy to narratives is significant for enhancing speaker’s motivation. The degree of empathy shown by responsive utterances is thought to depend on their type. However, the relation between responsive utterances and degrees of the empathy has not been explored yet. This paper describes the classification of responsive utterances based on the degree of empathy in order to explain that relation. In this research, responsive utterances are classified into five levels based on the effect of utterances and literature on attentive listening. Quantitative evaluations using 37,995 responsive utterances showed the appropriateness of the proposed classification. Keywords: communication robot, narrative, spoken dialogue, spoken language resource 1. Introduction To narrate is a fundamental need of human beings. The act of narrating is established only when there is a listener. In the modern society, there are more and more situations when there is no listener and an important issue is how to increase opportunities for people to narrate. To solve this issue, spoken dialogue agents such as com- munication robots and smart speakers can play the role of listeners of narratives. In order for such an agent to be rec- ognized as a listener of narratives, it is necessary to con- vey that it attentively listens to the speaker’s narrative. An effective explicit means of realizing this function is to re- spond to narratives, by generating gestures or utterances. Hereinafter, an utterance used for this function, that is an utterance that responds to the narratives for the purpose of showing an attitude of attentive listening, is simply called attentive listening response. Attentive listening responses show empathy to narrative speech and enhance speaker’s motivation to speak. A rep- resentative attentive listening response is a back-channel, whose generation methods have already been proposed (Kamiya et al., 2010; Yamaguchi et al., 2016). Besides the back-channel, there are various types of attentive lis- tening responses such as admiration response or evaluation response, and there are some works on generating such re- sponses (Kobayashi et al., 2010; Lala et al., 2017; Meguro et al., 2011; Shitaoka et al., 2017). The degree of …
Content truncated to 500 words
Labeled Text
AbstractMultiple-choice cloze (fill-in-the-blank) questions are widely used in knowledge testing and are commonly used for testing vocabulary knowledge. Word Quiz Constructor (WQC) is a Java application that is designed to produce such test items automatically from the Academic Word List (Coxhead, 2000) and using various online and offline resources. The present work evaluates recently added features of WQC to see whether they improve the production quality and well-formedness of vocabulary quiz items over previously implemented features in WQC. Results of a production test and a well-formedness survey using Amazon Mechanical Turk show that newly-introduced features (Linsear Write readability formula and Google Books NGrams frequency list) significantly improve the production quality of items over previous features (Automated Readability Index and frequency list derived from the British Academic Written English corpus). Items are produced faster and stem sentences are shorter in length without any degradation in their well-formedness. Approximately 90% of such items are judged well-formed, surpassing the rate of manually-produced items.
Publication Details
- Paper ID:
- 13292
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan