DP: A Detector for Presuppositions in survey questions

DP: A Detector for Presuppositions in survey questions Katja WIEMER-HASTINGS Psychology Department / Institute for Intelligent Systems University of Memphis Memphis, TN 38152 kwiemer @ latte.memphis.edu Peter WIEMER-HASTINGS Human Communication Research Centre University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW, UK peterwh@cogsci.ed.ac.uk Sonya RAJAN, Art GRAESSER, Roger KREUZ, & Ashish KARNAVAT Institute for Intelligent Systems, University of Memphis, Memphis, TN 38152 sonyarajan@hotmail.com, graesser@memphis.edu, rkreuz@memphis.edu, akarnavat@hotmail.com Abstract This paper describes and evaluates a detector of presuppositions (DP) for survey questions. Incorrect presuppositions can make it difficult to answer a question correctly. Since they can be difficult to detect, DP is a useful tool for questionnaire designer. DP performs well using local characteristics of presuppositions. It reports the presupposition to the survey methodologist who can determine whether the presupposition is valid. Introduction Presuppositions are propositions that take some information as given, or as "the logical assumptions underlying utterances" (Dijkstra & de Smedt, 1996, p. 255; for a general overview, see McCawley, 1981). Presupposed information includes state of affairs, such as being married; events., such as a graduation; possessions, such as a house, children, knowledge about something; and others. For example, the question, "when did you graduate from college", presupposes the event that the respondent did in fact graduate from college. The answer options may be ranges of years, such as "between 1970 and 1980". Someone who has never attended college can either not respond at all, or give a random (and false) reply. Thus, incorrect presuppositions cause two problems. First, the question is difficult to answer. Second, assuming that people feel obliged to answer them anyway, their answers present false information. This biases survey statistics, or, in an extreme case, makes them useless. The detector for presuppositions (DP) is part of the computer tool QUAID (Graesser, Wiemer- Hastings, Kreuz, Wiemer-Hastings & Marquis, in press), which helps survey methodologists design questions that are easy to process. DP detects a presupposition and reports it to the survey methodologist, who can examine if the presupposition is correct. QUAID is a computerized QUEST questionnaire evaluation aid. It is based on QUEST (Graesser & Franklin, 1990), a computational model of the cognitive processes underlying human question answering. QUAID critiques questions with respect to unfamiliar technical terms, vague terms, working memory overload, complex syntax, incorrect presuppositions, and unclear question purpose or category. These problems are a subset of potential problems that have been identified by Graesser, Bommareddy, Swamer, and Golding (1996; see also Graesser, Kennedy, Wiemer-Hastings & Ottati, 1999). QUAID performs reliably on the first five problem categories. In comparison to these five problems, presupposition detection is even more challenging. For unfamiliar technical terms, for example, QUAID reports words with frequencies below a certain threshold. Such an elegant solution is impossible for presuppositions. Their forms vary widely across presupposition types. Therefore, their detection requires a complex set of rules, carefully tuned to identify a variety of presupposition problems. DP prints out the 90 presuppositions of a question, and relies on the survey methodologist to make the final …

Content truncated to 500 words