A Lexicon-Based Approach for Detecting Hedges in Informal Text

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3109–3113 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3109 A Lexicon-Based Approach for Detecting Hedges in Informal Text Jumayel Islam1, Lu Xiao2, Robert E. Mercer1 1Department of Computer Science, The University of Western Ontario 2School of Information Studies, Syracuse University jislam3@uwo.ca, lxiao04@syr.edu, mercer@csd.uwo.ca Abstract Hedging is a commonly used strategy in conversational management to show the speaker’s lack of commitment to what they com- municate, which may signal problems between the speakers. Our project is interested in examining the presence of hedging words and phrases in identifying the tension between an interviewer and interviewee during a survivor interview. While there have been studies on hedging detection in the natural language processing literature, all existing work has focused on structured texts and formal communications. Our project thus investigated a corpus of eight unstructured conversational interviews about the Rwanda Genocide and identified hedging patterns in the interviewees’ responses. Our work produced three manually constructed lists of hedge words, booster words, and hedging phrases. Leveraging these lexicons, we developed a rule-based algorithm that detects sentence-level hedges in informal conversations such as survivor interviews. Our work also produced a dataset of 3000 sentences having the categories Hedge and Non-hedge annotated by three researchers. With experiments on this annotated dataset, we verify the efficacy of our proposed algorithm. Our work contributes to the further development of tools that identify hedges from informal conversations and discussions. Keywords: Hedging, Informal conversation, Discourse Markers 1. Introduction People use hedging when they try to avoid criticism or evade questions in conversations (Crystal, 1988). Using hedging gives interviewees an opportunity to organize their thoughts and make a suitable response in a one-to-one in- terview. This is frequent when there is a disjuncture be- tween the interviewer and the interviewee. Layman (2009) showed how interviewees employ such strategies during an oral history interview to avoid answering sensitive ques- tions. Most of the time, in such cases, their responses are dismissive and filled with hedging. This leads to interview- ers’ judgment whether to press the interviewee when it be- comes evident that the interviewee is reluctant to answer certain questions. Thus, it is important to analyze such phenomenon and build tools that can identify hedging in interview transcripts which will help researchers to under- stand the dynamics of such interviews and give them more control of the situation. The following two examples from a conversational interview transcript demonstrate the use of hedging for these purposes: (1) I think that every survivor’s story must be heard in the singularity of experience that it re- counts. (2) I don’t know if I would address the young sur- vivors specifically. The use of hedge terms “I think”, “I don’t know” and “would” demonstrates the instability in their narrative. Besides hedge words, people use discourse markers to hedge in conversations. These can be an utterance or a word or a phrase (such as “oh”, “like”, “well”, and …

Content truncated to 500 words